MuleSoft Connector for Spark SQL

Build 21.0.7930

Establishing a Connection

This section describes how to connect to Spark SQL from your AnyPoint Studio project. You can use wizards or the XML editor in AnyPoint Studio to define connection properties in a global element. Global elements can be reused across other building blocks in your project.

Create a Connection in the Visual Editor

Complete the following steps to visually add the connector to your project and create a connection:

  1. Create a new Mule Project in Anypoint Studio.
  2. From the palette, drag the connector into your Mule flow.
  3. On the Message Flow canvas, double-click the connector icon to open the connector properties in the Properties pane.
  4. In the General section, click Add and define the Spark SQL connection properties. Note that the user name and Password properties always appear here. The user name must be set as it is used as the key for the connection if connection pooling is enabled.

Create a Connection in the XML Editor

Set connection properties in the <cdata-sparksql:config> element in the root of the project's XML.

Note: The user name attribute must always be included here and serves as the key for the connection if connection pooling is enabled.

  <cdata-sparksql:config name="CData_SparkSQL_Configuration" username="MyName" connectionProperty1="MyProperty" ... doc:name="CData SparkSQL: Configuration"/> 

Connecting to Spark SQL

Specify the following to establish a connection with Spark SQL:

  • Server: Set this to the host name or IP address of the server hosting SparkSQL.
  • Port: Set this to the port for the connection to the SparkSQL instance.
  • TransportMode: The transport mode to use to communicate with the SparkSQL server. Accepted entries are BINARY and HTTP. BINARY is selected by default.

Securing Spark SQL Connections

To enable TLS/SSL in the connector, set UseSSL to True.

Authenticating to Spark SQL

The service may be authenticated to using the PLAIN, LDAP, NOSASL, KERBEROS auth schemes.

Authenticating with PLAIN

To authenticate with PLAIN, set the following connection properties:

  • AuthScheme: Set this to PLAIN.
  • User: Set this to user to login as.
  • Password: Set this to the password of the user.
To authenticate, set User and Password.

Authenticating with LDAP

To authenticate with LDAP, set the following connection properties:

  • AuthScheme: Set this to LDAP.
  • User: Set this to user to login as.
  • Password: Set this to the password of the user.
To authenticate, set User, Password, and AuthScheme.

Authenticating with NOSASL

When using NOSASL, no authentication is performed. Set the following connection properties:

  • AuthScheme: Set this to NOSASL.

Authenticating with Kerberos

Please see Using Kerberos for details on how to authenticate with Kerberos.

Connecting to Databricks

To connect to a Databricks cluster, set the properties as described below. Note: The needed values can be found in your Databricks instance by navigating to 'Clusters', selecting the desired cluster, and selecting the JDBC/ODBC tab under 'Advanced Options'.

  • Server: Set to the Server Hostname of your Databricks cluster.
  • Port: 443
  • TransportMode: HTTP
  • HTTPPath: Set to the HTTP Path of your Databricks cluster.
  • UseSSL: True
  • AuthScheme: PLAIN
  • User: Set this to user to login as
  • Password: Set to your personal access token (value can be obtained by navigating to the User Settings page of your Databricks instance and selecting the Access Tokens tab).

Configure Connector Operations

The connector is an operation-based connector; its operations correspond to SQL statements and expose other functionality such as Schema Discovery. See Using the Connector (Mule 3.8+) for more information on executing SQL to Spark SQL in Mule applications.

Copyright (c) 2021 CData Software, Inc. - All rights reserved.
Build 21.0.7930