Connecting to Spark SQL

Establishing a Connection shows how to authenticate to Spark SQL and configure any necessary connection properties. You can also configure cmdlet capabilities through the available Connection properties, from data modeling to firewall traversal. The Advanced Settings section shows how to set up more advanced configurations and troubleshoot connection errors.

Connecting from PowerShell

The CData Cmdlets PowerShell Module for Spark SQL provides a familiar way to interact with Spark SQL from PowerShell. The cmdlets provide a standard PowerShell interface and an SQL interface to live data. The CData cmdlets enable you to work with Spark SQL using standard PowerShell objects; you can chain the cmdlets to each other or other cmdlets in pipelines. The cmdlets also support PowerShell debug streams.

Data Manipulation with Cmdlets

See Establishing a Connection to learn how to get started with the Connect-SparkSQL cmdlet. You can then pass the SparkSQLConnection object returned to other cmdlets for accessing data:

• Select-SparkSQL
• Add-SparkSQL

Executing SQL from PowerShell

You can execute any SQL query with the Invoke-SparkSQL cmdlet.

Accessing Debug Output from Streams

See Capturing Errors and Logging to obtain the debug output through PowerShell streams.

PowerShell Version Support

The standard cmdlets are supported in PowerShell 2, 3, 4, and 5.

Spark SQL Version Support

The cmdlet leverages Spark Thrift to enable bidirectional SQL access to Spark SQL data. It supports Spark SQL version 1.6 and above.