SSIS Components for HDFS

Build 24.0.8963

Establishing a Connection

Enabling SSIS in Visual Studio 2022

If you're using Visual Studio 2022, you will need to install the SQL Server Integration Services Projects extension to use SSIS.

  1. Navigate to Extensions > Manage Extensions.
  2. In the Manage Extensions window's search box, search for "SQL Server Integration Services Projects 2022" and select the extension in the list.
  3. Click Download.
  4. Close Visual Studio and run the downloaded Microsoft.DataTools.IntegrationServices.exe installer. Proceed through the installer with default settings.
  5. Open Visual Studio. There should now be an "Integration Services Project" project template available.

Adding the HDFS Connection Manager

Create a new connection manager as follows:

  1. Create a Visual Studio project with the "Integration Services Project" template.
  2. In the project, right-click within the Connection Managers window and select New Connection from the menu.
  3. In the Description column, select CData HDFS Connection Manager and click Add...
  4. Configure the component as described in the next section.

Alternatively, if you have an existing project and CData HDFS Source or CData HDFS Destination:

  1. Right-click your CData HDFS source or destination component in your data flow
  2. Select Edit... to open an editor window.
  3. Click the New... button next to the Connection manager: dropdown selector to create a connection manager.
  4. Configure the component as described in the next section.

Connecting to HDFS

In order to connect, set the following connection properties:

  • Host: Set this value to the host of your HDFS installation.
  • Port: Set this value to the port of your HDFS installation. Default port: 50070
  • UseSSL: (Optional) Set this value to 'True', to negotiate TLS/SSL connections to the HDFS server. Default: 'False'.

Authenticating to HDFS

There are two authentication methods available for connecting to the HDFS data source, Anonymous Authentication and Negotiate (Kerberos) Authentication.

Anonymous Authentication

In some situations, HDFS may be connected to without any authentication connection properties. To do so, set the AuthScheme to None (default).

Kerberos

When authentication credentials are required, you can use Kerberos. See Using Kerberos for details on how to authenticate with Kerberos.

Copyright (c) 2024 CData Software, Inc. - All rights reserved.
Build 24.0.8963