Version 23.4.8843

• Establish a Connection
  • Plain
  • SCRAM
  • SCRAM-SHA-512
  • Kerberos
  • SSLCertificate
• More Information

You can use the Apache Kafka connector from the CData Sync application to capture data from Apache Kafka and move it to any supported destination. To do so, you need to add the connector, authenticate to the connector, and complete your connection.

Establish a Connection

To allow Sync to use data from Apache Kafka, you first must establish a connection to Apache Kafka. Follow these steps to connect Apache Kafka to your Sync account:

1. Open the Connections page of the Sync dashboard.

2. Click Add Connection to open the Select Connectors page.

3. Click the Sources tab and locate the Apache Kafka row.

4. Click the Configure Connection icon at the end of that row. If you do not see the Configure Connection icon, you need to add the connector according to the instructions in Connections.

5. Enter connection settings on the Settings tab:

   • Connection Name - Enter a connection name of your choice.

   • Bootstrap Servers - Enter the address of the Apache Kafka bootstrap servers to which you want to connect.

   • Auth Scheme - Select your authentication scheme below to proceed to the relevant section for your scheme. Then fill out the settings as specified in that section.

     • None (This Auth scheme does not require any additional settings. So, you can select this scheme and continue with the settings and steps below.)
     • Plain
     • SCRAM (for SCRAM-SHA-256)
     • SCRAM-SHA-512
     • Kerberos
     • SSLCertificate

   • Type Detection Scheme - Select a detection-scheme type to specify how Sync scans data to determine the fields and data types for the bucket.

     • None - Returns all columns as string type.

     • RowScan - Scans rows in order to determine the data type heuristically.

     • SchemaRegistry - Determines the Schema Registry API and uses a list of predefined AVRO schemas. For this scheme type, specify the following properties:

       • Registry Url - Enter the URL to the server for the schema registry. When you specify this property, Sync reads the Apache Avro schema from the server.

       • Registry Service - Select the Schema Registry service that you want to use for working with topic schemas.

         • Confluent

         • AWSGlue

       • Registry Auth Scheme - Select the scheme that you want to use to authenticate to the schema registry.

         • Auto - This scheme enables Sync to choose the scheme automatically based on the other connection properties that you set.

         • None - This scheme specifies that no authentication will be used.

         • Basic - When you select this scheme, you must specify values for Registry User and Registry Password.

           • Registry User - Enter the username that you use to authenticate to the server that is specified in the registry URL.

           • Registry Password - Enter the password that you use to authenticate to the server that is specified in the registry URL.

         • SSLCertificate - This scheme specifies that SSL client-certificate authentication should be used.

       • Registry Type (optional) - Select a registry type for the specific topic:

         • Auto (When you select Auto, Sync attempts to detect the valid schema registry type for the selected topic.)

         • JSON

         • AVRO

     • MessageOnly - Pushes all information as a single aggregate value on a column named Message.

   • Use SSL (optional) - Specify whether you want to use the Secure Sockets Layer (SSL) protocol. The default value is False.

6. Click Create & Test to create the connection.

7. Define advanced connection settings on the Advanced tab. (In most cases, though, you should not need these settings.)

Plain

1. For the Plain scheme, specify these settings:

   • User - Enter the username that you use to authenticate to your Kafka account.

   • Password - Enter the password that you use to authenticate to your Kafka account.

2. Complete the remaining steps.

SCRAM

1. For the SCRAM scheme, specify these settings:

   • User - Enter the username that you use to authenticate to your Kafka account.

   • Password - Enter the password that you use to authenticate to your Kafka account.

2. Complete the remaining steps.

SCRAM-SHA-512

1. For the SCRAM-SHA-512 scheme, specify these settings:

   • User - Enter the username that you use to authenticate to your Kafka account.

   • Password - Enter the password that you use to authenticate to your Kafka account.

2. Complete the remaining steps.

Kerberos

1. For the Kerberos scheme, specify these settings:

   • User - Enter the username that you use to authenticate to your Kafka account.

   • Password - Enter the password that you use to authenticate to your Kafka account.

   • Kerberos SPN - Enter the service principal name (SPN) for the Kerberos domain controller.

   • Kerberos Keytab File (optional) - Enter the keytab file that contains your pairs of the Kerberos principals and encrypted keys.

   • Use Kerberos Ticket Cache (optional) - Select True to use a ticket cache with the logged-in user instead of a keytab file. The default value is False.

2. Complete the remaining steps.

SSLCertificate

1. For the SSLCertificate scheme, specify these settings:

   • SSL Client Cert - Enter the SSL client certificate that is used to validate to the XXXXXXX broker.

   • SSL Client Cert Type - Select the format of the SSL client certificate that is used to connect to the XXXXXX broker:

     • JKSFILE

     • PFXFILE

     • PEMKEY_FILE

     • PEMKEY_BLOB

   • SSL Client Cert Password - Enter the password that is used to decrypt the SSL client certificate.

2. Complete the remaining steps.

More Information

For more information about interactions between CData Sync and Apache Kafka, see Apache Kafka Connector for CData Sync.