Connecting to Apache Kafka

Set BootstrapServers and the Topic properties to specify the address of your Apache Kafka server, as well as the topic you would like to interact with.

By default, the adapter communicates with the data source in PLAINTEXT, which means that all data is sent in the clear. To encrypt communication, you should configure the adapter to use SSL encryption. To do this, set UseSSL to true and configure SSLServerCert and SSLServerCertType to load the server certificates. JKS and PEM files are supported certificate stores.

Note that proxy settings like ProxyServer and firewall settings like FirewallServer do not affect the connection to the Apache Kafka broker. Internally the adapter connects to Apache Kafka using the official libraries which do not support proxies. These options are only used when the adapter connects to the schema registry as described in Extracting Metadata From Topics.

Authenticating to Apache Kafka

The Apache Kafka data source supports the following authentication methods:

• Anonymous
• Plain
• Scram
• Kerberos

Anonymous

Certain on-premise deployments of Apache Kafka are able to connect to Apache Kafka without setting any authentication connection properties. To do so, simply set the AuthScheme to "None", and you are ready to connect.

SASL Plain

The User and Password properties should be specified. AuthScheme should be set to Plain.

SCRAM login module

The User and Password properties should be specified. The AuthScheme should be set to 'SCRAM' (for SCRAM-SHA-256) or 'SCRAM-SHA-512'.

SSL client certificates

The SSLClientCert and SSLClientCertType properties should be specified and AuthScheme should be set to SSLCertificate. The JKS certificate format is recommended but both PEM and JKS are supported. Using PEM often requires several conversion steps as Java only supports a subset of the encodings and encryption methods supported by tools like OpenSSL.

Kerberos

To authenticate to Apache Kafka using Kerberos, set the following properties:

• AuthScheme: Set this to KERBEROS.
• KerberosServiceName: This should match to the principal name of the Kafka brokers. For example, the principal is "kafka/kafka1.hostname.com@EXAMPLE.COM", so: KerberosServiceName=kafka.
• KerberosKeytabFile: Set this to The Keytab file absolute path containing your pairs of Kerberos principals and encrypted keys.
• KerberosSPN: Set this to the service and host of the Apache Kafka Kerberos Principal. This will be the value prior to the '@' symbol (for instance, kafka/kafka1.hostname.com) of the principal value (for instance, kafka/kafka1.hostname.com@EXAMPLE.COM).

Use Ticket cache

You can set the UseKerberosTicketCache to TRUE in order to use a ticket cache instead of specifying the keytab file. In that case, the KerberosKeytabFile will be ignored even if it's specified.