Establishing a Connection
Connecting to Apache Kafka
Set BootstrapServers and the Topic properties to specify the address of your Apache Kafka server, as well as the topic you would like to interact with.
By default, the provider communicates with the data source in PLAINTEXT, which means that all data is sent in the clear. To encrypt communication, you should configure the provider to use SSL encryption. To do this, first set UseSSL to true. Then, Set the SSLCert to the client's certificate/public key, and set the SSLKey property to the client's private key. Additionally, you will also need to specify the trusted CA under the TrustStorePath property, so the appropriate server certificates can be trusted, and the SSLCertPassword if the private key is password protected.
For example: UseSSL=true; TrustStorePath=C:/Keys/KafkaKeys/ca-cert; SSLCert=C:/Keys/KafkaKeys/kafka.client.pem; SSLCertPassword=MyPrivateKeyPass123; SSLKey=C:/Keys/KafkaKeys/kafka.client.key;
Authenticating to Apache Kafka
The Apache Kafka data source supports the following authentication methods: Anonymous, Plain, Scram and Kerberos Authentication.
In some situations, Apache Kafka may be connected to without setting any authentication connection properties. To do so, simply set the AuthScheme to "None", and you are ready to connect.
Authenticate using SASL Plain Authentication
The User and Password properties should be specified. AuthScheme should be set to 'Plain'.
Authenticate using SCRAM login module
The User and Password properties should be specified. AuthScheme should be set to 'Scram'.
Authenticate using Kerberos AuthenticationTo authenticate to Apache Kafka using Kerberos, set the following properties:
- AuthScheme: Set this to KERBEROS
- KerberosServiceName: This should match to the principal name of the kafka brokers. In example the principal is "kafka/kafka1.hostname.com@EXAMPLE.COM", so: KerberosServiceName=kafka