JDBC Driver for Apache Kafka

Build 25.0.9434

Establishing a Connection

Creating a JDBC Data Source

You can create a JDBC data source to connect from your Java application. Creating a JDBC data source based on the CData JDBC Driver for Apache Kafka consists of three basic steps:

  1. Add the driver JAR file to the classpath. The JAR file is located in the lib subfolder of the installation directory. Note that the .lic file must be located in the same folder as the JAR file.
  2. Provide the driver class. For example: 
    cdata.jdbc.apachekafka.ApacheKafkaDriver
  3. Provide the JDBC URL. For example: 
    jdbc:apachekafka:User=admin;Password=pass;BootStrapServers=localhost:9091

or

jdbc:cdata:apachekafka:User=admin;Password=pass;BootStrapServers=localhost:9091

    You can use the second format above whenever there is a conflict in your application between drivers using the same URL format to ensure you are using the CData driver. The URL must start with either "jdbc:apachekafka:" or "jdbc:cdata:apachekafka:" and can include any of the connection properties in name-value pairs separated with semicolons.

Connecting to Apache Kafka

.NET-based editions rely on the Confluent.Kafka and librdkafka libraries to function. These assemblies are bundled with the installer and automatically installed alongside the driver. If you are using a different installation method, make sure to install Confluent.Kafka 2.10.0 from NuGet along with its dependencies.

To specify the address of your Apache Kafka server, use the BootstrapServers parameter.

By default, the driver communicates with the data source in PLAINTEXT, which means that all data is sent unencrypted. If you want communication to be encrypted:

  1. Configure the driver to use SSL encryption by setting UseSSL to true.
  2. Configure SSLServerCert and SSLServerCertType to load the server certificates.

JKS and PEM files are supported certificate stores.

Note: Proxy settings like ProxyServer and firewall settings like FirewallServer do not affect the connection to the Apache Kafka broker because the driver connects to Apache Kafka interally using the official libraries, which do not support proxies. These options are only used when the driver connects to the schema registry. For details, see Extracting Metadata From Topics.

Authenticating to Apache Kafka

The Apache Kafka data source supports the following authentication methods:

  • Anonymous
  • Plain
  • SCRAM login module
  • SSL client certificate
  • Kerberos

Anonymous

Certain on-premise deployments of Apache Kafka can connect to Apache Kafka without the need to set any authentication connection properties. Such connections are said to be anonymous.

To authenticate anonymously, set this property:

SASL Plain

Plain authentication employs a plain text login module for authentication.

Set these properties:

SCRAM Login Module

To authenticate using a SCRAM login module, set these properties:

  • AuthScheme: Specify SCRAM to use the SCRAM login module with SHA-256 hashing, or SCRAM-SHA-512 to use the SCRAM login module with SHA-512 hashing.
  • User: The authenticating user.
  • Password: The authenticating user's password.

SSL Client Certificate

To authenticate using an SSL client certificate, set these properties:

  • AuthScheme: SSLCertificate.
  • SSLClientCert: The SSL client certificate used to connect to the CData JDBC Driver for Apache Kafka broker.
  • SSLClientCertType: The format of the SSL client certificate used to connect to the CData JDBC Driver for Apache Kafka broker: PEMKEY_FILE (default), JKSFILE, or PMKEY_BLOB.

The JKS certificate format is recommended but both PEM and JKS are supported. Using PEM often requires several conversion steps as Java only supports a subset of the encodings and encryption methods supported by tools like OpenSSL.

Kerberos

Authenticating via Kerberos requires you to specify the system Kerberos configuration file. Set these properties:

Note: If you would rather use specify a ticket cache than a keytab file, set UseKerberosTicketCache to true. In this case, even if the KerberosKeytabFile is specified, it is ignored.

Kafka OpenID Connect

The driver supports a limited form of OAuth provided by the Apache Kafka client libraries. Unlike the authentication schemes connected to Azure Event Hubs and GCP Kakfa, this authentication scheme is not tied to a specific identity provider (IdP). It only requires that your IdP support the client credentials grant type and use JWTs as its access token format. If your IdP meets these requirements, you can register an OAuth application with it for use with the driver.

Note: Both your IdP and your Apache Kafka broker can be configured in a wide variety of ways. You should be aware of the requirements on both ends when connecting with the driver. For further information, refer to the documentation for your Apache Kafka broker and your IdP.

  • Identity Provider: IdPs generally require registering before an application can authenticate. Registering a client credentials application provides you with a client ID, client secret, and a token URL that accepts these values. Your IdP may also require specific scopes to generate an access token that the Apache Kafka broker accepts.
  • Apache Kafka Broker: The Apache Kafka broker supports several options for validating access tokens. The most important of these for the driver are the audience, issuer, and signing key that apply to the access token. If the access token does not have a matching audience or issuer, or is signed with an unrecognized key, the broker rejects the connection. In this situation you should check the broker logs to determine why the validation failed. The broker does not tell the driver why the validation failed so client logs are of limited use.

Set these properties to authenticate:

  • AuthScheme: KafkaOAuthClient.
  • OAuthAccessTokenURL: The URL where your IdP accepts OAuth token requests.
  • OAuthClientId: The client ID of the application you registered with your IdP.
  • OAuthClientSecret: The client secret of the application you registered with your IdP.
  • Scope (optional): If your IdP requires specific scopes to retrieve a valid access token, set them here.

Connecting to Azure Event Hubs

The driver supports connecting to Azure Event Hubs using OAuth and shared-access signatures. Before you begin, check that your Event Hubs namespace supports connections using the Kafka protocol. The driver requires this feature and it may not be available for certain pricing tiers.

In addition to the scheme-specific properties covered below, all connections to Azure require you to set the following properties:

Entra ID (Azure AD)

Note: Microsoft has rebranded Azure AD as Entra ID. In topics that require the user to interact with the Entra ID Admin site, we use the same names Microsoft does. However, there are still CData connection properties whose names or values reference "Azure AD".

Microsoft Entra ID is a multi-tenant, cloud-based identity and access management platform. It supports OAuth-based authentication flows that enable the driver to access Apache Kafka endpoints securely.

Authentication to Entra ID via a web application always requires that you first create and register a custom OAuth application. This enables your application to define its own redirect URI, manage credential scope, and comply with organization-specific security policies.

For full instructions on how to create and register a custom OAuth application, see Creating an Entra ID (Azure AD) Application.

After setting AuthScheme to AzureAD, the steps to authenticate vary, depending on the environment. For details on how to connect from desktop applications, web-based workflows, or headless systems, see the following sections.

Desktop Applications

You can authenticate from a desktop application using either the driver's embedded OAuth application or a custom OAuth application registered in Microsoft Entra ID.

Option 1: Use the Embedded OAuth Application

This is a pre-registered application, included with the driver. It simplifies setup and eliminates the need to register your own credentials and is ideal for development environments, single-user tools, or any setup where quick and easy authentication is preferred.

Set the following connection properties:

  • AuthScheme: AzureAD
  • InitiateOAuth:
    • GETANDREFRESH – Use for the initial login. Launches the login page and saves tokens.
    • REFRESH – Use this setting when you have already obtained valid access and refresh tokens. Reuses stored tokens without prompting the user again.

When you connect, the driver opens the Microsoft Entra sign-in page in your default browser. After signing in and granting access, the driver retrieves the access and refresh tokens and saves them to the path specified by OAuthSettingsLocation.

Option 2: Use a Custom OAuth Application

If your organization requires more control, such as managing security policies, redirect URIs, or application branding, you can instead register a custom OAuth application in Microsoft Entra ID and provide its values during connection.

During registration, record the following values:

  • OAuthClientId: The client Id that was generated when you registered your custom OAuth application.
  • OAuthClientSecret: The client secret that was that was generated when you registered your custom OAuth application.
  • CallbackURL: A redirect URI you defined during application registration.

For full instructions on how to register a custom OAuth application and configure redirect URIs, see Creating an Entra ID (Azure AD) Application.

Set the following connection properties:

  • AuthScheme: AzureAD
  • InitiateOAuth:
    • GETANDREFRESH – Use for the initial login. Launches the login page and saves tokens.
    • REFRESH – Use this setting when you have already obtained valid access and refresh tokens. Reuses stored tokens without prompting the user again.
  • OAuthClientId: The client Id that was generated when you registered your custom OAuth application.
  • OAuthClientSecret: The client secret that was generated when you registered your custom OAuth application.
  • CallbackURL: A redirect URI you defined during application registration.

After authentication, tokens are saved to OAuthSettingsLocation. These values persist across sessions and are used to automatically refresh the access token when it expires, so you don't need to log in again on future connections.

Web Applications

To authenticate from a web application, you must register a custom OAuth application in Microsoft Entra ID (formerly Azure Active Directory). Embedded OAuth apps are not supported in this context because web-based flows require a registered redirect URI and centralized credential management.

This approach is designed for hosted, multi-user environments where access must be delegated through a secure, standards-compliant OAuth workflow. It gives your organization control over the OAuth client, redirect URI, branding, and permissions scope.

Before you begin: Register a custom OAuth application in the Azure portal. During registration, collect the following values:

  • OAuthClientId: The client Id that was generated when you registered your custom OAuth application.
  • OAuthClientSecret: The client secret that was generated when you registered your custom OAuth application.
  • CallbackURL: A redirect URI you defined during application registration.

For full instructions on how to register a custom OAuth application and configure redirect URIs, see Creating an Entra ID (Azure AD) Application.

To authenticate using AzureAD in a web application, configure the following connection properties:

  • AuthScheme: AzureAD
  • InitiateOAuth: OFF – Disables automatic login prompts.
  • OAuthClientId: The client Id that was generated when you registered your custom OAuth application.
  • OAuthClientSecret: The client secret that was generated when you registered your custom OAuth application.
  • CallbackURL: A redirect URI you defined during application registration.

Because web applications typically manage OAuth flows manually on the server-side, InitiateOAuth must be set to OFF. This allows you to explicitly control when and how tokens are retrieved and exchanged using stored procedures.

After configuring these properties, follow the steps below to obtain and exchange OAuth tokens:

  1. Call the GetOAuthAuthorizationURL stored procedure:
    • CallbackURL: Set to your registered redirect URI
  2. Open the returned URL in a browser. Sign in with a Microsoft Entra ID account and grant access.
  3. After signing in, you are redirected to your CallbackURL with a code parameter in the query string.
  4. Extract the code and pass it to the GetOAuthAccessToken stored procedure:
    • AuthMode: WEB
    • Verifier: The authorization code from the CallbackURL
  5. The procedure returns:
    • OAuthAccessToken: Used for authentication.
    • OAuthRefreshToken: Used to refresh the access token.
    • ExpiresIn: The lifetime of the access token in seconds.

To enable automatic token refresh, configure the following connection properties:

When InitiateOAuth is set to REFRESH, the driver uses the provided refresh token to request a new access token automatically.

After a successful connection, the driver saves the updated access and refresh tokens to the file specified by OAuthSettingsLocation.

You only need to repeat the full OAuth authorization flow if the refresh token expires, is revoked, or becomes invalid.

For more background on OAuth flows in Microsoft Entra ID, see Microsoft Entra Authentication Overview.

Headless Machines

Headless environments like CI/CD pipelines, background services, or server-based integrations do not have an interactive browser. To authenticate using AzureAD, you must complete the OAuth flow on a separate device with a browser and transfer the authentication result to the headless system.

Setup options:

  • Obtain and exchange a verifier code
    • Use another device to sign in and retrieve a verifier code, which the headless system uses to request tokens.
  • Transfer an OAuth settings file
    • Authenticate on another device, then copy the stored token file to the headless environment.

Using a Verifier Code

  1. On a device with a browser:
    • If using a custom OAuth app, set the following properties:
      • InitiateOAuth: OFF
      • OAuthClientId: The client Id that was generated when you registered your custom OAuth application.
      • OAuthClientSecret: The client secret that was generated when you registered your custom OAuth application.
    • Call the GetOAuthAuthorizationURL stored procedure to generate a sign-in URL.
    • Open the returned URL in a browser. Sign in and grant grant permissions to the driver. You are redirected to the callback URL, which contains the verifier code.
    • After signing in, save the value of the code parameter from the redirect URL. You will use this later to set the OAuthVerifier connection property.
  2. On the headless machine:
    • Set the following properties:
    • After tokens are saved, reuse them by setting:
      • InitiateOAuth: REFRESH
      • OAuthSettingsLocation: Make sure this location grants read and write permissions to the driver to enable the automatic refreshing of the access token.
      • For custom applications:
        • OAuthClientId: The client Id that was generated when you registered your custom OAuth application.
        • OAuthClientSecret: The client secret that was generated when you registered your custom OAuth application.

Transferring OAuth Settings

  1. On a device with a browser:
    • Connect using the instructions in the Desktop Applications section.
    • After connecting, tokens are saved to the file path in OAuthSettingsLocation. The default filename is OAuthSettings.txt.

  2. On the headless machine:
    • Copy the OAuth settings file to the machine.
    • Set the following properties:
      • AuthScheme: AzureAD
      • InitiateOAuth: REFRESH
      • OAuthSettingsLocation: Make sure this location grants read and write permissions to the driver to enable the automatic refreshing of the access token.
      • For custom applications:
        • OAuthClientId: The client Id that was generated when you registered your custom OAuth application.
        • OAuthClientSecret: The client secret that was generated when you registered your custom OAuth application.

After setup, the driver uses the stored tokens to refresh the access token automatically, no browser or manual login is required.

Azure Service Principal

Note: Microsoft has rebranded Azure AD as Entra ID. In topics that require the user to interact with the Entra ID Admin site, we use the same names Microsoft does. However, there are still CData connection properties whose names or values reference "Azure AD".

Azure Service Principal is role-based application-based authentication. This means that authentication is done per application, rather than per user. All tasks taken on by the application are executed without a default user context, but based on the assigned roles. The application access to the resources is controlled through the assigned roles' permissions.

For information about how to set up Azure Service Principal authentication, see Creating a Service Principal App in Entra ID (Azure AD).

Managed Service Identity (MSI)

If you are running Apache Kafka on an Azure VM and want to automatically obtain Managed Service Identity (MSI) credentials to connect, set AuthScheme to AzureMSI.

User-Managed Identities

To obtain a token for a managed identity, use the OAuthClientId property to specify the managed identity's client_id.

If your VM has multiple user-assigned managed identities, you must also specify OAuthClientId.

Shared-Access Signature

The driver supports password-based authentication using shared-access signatures. After you create the shared secret, set these properties:

  • AuthScheme: Plain.
  • User: $ConnectionString.
  • Password: The Event Hubs connection string from the Shared Access Policies screen.

Connecting to GCP Kafka

The driver supports connecting to Google Managed Service for Apache Kafka (GCP Kafka). GCP Kafka uses OAuth authentication and supports service accounts, GCP instance accounts, and Workload Identity Federation.

All connections to GCP Kafka must set these properties:

  • BootstrapServers: bootstrap.myclustername.myregion.managedkafka.mygcpproject.cloud.goog:9092. This value is listed on the Cluster Configuration page, under the Configurations tab.
  • UseSSL: True.

After you set the appropriate scheme-specific properties (described below), you are ready to connect.

Authenticating to GCP Kafka

You can authenticate to GCP Kafka as a Google service account, a GCP instance account, or using Workload Identity Federation credentials.

Service Account

GCP Kafka supports authenticating as a Google service account. This service account must have the Managed Kafka Client role.

Provide the service account credentials to the driver with these properties:

GCP Instance Account

GCP Kafka supports connections using GCP instance accounts. This requires your Compute Engine instance to have a service account with the Managed Kafka Client role. The instance must also enable the Cloud Platform scope within the Cloud API Access Scopes.

To connect using a GCP instance account, set this property:

Workload Identity Federation Credentials

GCP Kafka supports connections using Workload Identity Federation credentials. However, it only supports these accounts via delegation with the RequestingServiceAccount property. As with normal service accounts, the delegated service account must have the Managed Kafka Client role.

To connect using Workload Identity Federation credentials, set these properties:

Copyright (c) 2025 CData Software, Inc. - All rights reserved.
Build 25.0.9434