TDV Adapter for Apache Kafka

Build 22.0.8462

Connection String Options

The connection string properties are the various options that can be used to establish a connection. This section provides a complete list of the options you can configure in the connection string for this provider. Click the links for further details.

For more information on establishing a connection, see Basic Tab.

Authentication


PropertyDescription
AuthSchemeThe scheme used for authentication with the Apache Kafka broker.
UserThe user who is authenticating to Apache Kafka.
PasswordThe password used to authenticate to Apache Kafka.
BootstrapServersThe address of the Apache Kafka BootstrapServers to which you are connecting to.
TopicThe topic used for read and write operations.
UseSSLThis field sets whether SSL is enabled. Automatically enabled if AuthScheme is set to SSL.

Connection


PropertyDescription
ConsumerGroupIdSpecifies which group the consumers created by the driver should belong to.
AutoCommitSpecifies if the Apache Kafka consumer should autocommit after each poll.

Kerberos


PropertyDescription
KerberosKeytabFileThe Keytab file containing your pairs of Kerberos principals and encrypted keys.
KerberosSPNThe service principal name (SPN) for the Kerberos Domain Controller.
KerberosServiceNameThe name of the Kerberos service you want to authenticate with.
UseKerberosTicketCacheSet this to use a ticket cache with the logged in user instead of a keytab file.

SSL


PropertyDescription
SSLServerCertThe SSL server certificate used to validate to the Apache Kafka broker.
SSLServerCertTypeThe format of the SSL server certificate used to verify the Apache Kafka broker.
SSLServerCertPasswordThe password used to decrypt the certificate in SSLServerCert .
SSLClientCertThe SSL client certificate used to connect to the Apache Kafka broker.
SSLClientCertTypeThe format of the SSL client certificate used to connect to the Apache Kafka broker.
SSLClientCertPasswordThe password used to decrypt the certificate in SSLClientCert .
SSLIdentificationAlgorithmThe endpoint identification algorithm used by the Apache Kafka data provider client app to validate server host name.

Schema Registry


PropertyDescription
RegistryUrlThe server for the schema registry. When this property is specified, the driver will read Apache Avro schema from the server.
RegistryTypeType of the schema specified for the a specific topic.
RegistryServiceThe Schema Registry service used for working with topic schemas.
RegistryAuthSchemeThe scheme used to authenticate to the schema registry.
RegistryUserUsername to authorize with the server specified in RegistryUrl .
RegistryPasswordPassword to authorize with the server specified in RegistryUrl .
RegistryClientCertThe TLS/SSL client certificate store for SSL Client Authentication (2-way SSL) with the schema registry.
RegistryClientCertTypeThe type of key store used by the TLS/SSL client certificate given in RegistryClientCert .
RegistryClientCertPasswordThe password for the TLS/SSL client certificate given in RegistryClientCert .
RegistryClientCertSubjectThe subject of the TLS/SSL client certificate given in RegistryClientCert .
RegistryVersionVersion of the schema read from RegistryUrl for the specified topic.
RegistryServerCertThe certificate to be accepted from the schema registry when connecting using TLS/SSL.

Firewall


PropertyDescription
FirewallTypeThe protocol used by a proxy-based firewall.
FirewallServerThe name or IP address of a proxy-based firewall.
FirewallPortThe TCP port for a proxy-based firewall.
FirewallUserThe user name to use to authenticate with a proxy-based firewall.
FirewallPasswordA password used to authenticate to a proxy-based firewall.

Proxy


PropertyDescription
ProxyAutoDetectThis indicates whether to use the system proxy settings or not. This takes precedence over other proxy settings, so you'll need to set ProxyAutoDetect to FALSE in order use custom proxy settings.
ProxyServerThe hostname or IP address of a proxy to route HTTP traffic through.
ProxyPortThe TCP port the ProxyServer proxy is running on.
ProxyAuthSchemeThe authentication type to use to authenticate to the ProxyServer proxy.
ProxyUserA user name to be used to authenticate to the ProxyServer proxy.
ProxyPasswordA password to be used to authenticate to the ProxyServer proxy.
ProxySSLTypeThe SSL type to use when connecting to the ProxyServer proxy.
ProxyExceptionsA semicolon separated list of destination hostnames or IPs that are exempt from connecting through the ProxyServer .

Logging


PropertyDescription
LogModulesCore modules to be included in the log file.

Schema


PropertyDescription
LocationA path to the directory that contains the schema files defining tables, views, and stored procedures.

Miscellaneous


PropertyDescription
AggregateMessagesSpecifies whether or not to return the message as a whole string.
CompressionTypeData compression type. Batches of data will be compressed together.
ConsumerPropertiesAdditional options used to configure Kafka consumers.
CreateTablePartitionsThe number of partitions assigned to a topic created with CREATE TABLE.
CreateTableReplicationFactorThe number of replicas assigned to a topic created with CREATE TABLE.
EnableIdempotenceIf set to true, the Apache Kafka will ensure messages are delivered in the correct order, and without duplicates.
FlattenArraysBy default, nested arrays won't show up if TypeDetectionScheme is set to SchemaRegistry. The FlattenArrays property can be used to flatten the elements of nested arrays into columns of their own. Set FlattenArrays to the number of elements you want to return from nested arrays.
GenerateSchemaFilesIndicates the user preference as to when schemas should be generated and saved.
MaximumBatchSizeSpecifies maximum batch size to gather before sending a request.
MaxRowsLimits the number of rows returned rows when no aggregation or group by is used in the query. This helps avoid performance issues at design time.
MessageKeyColumnIf specified, the message key sent to Apache Kafka will be read from this column.
MessageKeyTypeIf MessageKeyColumn is specified, this property must be set to the expected type for the pertinent column.
OffsetResetStrategySpecifies an offset for the consumer group.
OtherThese hidden properties are used only in specific use cases.
ProduceMetaSpecifies whether or not to send a meta message while producing the outgoing message.
ProducerPropertiesAdditional options used to configure Kafka producers.
ReadDurationThe duration which additional messages are allowed.
ReadonlyYou can use this property to enforce read-only access to Apache Kafka from the provider.
RowScanDepthThe maximum number of messages to scan for the columns available in the topic.
SerializationFormatSpecifies how to serialize/deserialize the incoming or outgoing message.
TimeoutThe value in seconds until the timeout error is thrown, canceling the operation.
TypeDetectionSchemeComma-separated list of options specifying how the provider will scan the data to determine the fields and datatypes for the bucket.
UseConfluentAvroFormatSpecifies how Avro data should be formatted during an INSERT.
UserDefinedViewsA filepath pointing to the JSON configuration file containing your custom views.
ValidateRegistryTopicsSpecifies whether or not to validate schema registry topics against the Apache Kafka broker. Only has an effect when TypeDetectionScheme =SchemaRegistry.

Copyright (c) 2023 CData Software, Inc. - All rights reserved.
Build 22.0.8462