The Apache Kafka Adapter dynamically models Apache Kafka topics as tables. A complete list of discovered topics can be obtained from the sys_tables system table.

Connections to Apache Kafka become part of a consumer group. Any message delivered to a consumer group will only come back in a SELECT query to the topic once.

When a topic is queried for the first time, the OffsetResetStrategy property influences which messages are retrieved.

Select Earliest to retrieve old and live messages, and select Latest to select only live messages.

This means that, if Earliest was chosen as the OffsetResetStrategy, historical data is only read the first time the topic is queried.

Each adapter connection is assigned a unique consumer group, and can therefore retrieve historical data again. Set ConsumerGroupId to persists the message offset across connections. If the ConsumerGroupId is persisted, queries will retrieve all messages since the last connection. Any consistent value will work, though GUIDs are recommended.

Tables

The adapter reads the metadata from Apache Kafka topics and exposes them as tables. Set the Topic to specify a specific Topic to read from.

SELECTing from a topic returns old messages from the topic, as well as live messages posted before the number of seconds specified by the ReadDuration have elapsed.

Stored Procedures

Stored Procedures are function-like interfaces to Apache Kafka. They can be used to create schema files, commit messages, and more.