Specifies if the Apache Kafka consumer should automatically commit read offsets.

Data Type

bool

Default Value

false

Remarks

By default the provider does not commit read offsets unless you invoke CommitOffset. If you do commit an offset for a topic, the provider starts reading at that offset for future queries (until the next CommitOffset). Otherwise each query starts reading at whatever position OffsetResetStrategy dictates.

Enabling autocommit makes the provider commit offsets periodically and at the end of each SELECT query. This means that each SELECT query "consumes" the messages that it requests from Kafka, and later SELECT queries do not return them. You must provide an ConsumerGroupId to autocommit. The commit offset is shared by the consumer group so later connections using the same group ID will use the same offsets.

There are a few details to be aware of when enabling autocommit:

• Some queries require the provider to consume more messages than they return. For example, a SELECT query with an OFFSET clause first consumes the number of rows specified by the offset, then returns rows after the OFFSET is satisfied. The messages consumed for the OFFSET are discarded and cannot be read again in the same consumer group.
• The TypeDetectionScheme options None and RowScan perform intenral reads against the topic to determine what columns are available. This produces unpredictable results because the Kafka library may automatically commit after the type detection read (if the autocommit timer expires). Use the SchemaRegistry or MessageOnly TypeDetectionScheme modes to avoid these reads and ensure consistent committed offsets.
• Be aware that autocommit operates in terms of messages and not rows. Some SerializationFormat options support reading multiple rows from a single Kafka message (for example, a Kafka message that contains a JSON array). If a SELECT query reads only the first few rows out of a message, Kafka still considers the entire message as consumed. This means that any unread rows at the end of a partially read message are discarded.