The provider supports reading bulk messages from topics using the CSV, JSON, or XML SerializationFormat. When the provider reads CSV data similar to the following block, it splits the CSV and outputs each line as a separate row. The values of other columns like the partition, timestamp, and key are the same across each row.

"1","alpha"
"2","beta"
"3","gamma"

Bulk messages are not supported for key values. When MessageKeyType is set to a bulk format, the provider reads only the first row of the key and ignores the rest. For example, when the provider reads the above CSV data as a message key, the entries on the alpha row are repeated across every bulk row from the message value. The entries on the beta and gamma rows are lost.

Bulk Limitations

Apache Kafka does not natively support bulk messages, which can lead to rows being skipped in some circumstances. For example:

1. A provider connection is created with ConsumerGroupId set to x.
2. The connection executes the query SELECT * FROM topic LIMIT 3.
3. The connection commits its offset and closes.
4. Another connection is created with the same ConsumerGroupId.
5. The connection executes the query SELECT * FROM topic.

Consider what happens if this procedure is performed on the following topic. The first connection consumes all rows from the first message and one row from the second. However, the provider has no way to report to Apache Kafka that only part of the second message was read. This means that step 3 commits the offset 3 and the second connection starts on row 5, skipping row 4.

"row 1"
"row 2"
/* End of message 1 */

"row 3"
"row 4"
/* End of message 2 */

"row 5"
"row 6"
/* End of message 3 */