Specifies how to serialize/deserialize message contents.

Possible Values

NONE, AUTO, JSON, CSV, CSV_WITH_HEADERS, XML, AVRO, LONG, INTEGER, FLOAT, DOUBLE, STRING

Data Type

string

Default Value

"AUTO"

Remarks

The provider uses this property differently based on the value of TypeDetectionScheme. See Extracting Metadata From Topics for details on how these properties interact.

Primitive and Complex Formats

This section applies only to the SchemaRegistry, None, and RowScan TypeDetectionScheme modes. MessageOnly always reports the message as a single column regardless of the format. The only difference is the column type.

The provider supports two different kinds of formats: primitive formats and complex formats. Primitive formats are reported in a single column called Message. The primitive formats use encodings that are compatible with the kafka-clients and Confluent.Kafka libraries.

Avro, CSV, CSV_WITH_HEADERS, XML, and JSON are all complex formats. The provider parses these formats into one or more columns, flattening nested Avro, XML, and JSON values as necessary.

Auto is also a complex format but the exact data format is determined at runtime. The provider determines whether a value is Avro, CSV, XML, or JSON by looking for either a specific header (the Avro OBJ header) or specific characters. If none of these methods succeed the provider assumes the value is CSV.

Available formats:

NONE	Message is always BASE64 encoded on both the consume and produce operations.
AUTO	Attempt to automatically figure out the current topic's serialization format. See Extracting Metadata From Topics for a discussion of how this occurs in different contexts.
JSON	Message is serialized using the JSON format.
CSV	Message is serialized using the CSV format.
CSV_WITH_HEADERS	Message is serialized using the CSV format with a separate header line before the data. Note that this option only applies to messages created using INSERT. It behaves the same as CSV in SELECT.
XML	Message is serialized using the XML format.
AVRO	Message is serialized using the Avro format.
LONG	Message is serialized as a 64-bit big-endian integer.
INTEGER	Message is serialized as a 32-bit big-endian integer.
FLOAT	Message is serialized as a 32-bit floating-point number.
DOUBLE	Message is serialized as a 64-bit floating-point number.
STRING	Message is serialized as text. By default the provider uses UTF-8, but setting Charset overrides this.