TDV Adapter for Apache Kafka

Build 22.0.8462

Changelog

General Changes

DateBuild
Number
Change
Type
Description
12/14/20228383GeneralChanged
  • Added the Default column to the sys_procedureparameters table.
11/22/20228361Apache KafkaAdded
  • Added support for detecting the correct serialization format when performing an INSERT. The driver will determine this from either the schema registry or by row scan, depending upon the TypeDetectionScheme propery. This allows connections with SerializationFormat=AUTO to insert data when the serialization format is unknown or different across topics. Other values of SerializationFormat keep their old behavior and skip this format detection.
10/26/20228334Apache KafkaChanged
  • Updated the versions of the Kafka libraries used in both Java and .NET. JDBC now bundles kafka-clients 3.3.1 while ADO.NET is built against Confluent.Kafka 1.9.3. The new version of Confluent.Kafka only supports .NET Framework versions 4.6.2 and up. The supported versions for other .NET variants (.NET Standard and .NET 5+) are unaffected.
09/30/20228308GeneralChanged
  • Added the IsPath column to the sys_procedureparameters table.
08/17/20228264GeneralChanged
  • We now support handling the keyword "COLLATE" as standard function name as well.
08/05/20228252Apache KafkaAdded
  • Added support for CreateTablePartitions and CreateTableReplicationFactor. This allows CREATE TABLE to be used on smaller clusters, and to be tuned for better read throughput or higher failover redundancy.
07/18/20228234Apache KafkaAdded
  • Added support for ConsumerProperties and ProducerProperties options. These allow for direct configuration of the underlying Kafka client libraries, in cases where the driver doesn't expose its own version of a property.
05/03/20228158Apache KafkaAdded
  • RegistryClientCert and related properties, as well as RegistryServerCert. These properties are used for the schema registry specifically and support the same SSL options as other providers. They are added to avoid confusion between what SSL properties apply to the schema registry vs. the Kafka broker.

Deprecated
  • The SSLKey, SSLCert and SSLCertPassword properties as well as TrustStorePassword and TrustStorePath. These hidden properties were required to connect to brokers via SSL, but they were specific to each edition and did not expose all of the certificate formats that the Kafka client libraries could accept.

Replacements
  • SSLClientCert and related properties, as well as SSLServerCert and related properties. Internally these configure the same Kafka client properties, but the names and values of these options more consistent with the registry SSL options. Only a subset of certificate types are supported due to limitations of the Kafka client libraries.
04/20/20228145Apache KafkaAdded
  • Added support for Confluent-compatible Avro encoding with the UseConfluentAvroFormat option. The default format is still the Avro file block format, but setting this option allows the provider to produce messages which can be read and validated by Confluent tools and libraries.
04/07/20228132Apache KafkaChanged
  • Updated the versions of the Kafka libraries used in both Java and .NET. .NET now requires the following assemblies to be present at runtime: Confluent.Kafka (and librdkafka.redist) : 1.8.2.0 System.Memory: 4.5.2 System.Buffers: 4.5.1 System.Runtime.CompilerServices.Unsafe: 4.5.3
03/30/20228124Apache KafkaAdded
  • Added support for authenticating using SCRAM-SHA-512.
12/13/20218017Apache KafkaRemoved
  • Removed unnecesary dependencies relating to embedded Kafka servers. This also removes the log4j component which was affected by the recent CVE reports.
09/23/20217936Apache KafkaAdded
  • Added support for Authenticating with Kerberos on both JDBC and ADO editions.
09/02/20217915GeneralAdded
  • Added support for the STRING_SPLIT table-valued function in the CROSS APPLY clause.
08/07/20217889GeneralChanged
  • Added the KeySeq column to the sys_foreignkeys table.
08/06/20217888GeneralChanged
  • Added the new sys_primarykeys system table.
07/23/20217874GeneralChanged
  • Updated the Literal Function Names for relative date/datetime functions. Previously relative date/datetime functions resolved to a different value when used in the projection vs te predicate. Ie: SELECT LAST_MONTH() AS lm, Col FROM Table WHERE Col > LAST_MONTH(). Formerly the two LAST_MONTH() methods would resolve to different datetimes. Now they will match.
  • As a replacement for the previous behavior, the relative date/datetime functions in the criteria may have an 'L' appended to them. Ie: WHERE col > L_LAST_MONTH(). This will continue to resolve to the same values that previously were calculated in the criteria. Note that the "L_" prefix will only work in the predicate - it not available for the projection.
07/12/20217863Apache KafkaRemoved
  • Removed the Topic static table. Previously it was used only to retrieve some static columns. Now it's no longer needed.
07/08/20217859GeneralAdded
  • Added the TCP Logging Module for the logging information happening on the TCP wire protocol. The transport bytes that are incoming and ongoing will be logged at verbosity=5.
08/26/20217844Apache KafkaAdded
  • Added support for AWS Glue Schema Registry service. Now we support Confluent and AWS Glue Schema Registry types.

Added
  • Added support for JSON Scheme type on SchemaRegistry mode. Now we support AVRO and JSON schema types.
04/23/20217785GeneralAdded
  • Added support for handling client side formulas during insert / update. For example: UPDATE Table SET Col1 = Concat(Col1, " - ", Col2) WHERE Col2 LIKE 'A%'
04/23/20217783GeneralChanged
  • Updated how display sizes are determined for varchar primary key and foreign key columns so they will match the reported length of the column.
04/16/20217776GeneralAdded
  • Non-conditional updates between two columns is now available to all drivers. For example: UPDATE Table SET Col1=Col2

Changed
  • Reduced the length to 255 for varchar primary key and foreign key columns.
  • Updated implicit and metadata caching to improve performance and support for multiple connections. Old metadata caches are not compatible - you would need to generate new metadata caches if you are currently using CacheMetadata.
  • Updated index naming convention to avoid duplicates
  • Updated and standardized Getting Started connection help.
  • Added the Advanced Features section to the help of all drivers.
  • Categorized connection property listings in the help for all editions.
04/15 /20217775GeneralChanged
  • Kerberos authentication is updated to use TCP by default, but will fall back to UDP if a TCP connection cannot be established

Copyright (c) 2023 CData Software, Inc. - All rights reserved.
Build 22.0.8462