Cmdlets for Databricks

Build 24.0.9060

Changelog

General Changes

DateBuild
Number		Change
Type		Description
09/26/20249035DatabricksAdded
  • Added a new AuthScheme OAuthU2M to support user-to-machine authentication.
  • Added a new AuthScheme OAuthM2M to support machine-to-machine authentication.
  • Added a new connection property OAuthLevel, which specifies the valid range of the generated access token. Allowed entries are WorkspaceLevel and AccountLevel.
  • Added a new connection property DatabricksAccountId, which specifies the Databricks account ID and is only required when OAuthU2MLevel is AccountLevel.
08/20/20248998DatabricksAdded
  • Added support for the following AWS regions: HYDERABAD, MELBOURNE, CALGARY, SPAIN, ISOLATEDUSEAST, ISOLATEDUSEASTB, and ISOLATEDUSWEST.
08/02/20248980DatabricksAdded
  • Added a new connection property ServerTimezone to allow users to specify the time zone ID of the current platform. Options are UTC (default), LOCAL, or a user-specified time zone.
07/05/20248952DatabricksDeprecated
  • Deprecated the following properties: AzureSubscriptionId, AzureResourceGroup, and AzureWorkspace. These properties are no longer required when using AzureServicePrincipal authentication, but they are retained for compatibility.
06/18/20248935DatabricksChanged
  • Improved performance of batch INSERT operations when the "UseUploadAPI" connection property is enabled.
05/09/20248895GeneralChanged
  • The ROUND function previously did not accept negative precision values. That feature has now been restored.
03/21/20248846DatabricksChanged
  • Change the behavior of the property "QueryAllMetadata". When it is False and the property "Database" is empty, the driver will query metadata from all schemas/databases under the default catalog, not only from the default schema/database under the default catalog.
03/15/20248840GeneralAdded
  • Created a new SQL function called STRING_COMPARE that provides java's String.compare() ability to SQL queries. Returns a number representative of the compared value of two strings
11/29/20238733GeneralChanged
  • The ROUND function doesn't accept the negative precision values anymore.

Changed
  • The returning types of the FDMonth, FDQuarter, FDWeek, LDMonth, LDQuarter, LDWeek functions are changed from Timestamp to Date.
  • The return type of the ABS function will be consistent with the parameter value type.
11/28/20238732GeneralAdded
  • Added the HMACSHA256 formatter to allow for secrets to be decoded if it is in base64 format
10/20/20238693DatabricksAdded
  • Support Bulk Update and Bulk Delete for Databricks SQL Endpoint(SQL Warehouse).
08/24/20238636DatabricksChanged
  • Support new AuthScheme "Basic", set this to authenticate with Databricks' user and access token.
07/20/20238601DatabricksAdded
  • Add new property "QueryAllMetadata" to specify whether to query all catalogs and schemas(databases) or only the default catalog and schema(database).
06/29/20238580PowerShell CmdletsRemoved
  • Removed support for .NET 2.0
06/20/20238571GeneralAdded
  • Added the new sys_lastresultinfo system table.
06/08/20238559DatabricksChanged
  • The default value of the "UseLegacyDataModel" property has been changed to True, indicating that the driver now supports multi-catalog by default.
05/15/20238535DatabricksAdded
  • Add new property "AllowPreparedStatement" to prepare a query statement before its execution.
04/25/20238515GeneralRemoved
  • Removed support for the SELECT INTO CSV statement. The core code doesn't support it anymore.
03/31/20238490DatabricksChanged
  • Support new AuthScheme "AzureAD".
03/15/20238474DatabricksAdded
  • Add new property "PersonalAccessTokenExpiresIn" to specifies the lifetime in seconds of the Databricks personal access token.
02/01/20238432DatabricksAdded
  • Add new property "ConnectRetryWaitTime" to specifies the number of seconds to wait to retry a connection request.
12/14/20228383GeneralChanged
  • Added the Default column to the sys_procedureparameters table.
11/11/20228350DatabricksAdded
  • Added a new "UseLegacyDataModel" property. Set it to "true" to support the Unity Catalog. The default is "false", which does not change the previous behavior (the catalog is always "CData").
09/30/20228308GeneralChanged
  • Added the IsPath column to the sys_procedureparameters table.
08/12/20228259DatabricksAdded
  • Added the connection property UseUploadApi to specify whether the Databricks Upload API will be used when executing Bulk Insert operations.
  • Added the connection property DeleteDataOnDrop to specify whether to completely delete the data and schema information of the delta table.
07/14/20228230DatabricksReplacements
  • Refactored metadata queries to improve the performance of GetTables and GetColumns.

Added
  • Added support for retrieving all schemas rather than just the default schema.
03/08/20228102DatabricksAdded
  • Added the connection property PrimaryKeyIdentifiers to specify primary keys.
  • Added support for Batch Update/Delete operations.
03/01/20228095DatabricksAdded
  • Added support for Delta Tables with the following ALTER features:
  • ALTER TABLE table_name
  • RENAME TO new_table_name
  • ADD [COLUMN | COLUMNS] (column_name column_type, ...)
  • ALTER COLUMN [SET | DROP] NOT NULL
  • RENAME COLUMN column_name TO new_column_name
02/08/20228074DatabricksAdded
  • Added the connection property ApplicationName to specify the HTTP header UserAgent.
02/07/20228073DatabricksAdded
  • Added the following Azure connection properties: AzureSubscriptionId, AzureResourceGroup and AzureWorkspace.

Removed
  • Removed the AuthScheme AzureAD.
  • Removed five AzureAD connection properties: AzureUsername, AzurePassword, AzureADTenantId, AzureADClientId and AzureADClientSecret.

Replacements
  • Use AzureServicePrincipal as the AuthScheme instead of AzureAd a replacement for AzureAD.
  • Added six AzureServicePrincipal connection properties as replacements: AzureTenantId, AzureClientId, AzureClientSecret.
01/14/20228049DatabricksAdded
  • Added support for the NATIVEQUERY table function. This function can be used after a FROM to execute a query using Databricks native SQL. For example, 
    SELECT * FROM NATIVEQUERY('SELECT * FROM default.tbl PIVOT (...)')
    will execute the inner query in Databricks directly and return the results. This will work even in tools which are not normally compatible with QueryPassthrough=true.
12/03/20218007DatabricksDeprecated
  • Connection properties UseShowDatabasesQuery, UseDescTableQuery, and UseShowTablesQuery are deprecated. They were not being used.
09/16/20217929DatabricksAdded
  • Added four connection properties SSLClientCert, SSLClientCertPassword, SSLClientCertSubject, SSLClientCertType.
09/02/20217915GeneralAdded
  • Added support for the STRING_SPLIT table-valued function in the CROSS APPLY clause.
08/07/20217889GeneralChanged
  • Added the KeySeq column to the sys_foreignkeys table.
08/06/20217888GeneralChanged
  • Added the new sys_primarykeys system table.
07/23/20217874GeneralChanged
  • Updated the Literal Function Names for relative date/datetime functions. Previously, relative date/datetime functions resolved to a different value when used in the projection as opposed to the predicate. For example: SELECT LAST_MONTH() AS lm, Col FROM Table WHERE Col > LAST_MONTH(). Formerly, the two LAST_MONTH() methods would resolve to different datetimes. Now, they will match.
  • As a replacement for the previous behavior, the relative date/datetime functions in the criteria may have an 'L' appended to them. For example: WHERE col > L_LAST_MONTH(). This will continue to resolve to the same values that were previously calculated in the criteria. Note that the "L_" prefix will only work in the predicate - it not available for the projection.
07/08/20217859GeneralAdded
  • Added the TCP Logging Module for the logging information happening on the TCP wire protocol. The transport bytes that are incoming and ongoing will be logged at verbosity=5.
04/25/20217785GeneralAdded
  • Added support for handling client side formulas during insert / update. For example: UPDATE Table SET Col1 = CONCAT(Col1, " - ", Col2) WHERE Col2 LIKE 'A%'
04/23/20217783GeneralChanged
  • Updated how display sizes are determined for varchar primary key and foreign key columns so they will match the reported length of the column.
04/16/20217776GeneralAdded
  • Non-conditional updates between two columns is now available to all drivers. For example: UPDATE Table SET Col1=Col2
04/16/20217776GeneralChanged
  • Reduced the length to 255 for varchar primary key and foreign key columns.

Changed
  • Updated index naming convention to avoid duplicates.
04/16/20217776GeneralChanged
  • Updated implicit and metadata caching to improve performance and support for multiple connections. Old metadata caches are not compatible - you need to generate new metadata caches if you are currently using CacheMetadata.
04/16/20217776PowerShell CmdletsAdded
  • Added support for PowerShell 7.

Copyright (c) 2024 CData Software, Inc. - All rights reserved.
Build 24.0.9060