SSIS Components for Google BigQuery

Build 24.0.9060

Changelog

General Changes

DateBuild
Number
Change
Type
Description
09/04/20249013Google BigQueryAdded
  • Added the DelegatedServiceAccounts connection property. When used in combination with the AuthScheme GCPInstanceAccounts, this property allows you to specify a space-delimited list of delegated service accounts. The default service account email assigned to the project will act as the requesting service account.
05/09/20248895GeneralChanged
  • The ROUND function previously did not accept negative precision values. That feature has now been restored.
03/15/20248840GeneralAdded
  • Created a new SQL function called STRING_COMPARE that provides java's String.compare() ability to SQL queries. Returns a number representative of the compared value of two strings
11/29/20238733GeneralChanged
  • The ROUND function doesn't accept the negative precision values anymore.

Changed
  • The returning types of the FDMonth, FDQuarter, FDWeek, LDMonth, LDQuarter, LDWeek functions are changed from Timestamp to Date.
  • The return type of the ABS function will be consistent with the parameter value type.
11/28/20238732GeneralAdded
  • Added the HMACSHA256 formatter to allow for secrets to be decoded if it is in base64 format
06/20/20238571GeneralAdded
  • Added the new sys_lastresultinfo system table.
05/11/20238531SSISRemoved
  • Removed support for SQL Server Integration Services 2012 and Visual Studio 2012/2013.
04/25/20238515GeneralRemoved
  • Removed support for the SELECT INTO CSV statement. The core code doesn't support it anymore.
03/24/20238483Google BigQueryAdded
  • Added support for the INTERVAL data type within all query types. INTERVAL values are represented as text and given to the driver in BigQuery SQL's native format.
01/04/20238404Google BigQueryAdded
  • Added support for the JSON data type, both within queries as well as DML insert operations. It has most of the same limitations as other aggregate types like array. It is also supported as a type name for CAST(col AS JSON) as well as CREATE TABLE t(col JSON)
12/14/20228383GeneralChanged
  • Added the Default column to the sys_procedureparameters table.
12/05/20228374SSISAdded
  • Added support for SQL Server Integration Services 2022 and Visual Studio 2022
11/03/20228342Google BigQueryAdded
  • Added support for performing a bulk UPDATE without having to specify primary keys via PrimaryKeyIdentifiers or schema files. As long as the platform native bulk interface is used (eg addBatch / executeBatch in JDBC) the provider can determine what goe into the SET vs. the WHERE clause.
09/30/20228308GeneralChanged
  • Added the IsPath column to the sys_procedureparameters table.
09/28/20228306Google BigQueryAdded
  • Added support for the FlattenObjects option. By default it is enabled and STRUCT vales are expanded as normal, but when disabled STRUCT values are reported as JSON aggregates in the same way ARRAY values are.
08/18/20228265SSISChanged
  • Changed Azure module to Az module.
06/06/20228192Google BigQueryAdded
  • Added support for MERGE statements. The syntax is a subset of what BigQuery natively supports, including everything except the MATCHED BY SOURCE, DEFAULT and WHEN MATCHED AND clauses.
04/27/20228152Google BigQueryAdded
  • Added support for the SAFE_CAST function. Due to syntax limitations, it must be written this way when passthrough mode is disabled (the default):
    SAFE_CAST(somecol, 'INTEGER')
    . It supports SQL type names (VARCHAR, INT, etc.) as well as BigQuery type names (STRING, INT64, etc.)
04/11/20228136Google BigQueryChanged
  • Changed what views are retrieved depending on the connection property UseLegacySQL. Connections using standard SQL will not report views that are only available in legacy SQL, and the same goes for standard views in legacy SQL connections.
03/01/20228095Google BigQueryAdded
  • Added support for CREATE VIEW AS statements.
02/18/20228084Google BigQueryAdded
  • Added support for ALTER TABLE. Due to BigQuery limitations, the only supported operations are ADD COLUMN, DROP COLUMN, ALTER COLUMN and (table) RENAME TO.
01/26/20228061Google BigQueryAdded
  • Added support for querying snapshot tables. Append it to the AllowedTableTypes connection property (along with other allowed table types) to make them show up.
01/13/20228048Google BigQueryAdded
  • Added support for the NATIVEQUERY table function. This function can be used after a FROM to execute a query using BigQuery-native SQL. For example,
    SELECT * FROM NATIVEQUERY('SELECT * FROM UNNEST([1,2,3]) AS a')
    will execute the inner query in BigQuery directly and return the results. This will work even in tools which are not normally compatible with QueryPassthrough=true.
01/04/20228039Google BigQueryAdded
  • Added support for querying materialized views.
12/20/20218024Google BigQueryAdded
  • Added support for using the BigQuery TABLESAMPLE without enabling QueryPassthrough. If TableSamplePercent is set then the random row selection happens server-side for each table used in a query.
12/10/20218014Google BigQueryChanged
  • Migrate to the latest version of the Storage API.
10/21/20217964Google BigQueryAdded
  • Added support for reading policy tags from the Data Catalog service. These are exposed using the PolicyTags column on sys_tablecolumns.
10/20/20217963Google BigQueryAdded
  • Added support for parameterized data types, both on the read side with and without UseStorageAPI as well as on the metadata side. The metadata is only reported on the table itself and not in queries, since BigQuery only enforces length/precision/scale when the data is written.
09/02/20217915GeneralAdded
  • Added support for the STRING_SPLIT table-valued function in the CROSS APPLY clause.
09/02/20217915Google BigQueryAdded
  • Added support for bulk UPDATE and DELETE operations. These make better use of BigQuery's DML query limits because each batch is just one MERGE operation. They are also faster because they use the bulk upload methods to send the list of rows to be changed. Primary keys must be defined on the tables to use DELETE.
08/09/20217891Google BigQueryAdded
  • Support for staging data in all formats allowed by BigQuery. JSON and Avro were already supported, this adds options that allow uploading data stored in the CSV, Parquet and ORC formats.
08/07/20217889GeneralChanged
  • Added the KeySeq column to the sys_foreignkeys table.
08/06/20217888GeneralChanged
  • Added the new sys_primarykeys system table.
07/28/20217879Google BigQueryAdded
  • Added support for reading TIME, DATETIME and TIMESTAMP data at the full precision supported by BigQuery. We now report microsecond precision in .NET-based editions and millisecond precision in Java-based editions.
07/23/20217874GeneralChanged
  • Updated the Literal Function Names for relative date/datetime functions. Previously, relative date/datetime functions resolved to a different value when used in the projection as opposed to the predicate. For example: SELECT LAST_MONTH() AS lm, Col FROM Table WHERE Col > LAST_MONTH(). Formerly, the two LAST_MONTH() methods would resolve to different datetimes. Now, they will match.
  • As a replacement for the previous behavior, the relative date/datetime functions in the criteria may have an 'L' appended to them. For example: WHERE col > L_LAST_MONTH(). This will continue to resolve to the same values that were previously calculated in the criteria. Note that the "L_" prefix will only work in the predicate - it not available for the projection.
07/22/20217873SSISAdded
  • Added a UI type editor for the SQLStatement property of the SQLExecutor component.
07/20/20217871Google BigQueryAdded
  • Added support for staging data with Google Cloud Storage when performing inserts. This is similar to the existing Upload insert mode, but transfers the row data into GCS and then creates a load job referencing it instead of performing the upload directly into BigQuery.
06/18/20217839Google BigQueryAdded
  • Added support for the GOOGLEJSONBLOB JWT certificate type. This works like the existing GOOGLEJSON certificate type except that the certificate is provided as JSON text instead of as a file path.
05/27/20217817Google BigQueryAdded
  • Added support for automatically reconnecting on Storage connections that have long idle times. This is useful most for ETL applications and other scenarios where the consumer reading data out of the driver is expected to be slower than the driver itself. In these cases the connection can be dropped by the other side (either by the Storage API itself or network appliances like firewalls and proxies) leading to timeouts. To avoid this the driver will restart the read from the same position using a new connection.
05/26/20217816SSISAdded
  • Added support for semicolon-separated SQL queries.
04/25/20217785GeneralAdded
  • Added support for handling client side formulas during insert / update. For example: UPDATE Table SET Col1 = CONCAT(Col1, " - ", Col2) WHERE Col2 LIKE 'A%'
04/23/20217783GeneralChanged
  • Updated how display sizes are determined for varchar primary key and foreign key columns so they will match the reported length of the column.
04/16/20217776GeneralAdded
  • Non-conditional updates between two columns is now available to all drivers. For example: UPDATE Table SET Col1=Col2
04/16/20217776GeneralChanged
  • Reduced the length to 255 for varchar primary key and foreign key columns.

Changed
  • Updated index naming convention to avoid duplicates.
04/16/20217776SSISAdded
  • Added a create table button for the Destination Component to make it easier to add a new destination in SSIS.
  • Improved user experience for Source Component by allowing users to select columns via a new checkbox dialog.

Changed
  • Improved query dialog for building parameterized statements.
04/05/20217765Google BigQueryAdded
  • Added support for data unnesting. This mode performs client-side expansion of array data across multiple rows in a way that emulates the compact preview in the BigQuery UI. Server-side support of some options is limited in this mode (LIMIT and OFFSET) but otherwise behaves like the nested mode with the exception that it always outputs flat data and never outputs aggregates.

Changed
  • Updated the default value for WaitForBatchResults to true since not checking the job status at smaller batch sizes can lead to lost jobs.

Deprecated
  • Deprecated the UseStreamingInserts connection property in favor of InsertMode. The default for InsertMode is Streaming.
03/04/20217733Google BigQueryRemoved
  • Removed the TempTableDataset and TempTableExpirationTime connection properties. These properties have been defunct since we moved to using the BigQuery-managed query cache for large resultsets.
01/29/20217699Google BigQueryAdded
  • Added support for the BIGNUMERIC type, which is twice the size of the existing numeric type. JDBC supports this type at full precision while .NET requires using the connection property IgnoreTypes=decimal because the full precision of BIGNUMERIC (as with regular NUMERIC) is too high for System.Decimal.
01/08/20217678Google BigQueryAdded
  • Added support for using the upload API for batch inserts. This is an API that is slower than the streaming API for large volumes of data, but is free to use and doesn't have the same buffering delays that streaming does. It is also async by default although the option to wait on the batch is available via the new WaitForBatchResults connection property.
12/23/20207662Google BigQueryAdded
  • Added support for unnesting metadata with the Storage API.
12/21/20207660Google BigQueryAdded
  • Added support for unnesting metadata with the REST API. By enabling UnnestArrays, metadata for fields within REPEATED RECORD types is returned as separate columns instead of generating JSON aggregates. These values can be selected within queries but are currently treated as NULL.
12/07/20207646Google BigQueryAdded
  • Added support for injecting partition filters into queries that require them but do not have them, if the user enables the InsertPartitionFilterOption. This modifies queries against partitioned tables that require a partition filter so that they always contain a valid partition filter, which allows queries like simple SELECT * FROM t to work. Currently all partitions are selected by the generated filter.

Copyright (c) 2024 CData Software, Inc. - All rights reserved.
Build 24.0.9060