SSIS Components for Parquet

Build 24.0.9060

Changelog

General Changes

DateBuild
Number
Change
Type
Description
10/15/20249054ParquetReplaced
  • In the ListFiles stored procedure, replaced the AbsolutePath input with the Path input. This path is relative to the path provided in the URI connection property.
09/30/20249039ParquetAdded
  • Added the ListFiles stored procedure. By default, this procedure lists all Parquet files in the directory specified by the URI connection property. If an alternative directory is supplied in the AbsolutePath input, this procedure will instead list all Parquet files in the supplied directory.
08/23/20249001ParquetAdded
  • Added OAuthPKCE as an authentication option when ConnectionType is set to Google Cloud Storage and Google Drive.
  • Added OAuthClient as an authentication option when ConnectionType is set to Box.
08/20/20248998ParquetAdded
  • Added support for the following AWS regions: HYDERABAD, MELBOURNE, CALGARY, SPAIN, ISOLATEDUSEAST, ISOLATEDUSEASTB, and ISOLATEDUSWEST.
06/28/20248945ParquetRemoved
  • Removed the value "Auto" from the ConnectionType connection property. The default value is now "Local".

Added
  • Added the "Custom Stream" value to the ConnectionType connection property.
06/18/20248935GeneralAdded
  • Added support for the "TELAVIV" region (Israel) in the AWSRegion connection property.
05/31/20248917ParquetChanged
  • The format of the file name generated by the Relational data model now always includes the name of the originating file, prepended to the table name. For example, for the file 'github.parquet', the names of the tables generated by the Relational data model have the format 'github.tablename'.
05/09/20248895GeneralChanged
  • The ROUND function previously did not accept negative precision values. That feature has now been restored.
03/15/20248840GeneralAdded
  • Created a new SQL function called STRING_COMPARE that provides java's String.compare() ability to SQL queries. Returns a number representative of the compared value of two strings
11/29/20238733GeneralChanged
  • The ROUND function doesn't accept the negative precision values anymore.

Changed
  • The returning types of the FDMonth, FDQuarter, FDWeek, LDMonth, LDQuarter, LDWeek functions are changed from Timestamp to Date.
  • The return type of the ABS function will be consistent with the parameter value type.
11/28/20238732GeneralAdded
  • Added the HMACSHA256 formatter to allow for secrets to be decoded if it is in base64 format
08/14/20238626ParquetAdded
  • Added AzureServicePrincipalCert as a separate AuthScheme to be used for Azure Service Principal authentication with a Certificate.
06/20/20238571GeneralAdded
  • Added the new sys_lastresultinfo system table.
05/11/20238531SSISRemoved
  • Removed support for SQL Server Integration Services 2012 and Visual Studio 2012/2013.
04/25/20238515GeneralRemoved
  • Removed support for the SELECT INTO CSV statement. The core code doesn't support it anymore.
12/14/20228383GeneralChanged
  • Added the Default column to the sys_procedureparameters table.
12/05/20228374SSISAdded
  • Added support for SQL Server Integration Services 2022 and Visual Studio 2022
09/30/20228308GeneralChanged
  • Added the IsPath column to the sys_procedureparameters table.
08/30/20228277ParquetAdded
  • Added support for uploading server-side encrypted objects for Amazon S3 storage source via ServerSideEncryption connection property.
08/18/20228265SSISChanged
  • Changed Azure module to Az module.
07/22/20228238ParquetRemoved
  • Removed the ParallelPagingSizeLimit connection property.
06/21/20228207ParquetChanged
  • Return an empty table list instead of throwing error when a folder is devoid of Parquet files.
05/27/20228182ParquetAdded
  • Added the ConnectionType connection property to determine the storage type of specified URI.
05/09/20228164ParquetAdded
  • Added write support (CREATE TABLE and INSERT INTO) for storage sources.
03/10/20228104ParquetAdded
  • Added support for Apache HDFS (webhdfs://) as a storage source.
03/09/20228103ParquetChanged
  • Changed the default method for submitting client_credentials grants with OAuth. Before client_credentials would submit OAuthClientId and OAuthClientSecret using the HTTP Authorization header. Now the OAuthPasswordGrantType controls this setting and it defaults to OAuthPasswordGrantType=Post. The previous behavior is still available by setting OAuthPasswordGrantType=Basic.
01/13/20228048ParquetAdded
  • Added support for IBM Cloud Object Storage (ibmobjectstorage://) as a storage source.
10/07/20217950ParquetAdded
  • Added support for excluding files based on the ExcludeFiles connection property.
  • Added support for datetime filters in IncludeFiles and ExcludeFiles connection properties. We currently support CreatedDate and ModifiedDate. Ex:
    ExcludeFiles="TXT,CreatedDate<='2020-11-26T07:39:34-05:00'" 
    ExcludeFiles="TXT,ModifiedDate<=DATETIMEFROMPARTS(2020, 11, 26, 7, 40, 50, 000)"
    ExcludeFiles="ModifiedDate>=DATETIMEFROMPARTS(2020, 11, 26, 7, 40, 49, 000),ModifiedDate<=CURRENT_TIMESTAMP()" 
  • Only the following storage sources will have support for filtering by CreatedDate:
    • Box (box://)
    • GCloudStorage (gs://)
    • GoogleDriveStorageSource (gdrive://)
    • OneDrive (onedrive://)
    • SharePoint (sp://)
    • SharePointREST (sprest://)
09/02/20217915GeneralAdded
  • Added support for the STRING_SPLIT table-valued function in the CROSS APPLY clause.
08/07/20217889GeneralChanged
  • Added the KeySeq column to the sys_foreignkeys table.
08/06/20217888GeneralChanged
  • Added the new sys_primarykeys system table.
07/23/20217874GeneralChanged
  • Updated the Literal Function Names for relative date/datetime functions. Previously, relative date/datetime functions resolved to a different value when used in the projection as opposed to the predicate. For example: SELECT LAST_MONTH() AS lm, Col FROM Table WHERE Col > LAST_MONTH(). Formerly, the two LAST_MONTH() methods would resolve to different datetimes. Now, they will match.
  • As a replacement for the previous behavior, the relative date/datetime functions in the criteria may have an 'L' appended to them. For example: WHERE col > L_LAST_MONTH(). This will continue to resolve to the same values that were previously calculated in the criteria. Note that the "L_" prefix will only work in the predicate - it not available for the projection.
07/22/20217873SSISAdded
  • Added a UI type editor for the SQLStatement property of the SQLExecutor component.
06/24/20217845ParquetAdded
  • Added support for the GOOGLEJSONBLOB JWT certificate type. This works like the existing GOOGLEJSON certificate type except that the certificate is provided as JSON text instead of as a file path.
06/08/20217829ParquetAdded
  • Added support for the AzureServicePrinciple authentication scheme, which will support both using an OAuthClientSecret, or alternatively using a JWT cert.
06/05/20217826ParquetAdded
  • Added support to authenticate submitting JWT certs instead of the OAuthClientSecret for the AzureAD authentication scheme.
05/26/20217816SSISAdded
  • Added support for semicolon-separated SQL queries.
04/23/20217783GeneralChanged
  • Updated how display sizes are determined for varchar primary key and foreign key columns so they will match the reported length of the column.
04/23/20217783ParquetDeprecated
  • SSODomain will not be used anymore for OKTA, OneLogin and ADFS. Instead, Domain should be specified.
  • The AzureAccount connection property is deprecated. Instead, AzureStorageAccount should be used.
04/16/20217776GeneralChanged
  • Reduced the length to 255 for varchar primary key and foreign key columns.

Changed
  • Updated index naming convention to avoid duplicates.
04/16/20217776SSISAdded
  • Added a create table button for the Destination Component to make it easier to add a new destination in SSIS.
  • Improved user experience for Source Component by allowing users to select columns via a new checkbox dialog.

Changed
  • Improved query dialog for building parameterized statements.
04/15/20217775GeneralChanged
  • Kerberos authentication is updated to use TCP by default, but will fall back to UDP if a TCP connection cannot be established.
01/14/20217684ParquetAdded
  • Added Insert / Update support.
12/23/20207662ParquetChanged
  • Added performance improvements for file parsing.

Copyright (c) 2024 CData Software, Inc. - All rights reserved.
Build 24.0.9060