Comma-separated list of file extensions to include into the set of the files modeled as tables.

Data Type

string

Default Value

"parquet"

Remarks

Comma-separated list of file extensions to include into the set of the files modeled as tables. For example, IncludeFiles=parquet,TXT. The default is parquet.

A '*' value can be specified to include all files. A 'NOEXT' value can be specified to include files without an extension.

It is also possible to specify datetime filters. We currently support CreatedDate and ModifiedDate. All extension filters are evaluated in disjunction (using OR operator), and then the resulting filter is evaluated in conjunction (using AND operator) with the datetime filters.

Examples:

IncludeFiles="TXT,CreatedDate<='2020-11-26T07:39:34-05:00'"
IncludeFiles="TXT,ModifiedDate<=DATETIMEFROMPARTS(2020, 11, 26, 7, 40, 50, 000)"
IncludeFiles="ModifiedDate>=DATETIMEFROMPARTS(2020, 11, 26, 7, 40, 49, 000),ModifiedDate<=CURRENT_TIMESTAMP()"