CData Python Connector for Azure Data Lake Storage

Build 22.0.8388

Resources

Lists the contents of the supplied path.

Table Specific Information

Select

This will return a list of all the files and directories in your system. By default, all the files and folders of the first level will be retrieved. You can configure the provider to read all files and folders recursively by setting the IncludeSubDirectories property to true. All filters are executed client-side within the provider.

A simple query such as SELECT * FROM Resources will have different results with different combinations of IncludeSubDirectories and Directory connection properties.

  • If IncludeSubDirectories=true and Directory is not set the table will return all files and folder with its subfolders in all the storage.
  • If IncludeSubDirectories=true and Directory=path the table will return all files and folders with its subfolders in the path specified in Directory.
  • If IncludeSubDirectories=false and Directory is not set the table will return all files and folders of the root folder.
  • If IncludeSubDirectories=false and Directory=path the table will return all files and folders of the folder specified in path.

You can alternatively specify the Directory column in the WHERE clause conditions to list the files in a specific directory.

For example, the below query will list the files located in the first level of the 'SecondLevelDir1' directory.

SELECT * FROM Resources WHERE Directory='FirstLevelDir/SecondLevelDir1'

You can also make use of the IN operator, in order to list the files located in the first levels of multiple specified directories.

SELECT * FROM Resources WHERE Directory IN ('FirstLevelDir', 'FirstLevelDir/SecondLevelDir1', 'FirstLevelDir/SecondLevelDir2')

Note: The provider will use the Azure Data Lake Storage API to process WHERE clause conditions built with the following Directory column, and the '=' operator. The rest of the filter is executed client side within the provider.

Columns

Name Type Description
Name [KEY] String The path of the file or folder.
Directory String The directory path of the file or folder.
IsDirectory String Determines if the resource is a folder or a file.
ContentLength Long Determines the size of the file in bytes.
LastModified Timestamp The timestamp when the file was modified for the last time.
Owner String The name of the owner.
Permissions String The permissions set to the file.
ETag String Unique identifier of the file or folder.

Pseudo-Columns

Pseudo column fields are used in the WHERE clause of SELECT statements and offer a more granular control over the tuples that are returned from the data source.

Name Type Description
Recursive Boolean Set this to true to retrieve all sub folders and files.

The default value is false.

Copyright (c) 2022 CData Software, Inc. - All rights reserved.
Build 22.0.8388