CData Python Connector for HDFS
The CData Python Connector for HDFS allows developers to write Python scripts with connectivity to HDFS. The connector wraps the complexity of accessing HDFS data in an interface commonly used by python connectors to common database systems.
- A variety of WHL and TAR.GZ files that accommodate several execution environments after installation with "pip install".
- Supported for Python versions 3.6, 3.7, 3.8, and 3.9, within both Windows and Linux environments. Python 3.8 distributions on Mac are also supported.
- Write and execute SQL queries to fetch data in HDFS.
- Custom dialect class that enables SQLAlchemy ORM to use this connector.
See Getting Started to install the connector to your python distribution and to create a basic connection to HDFS.
Advanced Features details additional features supported by the provider, such as defining user defined views, ssl configuration, remoting, caching, firewall/proxy settings, and advanced logging.
Using the Python Connector
See Using the Connector for examples of executing basic SELECT, INSERT, UPDATE, DELETE, and EXECUTE queries with the module's provided classes. See Using from Tools to connect HDFS data to tools such as Pandas or Petl.
SQLAlchemy can be leveraged to model the tables in HDFS with mapped classes. See From SQLAlchemy for instructions for configuring the Python connector with SQLAlchemy.
Pandas' DataFrames can be used alongside the connector to generate analytical graphics. See From Pandas for a guide.
See Schema Discovery to query the provided system tables, which allows users to discover the available tables, views, and stored procedure, alongside additional information about their columns or parameters.
See SQL Compliance for a syntax reference and code examples outlining the supported SQL.
See Data Model for the available database objects. This section also provides more detailed information on querying specific HDFS entities.
Connection String Options
The Connection properties describe the various options that can be used to establish a connection.