Connecting to Azure Data Lake Storage
Connecting to Azure Data Lake Storage
Set AzureStorageAccount to the name of your Azure Data Lake Storage account.
Authenticating to Azure Data Lake Storage
Azure Data Lake Storage supports authentication via Access Key, Shared Access Signature (SAS), AzureAD user, Azure Service Principal, or Azure MSI.
Access Key
To authenticate with an Azure Access key, set these connection properties:
- AuthScheme: AccessKey.
- AzureAccessKey: The storage key associated with your Azure Data Lake Storage account.
Shared Access Signature (SAS)
To authenticate with a Shared Access Signature (SAS), you must first generate the SAS at the Azure Portal:- Sign into the Azure Portal (https://portal.azure.com/) with your root account credentials.
- Click Storage accounts and select the storage account you want to use.
- Under Security + networking, click Shared access signature.
- Set any required permissions.
- Specify a time when you want the token to expire.
- To generate the shared access signature, click Generate SAS and connection string.
- When the SAS has been generated, copy it.
Once you have obtained the SAS, set these connection properties:
- AuthScheme: AzureStorageSAS.
- AzureSharedAccessSignature: The SAS associated with your Azure Blob Storage account, which you just generated.
AzureAD User
AzureAD User authentication supports connection via a desktop application, a web application, or a headless machine. In all cases, you must set AuthScheme to AzureAD.
Desktop Applications
CData provides an embedded OAuth application that simplifies OAuth desktop Authentication. You can also choose to connect via a custom OAuth application, which provides further flexibility. For further information,see Creating a Custom Azure AD Application.Get and Refresh the OAuth Access Token
Build your connection string using the following connection properties:
- OAuthClientId (custom applications only): The client Id that was assigned when you registered your custom application.
- OAuthClientSecret (custom applications only): The client secret that was assigned when you registered your custom application.
- CallbackURL (custom application only): The redirect URI you defined when you registered your custom application. For example: http://localhost:33333
Headless Machines
To configure the driver, use OAuth with a user account on a headless machine. You need to authenticate on another device that has an internet browser.
- Choose one of two options:
- Option 1: Obtain the OAuthVerifier value as described in "Obtain and Exchange a Verifier Code" below.
- Option 2: Install the connector on a machine with an internet browser and transfer the OAuth authentication values after you authenticate through the usual browser-based flow.
- Then configure the connector to automatically refresh the access token on the headless machine.
Option 1: Obtain and Exchange a Verifier Code
To obtain a verifier code, you must authenticate at the OAuth authorization URL.
Follow the steps below to authenticate from the machine with an internet browser and obtain the OAuthVerifier connection property.
- Choose one of these options:
- If you are using the Embedded OAuth Application, call the GetOAuthAuthorizationURL stored procedure. Open the URL returned by the stored procedure in a browser.
- If you are using a custom OAuth application, set the following properties:
- InitiateOAuth: OFF.
- OAuthClientId: The client Id assigned when you registered your application.
- OAuthClientSecret: The client secret assigned when you registered your application.
- Log in and grant permissions to the connector. You are then redirected to the redirect URI.
There will be a parameter called code appended to the redirect URI. Note the value of this parameter. Later you will set this in the OAuthVerifier connection property.
Next, you need to exchange the OAuth verifier code for OAuth refresh and access tokens.
On the headless machine, set the following connection properties to obtain the OAuth authentication values:
- InitiateOAuth: REFRESH.
- OAuthVerifier: The value of the code parameter in the redirect URI.
- OAuthClientId (custom applications only): The client Id in your custom OAuth application settings.
- OAuthClientSecret (custom applications only): The client secret in the custom OAuth application settings.
- OAuthSettingsLocation: Set this to persist the encrypted OAuth authentication values to the specified location.
Test the connection to generate the OAuth settings file, then re-set the following properties to connect:
- InitiateOAuth: REFRESH.
- OAuthClientId (custom applications only): The client Id assigned when you registered your custom OAuth application.
- OAuthClientSecret (custom applications only): The client secret assigned when you registered your custom OAuth application.
- OAuthSettingsLocation: The location containing the encrypted OAuth authentication values. Make sure this location gives read and write permissions to the connector to enable the automatic refreshing of the access token.
Option 2: Transfer OAuth Settings
Prior to connecting on a headless machine, you need to install and create a connection with the driver on a device that supports an internet browser. Set the connection properties as described in "Desktop Applications" above.
After completing the instructions in "Desktop Applications", the resulting authentication values are encrypted and written to the location specified by OAuthSettingsLocation. The default filename is OAuthSettings.txt.
Test the connection to generate the OAuth settings file, then copy the OAuth settings file to your headless machine.
On the headless machine, set the following connection properties to connect to data:
- InitiateOAuth: REFRESH.
- OAuthClientId (custom applications only): The client Id assigned when you registered your custom OAuth application.
- OAuthClientSecret: (custom applications only) The client secret assigned when you registered your custom OAuth application.
- OAuthSettingsLocation: The location of the OAuth settings file you copied from the machine with the browser. Make sure this location gives read and write permissions to the connector to enable the automatic refreshing of the access token.
Azure Service Principal
Authentication as an Azure Service Principal is handled via the OAuth Client Credentials flow. It does not involve direct user authentication. Instead, credentials are created for just the application itself.All tasks taken by the application are done without a default user context, but based on the assigned roles. The application access to the resources is controlled through the assigned roles' permissions.
For Azure Service Principal authentication, set AuthScheme to AzureServicePrincipal.
Creating an AzureAD App and an Azure Service Principal
If you will authenticate using an Azure Service Principal, you must first create and register an Azure AD application with an Azure AD tenant, as described in Creating an Entra ID (Azure AD) Application.
In the Azure portal, navigate to App registrations > API permissions. Select the Microsoft Graph permissions. There are two distinct sets of permissions: Delegated permissions and Application permissions. The permissions used during client credential authentication are under Application Permissions.
Assigning a role to the application
To access resources in your subscription, you must assign an appropriate role to the custom Azure AD application. Do the following:
- Use the search bar to locate the Subscriptions service.
- Open the Subscriptions page.
- Select the subscription to which to assign the application.
- Open Access control (IAM) and select Add > Add role assignment. The Add role assignment page opens.
- Assign your custom Azure AD application the Owner role.
Setting the connection properties
The connection properties you set to connect with your custom Azure AD application will vary, depending on whether you want to authenticate using a Client Secret or a Certificate.
After you connect, authentication with client credentials takes place automatically like any other connection, except that no window opens to prompt the user. Because there is no user context, there is no need for a browser popup. Connections take place and are handled internally.
Client Secret Connection Properties
- AuthScheme: AzureServicePrincipal.
- InitiateOAuth: GETANDREFRESH. You can use InitiateOAuth to avoid repeating the OAuth exchange and manually setting the OAuthAccessToken.
- AzureTenant: The tenant to which you want to connect.
- OAuthClientId: The client Id in your custom Azure AD application settings.
- OAuthClientSecret: The client secret in your custom Azure AD application settings.
Certificate Connection Properties
- AuthScheme: AzureServicePrincipalCert.
- InitiateOAuth: GETANDREFRESH. You can use InitiateOAuth to avoid repeating the OAuth exchange and manually setting the OAuthAccessToken.
- AzureTenant: The tenant to which you want to connect.
- OAuthJWTCert: The JWT Certificate store.
- OAuthJWTCertType: The type of the certificate store specified by OAuthJWTCert.
- OAuthJWTIssuer: The issuer of the Java Web Token.
Note: In most cases, OAuthJWTIssuer takes the value of the OAuthClientId property and does not need to be individually set.