SSIS Components for Amazon Athena

Build 24.0.9060

Establishing a Connection

Enabling SSIS in Visual Studio 2022

If you're using Visual Studio 2022, you will need to install the SQL Server Integration Services Projects extension to use SSIS.

  1. Navigate to Extensions > Manage Extensions.
  2. In the Manage Extensions window's search box, search for "SQL Server Integration Services Projects 2022" and select the extension in the list.
  3. Click Download.
  4. Close Visual Studio and run the downloaded Microsoft.DataTools.IntegrationServices.exe installer. Proceed through the installer with default settings.
  5. Open Visual Studio. There should now be an "Integration Services Project" project template available.

Adding the Amazon Athena Connection Manager

Create a new connection manager as follows:

  1. Create a Visual Studio project with the "Integration Services Project" template.
  2. In the project, right-click within the Connection Managers window and select New Connection from the menu.
  3. In the Description column, select CData Amazon Athena Connection Manager and click Add...
  4. Configure the component as described in the next section.

Alternatively, if you have an existing project and CData Amazon Athena Source or CData Amazon Athena Destination:

  1. Right-click your CData Amazon Athena source or destination component in your data flow
  2. Select Edit... to open an editor window.
  3. Click the New... button next to the Connection manager: dropdown selector to create a connection manager.
  4. Configure the component as described in the next section.

Connecting to Amazon Athena

Specify the following to connect to data:
  • DataSource: The name of the Amazon Athena data source to connect to.
  • Database: The name of the Amazon Athena database to connect to.
  • AWSRegion: Set this to the region where your Amazon Athena data is hosted.
  • S3StagingDirectory: Set this to a folder in S3 where you would like to store the results of queries.

If Database and DataSource are not specified, the component tries to list all databases from the available data sources in Amazon Athena. Setting both properties improves the performance of the component.

Authenticating to Amazon Athena

In addition to specifying basic connection information required for all connections, you must choose an authentication method from one of the following options.

Obtain AWS Keys

To obtain the credentials for an IAM user:
  1. Sign into the IAM console.
  2. In the navigation pane, select Users.
  3. To create or manage the access keys for a user, select the user and then go to the Security Credentials tab.
To obtain the credentials for your AWS root account:
  1. Sign into the AWS Management console with the credentials for your root account.
  2. Select your account name or number.
  3. In the menu that displays, select My Security Credentials.
  4. To manage or create root account access keys, click Continue to Security Credentials and expand the "Access Keys" section.

Root Credentials

To authenticate using account root credentials, set these configuration parameters:

  • AuthScheme: AwsRootKeys.
  • AWSAccessKey: The access key associated with the AWS root account.
  • AWSSecretKey: The secret key associated with the AWS root account.

Note: Use of this authentication scheme is discouraged by Amazon for anything but simple tests. The account root credentials have the full permissions of the user, making this the least secure authentication method.

If multi-factor authentication is required, specify the following:

  • CredentialsLocation: The location of the settings file where MFA credentials are saved. See the Credentials File Location page under Connection String Options for more information.
  • MFASerialNumber: The serial number of the MFA device if one is being used.
  • MFAToken: The temporary token available from your MFA device.
This causes the component to submit the MFA credentials in the request to retrieve temporary authentication credentials.

Note: If you want to control the duration of the temporary credentials, set the TemporaryTokenDuration property (default: 3600 seconds).

Temporary Credentials

To authenticate using temporary credentials, specify the following:

  • AuthScheme: TemporaryCredentials.
  • AWSAccessKey: The access key of the IAM user to assume the role for.
  • AWSSecretKey: The secret key of the IAM user to assume the role for.
  • AWSSessionToken: Your AWS session token, provided with your temporary credentials. For details, see AWS Identity and Access Management User Guide.

The component can now request resources using the same permissions provided by long-term credentials (such as IAM user credentials) for the lifespan of the temporary credentials.

To authenticate using both temporary credentials and an IAM role, set all the parameters described above, and specify these additional parameters:

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This prompts the component to retrieve credentials for the specified role.
  • AWSExternalId (optional): Only required if you are assuming a role in another AWS account.

If multi-factor authentication is required, specify the following:

  • CredentialsLocation: The location of the settings file where MFA credentials are saved. See the Credentials File Location page under Connection String Options for more information.
  • MFASerialNumber: The serial number of the MFA device if one is being used.
  • MFAToken: The temporary token available from your MFA device.
This causes the component to submit the MFA credentials in the request to retrieve temporary authentication credentials.

Note: If you want to control the duration of the temporary credentials, set the TemporaryTokenDuration property (default: 3600 seconds).

EC2 Instances

Set AuthScheme to AwsEC2Roles.

If you are using the component from an EC2 Instance and have an IAM Role assigned to the instance, you can use the IAM Role to authenticate. Since the component automatically obtains your IAM Role credentials and authenticates with them, it is not necessary to specify AWSAccessKey and AWSSecretKey.

If you are also using an IAM role to authenticate, you must additionally specify the following:

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the component to attempt to retrieve credentials for the specified role.
  • AWSExternalId (optional): Only required if you are assuming a role in another AWS account.

IMDSv2 Support

The Amazon Athena component now supports IMDSv2. Unlike IMDSv1, the new version requires an authentication token. Endpoints and response are the same in both versions.

In IMDSv2, the Amazon Athena component first attempts to retrieve the IMDSv2 metadata token and then uses it to call AWS metadata endpoints. If it is unable to retrieve the token, the component reverts to IMDSv1.

AWS Web Identity

Set AuthScheme to AwsWebIdentity.

If you are using the component from a container configured to assume role with web identity (such as a Pod in an EKS cluster with an OpenID Provider) or have obtained an identity token by authenticating with a web identity provider associated with an IAM role, you can exchange the web identity token and IAM role information for temporary security credentials to authenticate and access AWS services. The component automatically obtains the credentials if the container has AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE specified in the environment variables. Alternatively, you can specify both AWSRoleARN and AWSWebIdentityToken to execute the AssumeRoleWithWebIdentity API operation and authenticate.

AWS IAM Roles

Set AuthScheme to AwsIAMRoles.

In many situations, it may be preferable to use an IAM role for authentication instead of the direct security credentials of an AWS root user. If you are specifying the AWSAccessKey and AWSSecretKey of an AWS root user, you may not use roles.

To authenticate as an AWS role, set these properties:

  • AWSAccessKey: The access key of the IAM user to assume the role for.
  • AWSSecretKey: The secret key of the IAM user to assume the role for.
  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the component to attempt to retrieve credentials for the specified role.
  • AWSExternalId (optional): Only required if you are assuming a role in another AWS account.

If multi-factor authentication is required, specify the following:

  • CredentialsLocation: The location of the settings file where MFA credentials are saved. See the Credentials File Location page under Connection String Options for more information.
  • MFASerialNumber: The serial number of the MFA device if one is being used.
  • MFAToken: The temporary token available from your MFA device.
This causes the component to submit the MFA credentials in the request to retrieve temporary authentication credentials.

Note: If you want to control the duration of the temporary credentials, set the TemporaryTokenDuration property (default: 3600 seconds).

ADFS

To connect to ADFS, set the AuthScheme to ADFS, and set these properties:

  • User: The ADFS user.
  • Password: The ADFS user's password.
  • SSOLoginURL: The SSO provider's login url.

Example connection string: 

AuthScheme=ADFS; AWSRegion=Ireland; Database=sampledb; [email protected]; Password=CH8WerW121235647iCa6; SSOLoginURL='https://adfs.domain.com'; AWSRoleArn=arn:aws:iam::1234:role/ADFS_SSO; AWSPrincipalArn=arn:aws:iam::1234:saml-provider/ADFSProvider; S3StagingDirectory=s3://athena/staging;

ADFS Integrated

The ADFS Integrated flow indicates you are connecting with the currently logged in Windows user credentials. To use the ADFS Integrated flow, do not specify the User and Password, but otherwise follow the same steps in the ADFS guide above.

Okta

To connect to Okta, set the AuthScheme to Okta, and set these properties:

  • User: The Okta user.
  • Password: The Okta user's password.
  • SSOLoginURL: The SSO provider's login URL.

If you are using a trusted application or proxy that overrides the Okta client request OR configuring MFA, you must use combinations of SSOProperties to authenticate using Okta. Set any of the following, as applicable:

  • APIToken: When authenticating a user via a trusted application or proxy that overrides the Okta client request context, set this to the API Token the customer created from the Okta organization.
  • MFAType: If you have configured the MFA flow, set this to one of the following supported types: OktaVerify, Email, or SMS.
  • MFAPassCode: If you have configured the MFA flow, set this to a valid passcode.
    If you set this to empty or an invalid value, the component issues a one-time password challenge to your device or email. After the passcode is received, reopen the connection where the retrieved one-time password value is set to the MFAPassCode connection property.
  • MFARememberDevice: True by default. Okta supports remembering devices when MFA is required. If remembering devices is allowed according to the configured authentication policies, the component sends a device token to extend MFA authentication lifetime. If you do not want MFA to be remembered, set this variable to False.

Example connection string: 

AuthScheme=Okta; AWSRegion=Ireland; Database=sampledb; [email protected]; Password=CH8WerW121235647iCa6; SSOLoginURL='https://cdata-us.okta.com/home/amazon_aws/0oa35m8arsAL5f5NrE6NdA356/272'; SSOProperties='ApiToken=01230GGG2ceAnm_tPAf4MhiMELXZ0L0N1pAYrO1VR-hGQSf;'; AWSRoleArn=arn:aws:iam::1234:role/Okta_SSO; AWSPrincipalARN=arn:aws:iam::1234:saml-provider/OktaProvider; S3StagingDirectory=s3://athena/staging;

To connect to PingFederate, set AuthScheme to PingFederate, and set these properties:

  • User: The PingFederate user.
  • Password: The PingFederate user's password.
  • SSOLoginURL: The SSO provider's login url.
  • AWSRoleARN (optional): If you have multiple role ARNs, specify the one you want to use for authorization.
  • AWSPrincipalARN (optional): If you have multiple principal ARNs, specify the one you want to use for authorization.
  • SSOExchangeUrl: The Partner Service Identifier URI configured in your PingFederate server instance under: SP Connections > SP Connection > WS-Trust > Protocol Settings. This should uniquely identify a PingFederate SP Connection, so it is a good idea to set it to your AWS SSO ACS URL. You can find it under AWS SSO > Settings > View Details next to the Authentication field.
  • SSOProperties (optional): Authscheme=Basic if you want to include your username and password as an authorization header in requests to Amazon S3.

To enable mutual SSL authentication for SSOLoginURL, the WS-Trust STS endpoint, configure these SSOProperties:

  • SSLClientCert
  • SSLClientCertType
  • SSLClientCertSubject
  • SSLClientCertPassword

Example connection string: 

authScheme=pingfederate;SSOLoginURL=https://mycustomserver.com:9033/idp/sts.wst;SSOExchangeUrl=https://us-east-1.signin.aws.amazon.com/platform/saml/acs/764ef411-xxxxxx;user=admin;password=PassValue;AWSPrincipalARN=arn:aws:iam::215338515180:saml-provider/pingFederate;AWSRoleArn=arn:aws:iam::215338515180:role/SSOTest2;AWSRegion=Ireland;S3StagingDirectory=s3://somedirectory/staging;Database=athenadatabase;

Credentials Files

You can use a credentials file to authenticate. Any configurations related to AccessKey/SecretKey authentication, temporary credentials, role authentication, or MFA can be used. To do so, set the following properties to authenticate:

  • AuthScheme: AwsCredentialsFile.
  • AWSCredentialsFile: The location of your credentials file.
  • AWSCredentialsFileProfile (optional): The name of the profile you would like to use from the specified credentials file. If not specified, the default profile is used.
For details, see AWS Command Line Interface User Guide.

AWS Cognito Credentials

If you want to use the component with a user registered in a User Pool in AWS Cognito, set the following properties to authenticate:

  • AuthScheme: Set this to AwsCognitoSrp (recommended). You can also use AwsCognitoBasic.
  • AWSCognitoRegion: Set this to the region of the User Pool.
  • AWSUserPoolId: Set this to the User Pool Id.
  • AWSUserPoolClientAppId: Set this to the User Pool Client App Id.
  • AWSUserPoolClientAppSecret: Set this to the User Pool Client Secret.
  • AWSIdentityPoolId: Set this to the Identity Pool Id of the Identity Pool that is linked with the User Pool.
  • User: Set this to the username of the user registered in the User Pool.
  • Password: Set this to the password of the user registered in the User Pool.

Azure Active Directory/Microsoft Entra ID

NOTE: Azure AD is now Microsoft Entra ID.

This configuration uses 2.0 On-Behalf-Of flow (Microsoft identity platform and OAuth 2.0 On-Behalf-Of flow ), which requires two Microsoft Entra applications:

  • the "Amazon Athena" application used for single sign-on
  • a separate "connector" application with user_impersonation permission on the "Amazon Athena" application

You must follow the steps outlined in Microsoft's AWS Single-Account Access documentation to create the SSO application and set Microsoft Entra ID as the SAML IdP for an AWS single-account app.

To create an application with user_impersonation permission on the SSO application, follow these steps:

  1. Sign into the Microsoft Entra admin center as at least a Cloud Application Administrator.
  2. Navigate to Identity > Applications > App registrations.
  3. Click New application at the top of the page.
  4. Enter a name and select supported account types as desired.
  5. Select Web in the Redirect URI dropdown, and enter http://localhost:33333.
  6. Select Register.
  7. In the "Overview" section, you can see your OAuthClientId (Application ID).
  8. Select API permissions > Add a permission.
  9. Select the API of your SSO application by specifying its name or Application Id in the "APIs my organization uses" tab.
  10. Select user_impersonation from the available list of Permissions.
  11. Select Add permissions.
  12. Select Certificates & secrets.
  13. Select New client secret.
  14. Optionally, enter a description and change the default Expires value, then select Add.
  15. Save your client secret (the Value field of the OAuth secret that is displayed).
After configuring both applications, specify the following OAuth connection properties to connect with the driver:

  • OAuthClientId: The application Id of the connector application, listed in the Overview section of the application registration.
  • OAuthClientSecret : The client secret value of the connector application. Azure AD displays this when you create a new client secret.
  • CallbackURL: Set this to the redirect URI of the connector application. For example: https://localhost:33333.
  • InitiateOAuth: GETANDREFRESH.

Use the following SSOProperties to authenticate to AzureAD:

  • Resource: The application Id URI of the Amazon Athena application, listed in the Overview section of the application registration. In most cases this is the URL of your custom Amazon Athena domain.
  • AzureTenant: The Id of the Azure AD tenant where the applications are registered.

The following is an example connection string: 

AuthScheme=AzureAD;InitiateOAuth=GETANDREFRESH;OAuthClientId=3ea1c786-d527-4399-8c3b-2e3696ae4b48;OauthClientSecret=xxx;CallbackUrl=https://localhost:33333;SSOProperties='Resource=https://signin.aws.amazon.com/saml;AzureTenant=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx';

Copyright (c) 2024 CData Software, Inc. - All rights reserved.
Build 24.0.9060