BizTalk Adapter for Amazon Athena

Build 21.0.7929

Connection Properties

Connection String Options  

The connection string properties describe the various options that can be used to establish a connection.

Adapter

CData.BizTalk.AmazonAthenaAdapter.dll

Remarks

The connection string can be set to a series of "option=value" pairs, separated by semicolons. If a connection string property value has special characters such as semicolons, single quotes, spaces, etc., then you must quote the value using either single or double quotes.

Connection options are case insensitive.

Connecting to Amazon Athena

Specify the following to connect to data:

  • DataSource: The name of the Amazon Athena data source to connect to.
  • Database: The name of the Amazon Athena database to connect to.
  • AWSRegion: Set this to the region where your Amazon Athena data is hosted.
  • S3StagingDirectory: Set this to a folder in S3 where you would like to store the results of queries.

If Database or DataSource is not set in the connection, the adapter connects to the default data source and database set in Amazon Athena.

Authenticating to Amazon Athena

In addition to specifying basic connection information which will be necessary for all connections, you will need to choose an authentication method from one of the below listed options.

Obtaining the Access Key

To obtain the credentials for an IAM user, follow the steps below:

  1. Sign into the IAM console.
  2. In the navigation pane, select Users.
  3. To create or manage the access keys for a user, select the user and then select the Security Credentials tab.

To obtain the credentials for your AWS root account, follow the steps below:

  1. Sign into the AWS Management console with the credentials for your root account.
  2. Select your account name or number and select My Security Credentials in the menu that is displayed.
  3. Click Continue to Security Credentials and expand the Access Keys section to manage or create root account access keys.

Authenticating with Root Credentials

To authenticate using account root credentials, set the following:

Note: Use of this authentication scheme is discouraged by Amazon for anything but simple tests. The account root credentials have the full permissions of the user, making this the least secure authentication method.

Authenticating with Temporary Credentials

To authenticate using temporary credentials, specify the following:

  • AuthScheme: Set this to TemporaryCredentials.
  • AWSAccessKey: The access key of the IAM user to assume the role for.
  • AWSSecretKey: The secret key of the IAM user to assume the role for.
  • AWSSessionToken: Your AWS session token. This will have been provided alongside your temporary credentials. See this link for more info.

The adapter can now request resources using the same permissions provided by long-term credentials (such as IAM user credentials) for the lifespan of the temporary credentials.

If you are also using an IAM role to authenticate, you must additionally specify the following:

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the adapter to attempt to retrieve credentials for the specified role.
  • AWSExternalId: Only if required when you assume a role in another account.

Authenticating from an EC2 Instance

If you are using the adapter from an EC2 Instance and have an IAM Role assigned to the instance, you can use the IAM Role to authenticate. To do so, set the following properties to authenticate:

Do not specify AWSAccessKey and AWSSecretKey because the adapter will automatically obtain your IAM Role credentials and authenticate with them.

If you are also using an IAM role to authenticate, you must additionally specify the following:

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the adapter to attempt to retrieve credentials for the specified role.
  • AWSExternalId: Only if required when you assume a role in another account.

Authenticating as an AWS Role

In many situations it may be preferable to use an IAM role for authentication instead of the direct security credentials of an AWS root user.

To authenticate as an AWS role, set the following:

  • AuthScheme: Set this to AwsIAMRoles.
  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the adapter to attempt to retrieve credentials for the specified role.
  • AWSExternalId: Only if required when you assume a role in another account.
  • AWSAccessKey: The access key of the IAM user to assume the role for.
  • AWSSecretKey: The secret key of the IAM user to assume the role for.

Note: Roles may not be used when specifying the AWSAccessKey and AWSSecretKey of an AWS root user.

Authenticating with ADFS

Set the AuthScheme to ADFS. The following connection properties need to be set:

Below is an example connection string:
AuthScheme=ADFS; AWSRegion=Ireland; Database=sampledb; User=user@cdata.com; Password=CH8WerW121235647iCa6; SSOLoginUrl='https://adfs.domain.com'; AWSRoleArn=arn:aws:iam::1234:role/ADFS_SSO; AWSPrincipalArn=arn:aws:iam::1234:saml-provider/ADFSProvider; S3StagingDirectory=s3://athena/staging;

ADFS Integrated

The ADFS Integrated flow indicates you are connecting with the currently logged in Windows user credentials. To use the ADFS Integrated flow, simply do not specify the User and Password, but otherwise follow the same steps in the ADFS guide above.

Authenticating with Okta

Set the AuthScheme to Okta. The following connection properties are used to connect to Okta:

The following SSOProperties are needed to authenticate to Okta:
  • APIToken (optional): Set this to the API Token that the customer created from the Okta org. It should be used when authenticating a user via a trusted application or proxy that overrides OKTA client request context.

The following is an example connection string:

AuthScheme=Okta; AWSRegion=Ireland; Database=sampledb; User=user@cdata.com; Password=CH8WerW121235647iCa6; SSOLoginUrl='https://cdata-us.okta.com/home/amazon_aws/0oa35m8arsAL5f5NrE6NdA356/272'; SSOProperties='ApiToken=01230GGG2ceAnm_tPAf4MhiMELXZ0L0N1pAYrO1VR-hGQSf;'; AWSRoleArn=arn:aws:iam::1234:role/Okta_SSO; AWSPrincipalARN=arn:aws:iam::1234:saml-provider/OktaProvider; S3StagingDirectory=s3://athena/staging;

Authenticating with PingFederate

Set the AuthScheme to PingFederate. The following connection properties need to be set:

  • User: Set this to the PingFederate user.
  • Password: Set this to PingFederate password for the user.
  • RSBAmazonAthena_p_SSOLoginURL: Set this to the login url used by the SSO provider.
  • SSOExchangeUrl: The 'Partner Service Identifier' URI configured in your PingFederate server instance under: SP Connections > SP Connection > WS-Trust > Protocol Settings. This should uniquely identify a PingFederate SP Connection, so it is a good idea to set it to your 'AWS SSO ACS URL'. You can find it under AWS SSO > Settings > Click on 'View Details' next to the Authentication field.
The following SSOProperties are needed to authenticate to PingFederate:
  • AuthScheme (optional): The authorization scheme to be used for the IdP endpoint. The allowed values for this IdP are None or Basic.
Additionally, you can use the following SSOProperties to configure mutual SSL authentication for RSBAmazonAthena_p_SSOLoginURL, the WS-Trust STS endpoint: Below is an example connection string:
authScheme=pingfederate;SSOLoginUrl=https://mycustomserver.com:9033/idp/sts.wst;SSOExchangeUrl=https://us-east-1.signin.aws.amazon.com/platform/saml/acs/764ef411-xxxxxx;user=admin;password=PassValue;AWSPrincipalARN=arn:aws:iam::215338515180:saml-provider/pingFederate;AWSRoleArn=arn:aws:iam::215338515180:role/SSOTest2;AWSRegion=Ireland;S3StagingDirectory=s3://somedirectory/staging;Database=athenadatabase;

Authenticating with MFA

For users and roles that require Multi-factor Authentication, specify the following to authenticate:

  • AuthScheme: Set this to AwsMFA.
  • CredentialsLocation: The location of the settings file where MFA credentials are saved. See the Credentials File Location page under Connection String Options for more information.
  • MFASerialNumber: The serial number of the MFA device if one is being used.
  • MFAToken: The temporary token available from your MFA device.
If you are connecting to AWS (instead of already being connected such as on an EC2 instance), you must additionally specify the following:
  • AWSAccessKey: The access key of the IAM user for whom MFA will be issued.
  • AWSSecretKey: The secret key of the IAM user whom MFA will be issued.
If you are also using an IAM role to authenticate, you must additionally specify the following:
  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the adapter to attempt to retrieve credentials for the specified role using MFA.
  • AWSExternalId: Only if required when you assume a role in another account.
This will cause the adapter to submit the MFA credentials in a request to retrieve temporary authentication credentials.

Note that the duration of the temporary credentials may be controlled via the TemporaryTokenDuration property (default 3600 seconds).

Authenticating with a Credential File

You may use a credentials file to authenticate. Any configurations related to AccessKey/SecretKey authentication, temporary credentials, role authentication, or MFA will be used. To do so, set the following properties to authenticate:

  • AuthScheme: Set this to AwsCredentialsFile.
  • AWSCredentialsFile: Set this to the location of your credentials file.
  • AWSCredentialsFileProfile: Optionally set this to the name of the profile you would like to use from the specified credentials file. If not specified, the profile with the name default will be used.
See this link for more information.

Connection String Options


The connection string options available in this provider are divided into the following categories. Click on the links for further details.

Authentication


PropertyDescription
AthenaVPCEndpointYour Athena VPC Endpoint. It will override the default athena.{region}.amazonaws.com endpoint.
S3StagingDirectorySpecifies the location Amazon Athena will use to store the results of a query.
EncryptionTypeSpecifies the encryption option for query results in Athena.
EncryptionKeySpecifies the encryption key when encrypting results stored in S3.
DatabaseThe name of the Athena database.
DataSourceThe name of the Athena Data Source.

Connection


PropertyDescription
SimpleUploadLimitThis setting specifies the threshold, in bytes, above which the provider will choose to perform a multipart upload rather than uploading everything in one request.

AWSAuthentication


PropertyDescription
AuthSchemeThe scheme used for authentication. Accepted entries are: Auto, , AwsRootKeys , AwsIAMRoles , AwsEC2Roles , AwsMFA , ADFS, Okta, PingFederate , AwsCredentialsFile.
AWSAccessKeyYour AWS account access key. This value is accessible from your AWS security credentials page.
AWSSecretKeyYour AWS account secret key. This value is accessible from your AWS security credentials page.
AWSRoleARNThe Amazon Resource Name of the role to use when authenticating.
AWSPrincipalArnThe ARN of the SAML Identity provider in your AWS account.
AWSRegionThe hosting region for your Amazon Web Services.
AWSCredentialsFileThe path to the AWS Credentials File to be used for authentication.
AWSCredentialsFileProfileThe name of the profile to be used from the supplied AWSCredentialsFile.
AWSSessionTokenYour AWS session token.
AWSExternalIdA unique identifier that might be required when you assume a role in another account.
MFASerialNumberThe serial number of the MFA device if one is being used.
MFATokenThe temporary token available from your MFA device.
CredentialsLocationThe location of the settings file where MFA credentials are saved.
TemporaryTokenDurationThe amount of time (in seconds) a temporary token will last.

SSO


PropertyDescription
UserThe IDP user used to authenticate the IDP via SSO.
PasswordThe password used to authenticate the IDP user via SSO.
SSOLoginURLThe identity provider's login URL.
SSOPropertiesAdditional properties required to connect to the identity provider in a semicolon-separated list.
SSOExchangeUrlThe url used for consuming the SAML response and exchanging it with Amazon Athena specific credentials.

SSL


PropertyDescription
SSLServerCertThe certificate to be accepted from the server when connecting using TLS/SSL.

Firewall


PropertyDescription
FirewallTypeThe protocol used by a proxy-based firewall.
FirewallServerThe name or IP address of a proxy-based firewall.
FirewallPortThe TCP port for a proxy-based firewall.
FirewallUserThe user name to use to authenticate with a proxy-based firewall.
FirewallPasswordA password used to authenticate to a proxy-based firewall.

Proxy


PropertyDescription
ProxyAutoDetectThis indicates whether to use the system proxy settings or not. This takes precedence over other proxy settings, so you'll need to set ProxyAutoDetect to FALSE in order use custom proxy settings.
ProxyServerThe hostname or IP address of a proxy to route HTTP traffic through.
ProxyPortThe TCP port the ProxyServer proxy is running on.
ProxyAuthSchemeThe authentication type to use to authenticate to the ProxyServer proxy.
ProxyUserA user name to be used to authenticate to the ProxyServer proxy.
ProxyPasswordA password to be used to authenticate to the ProxyServer proxy.
ProxySSLTypeThe SSL type to use when connecting to the ProxyServer proxy.
ProxyExceptionsA semicolon separated list of destination hostnames or IPs that are exempt from connecting through the ProxyServer .

Logging


PropertyDescription
LogfileA filepath which designates the name and location of the log file.
LogModulesCore modules to be included in the log file.
MaxLogFileSizeA string specifying the maximum size in bytes for a log file (for example, 10 MB).
MaxLogFileCountA string specifying the maximum file count of log files.

Schema


PropertyDescription
LocationA path to the directory that contains the schema files defining tables, views, and stored procedures.
BrowsableSchemasThis property restricts the schemas reported to a subset of the available schemas. For example, BrowsableSchemas=SchemaA,SchemaB,SchemaC.
TablesThis property restricts the tables reported to a subset of the available tables. For example, Tables=TableA,TableB,TableC.
ViewsRestricts the views reported to a subset of the available tables. For example, Views=ViewA,ViewB,ViewC.

Caching


PropertyDescription
AutoCacheAutomatically caches the results of SELECT queries into a cache database specified by either CacheLocation or both of CacheConnection and CacheProvider .
CacheProviderThe name of the provider to be used to cache data.
CacheConnectionThe connection string for the cache database. This property is always used in conjunction with CacheProvider . Setting both properties will override the value set for CacheLocation for caching data.
CacheLocationSpecifies the path to the cache when caching to a file.
CacheToleranceThe tolerance for stale data in the cache specified in seconds when using AutoCache .
OfflineUse offline mode to get the data from the cache instead of the live source.
CacheMetadataThis property determines whether or not to cache the table metadata to a file store.

Miscellaneous


PropertyDescription
CleanQueryResultsAmazon Athena produces cache files with every query, in the folder specified in S3StagingDirectory . CleanQueryResults specifies whether these files should be deleted once the connection is closed.
EnableFIPSModeA boolean indicating if FIPS URLs should be enabled.
IncludeTableTypesIf set to true, the provider will query for the types of individual tables.
MaximumColumnSizeThe maximum column size.
MaxRowsLimits the number of rows returned rows when no aggregation or group by is used in the query. This helps avoid performance issues at design time.
MetadataDiscoveryMethodAPI to use to retrieve the metadata for Amazon Athena.
OtherThese hidden properties are used only in specific use cases.
PageSizeThe number of results to return per page of data retrieved from Amazon Athena.
PollingIntervalThis determines the polling interval in milliseconds to check whether the result is ready to be retrieved.
PseudoColumnsThis property indicates whether or not to include pseudo columns as columns to the table.
QueryCachingLevelModifies the usage of query results stored in S3StagingDirectory . Cached queries will not be scanned again by Amazon Athena until the amount of seconds specified in QueryTolerance has passed.
QueryPassthroughThis option passes the query to the Amazon Athena server as is.
QueryTimeoutThe timeout in seconds for requests issued by the provider to download large result sets.
QueryToleranceSpecifies the amount of time in seconds queries cache for if QueryCachingLevel is not set to None.
RTKThe runtime key used for licensing.
ShowS3FilepathWhether or not to expose the _S3Path column for the Amazon Athena tables.
SkipHeaderLineCountSpecifies the number of header rows to skip for SELECT queries.
TimeoutThe value in seconds until the timeout error is thrown, canceling the operation.
WorkgroupWork group of the executed queries.

Copyright (c) 2021 CData Software, Inc. - All rights reserved.
Build 21.0.7929