Cmdlets for Amazon Athena

Build 20.0.7654

Establishing a Connection

With the CData Cmdlets users can install a data module, set the connection properties, and start scripting. This section provides examples of using our AmazonAthena Cmdlets with native PowerShell cmdlets, like the CSV import and export cmdlets.

Installing and Connecting

If you have PSGet, installing the cmdlets can be accomplished from the PowerShell Gallery with the following command. You can also obtain a setup from the CData site.

Install-Module AmazonAthenaCmdlets

The following line is then added to your profile, loading the cmdlets on the next session:

Import-Module AmazonAthenaCmdlets;

You can then use the Connect-AmazonAthena cmdlet to create a connection object that can be passed to other cmdlets:

$conn = Connect-AmazonAthena AWSAccessKey "a123" -AWSSecretKey "s123" -AWSRegion "IRELAND" -Database "sampledb" -S3StagingDirectory "s3://bucket/staging/"

Obtaining the Access Key

To obtain the credentials for an IAM user, follow the steps below:

  1. Sign into the IAM console.
  2. In the navigation pane, select Users.
  3. To create or manage the access keys for a user, select the user and then select the Security Credentials tab.

To obtain the credentials for your AWS root account, follow the steps below:

  1. Sign into the AWS Management console with the credentials for your root account.
  2. Select your account name or number and select My Security Credentials in the menu that is displayed.
  3. Click Continue to Security Credentials and expand the Access Keys section to manage or create root account access keys.

Authenticating with Root Credentials

To authenticate using account root credentials, set the following:

  • AuthScheme: Set this to AwsRootKeys.
  • AWSAccessKey: The access key associated with the AWS root account.
  • AWSSecretKey: The secret key associated with the AWS root account.

Note: Use of this authentication scheme is discouraged by Amazon for anything but simple tests. The account root credentials have the full permissions of the user, making this the least secure authentication method.

Authenticating with Temporary Credentials

To authenticate using temporary credentials, specify the following:

  • AuthScheme: Set this to TemporaryCredentials.
  • AWSAccessKey: The access key of the IAM user to assume the role for.
  • AWSSecretKey: The secret key of the IAM user to assume the role for.
  • AWSSessionToken: Your AWS session token. This will have been provided alongside your temporary credentials. See this link for more info.

The cmdlet can now request resources using the same permissions provided by long-term credentials (such as IAM user credentials) for the lifespan of the temporary credentials.

If you are also using an IAM role to authenticate, you must additionally specify the following:

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the cmdlet to attempt to retrieve credentials for the specified role.
  • AWSExternalId: Only if required when you assume a role in another account.

Authenticating from an EC2 Instance

If you are using the cmdlet from an EC2 Instance and have an IAM Role assigned to the instance, you can use the IAM Role to authenticate. To do so, set the following properties to authenticate:

  • AuthScheme: Set this to AwsEC2Roles.
Leave AWSAccessKey and AWSSecretKey empty. The cmdlet will automatically obtain your IAM Role credentials and authenticate with them.

If you are also using an IAM role to authenticate, you must additionally specify the following:

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the cmdlet to attempt to retrieve credentials for the specified role.
  • AWSExternalId: Only if required when you assume a role in another account.

Authenticating as an AWS Role

In many situations it may be preferable to use an IAM role for authentication instead of the direct security credentials of an AWS root user.

To authenticate as an AWS role, set the following:

  • AuthScheme: Set this to AwsIAMRoles.
  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the cmdlet to attempt to retrieve credentials for the specified role.
  • AWSExternalId: Only if required when you assume a role in another account.
If you are connecting to AWS (instead of already being connected such as on an EC2 instance), you must additionally specify the following:
  • AWSAccessKey: The access key of the IAM user to assume the role for.
  • AWSSecretKey: The secret key of the IAM user to assume the role for.

Note: Roles may not be used when specifying the AWSAccessKey and AWSSecretKey of an AWS root user.

Authenticating with SSO

For users and roles that require SSO Authentication, specify the following to authenticate:

  • AuthScheme: Set this to either AwsOKTA or AwsADFS based on the identity provider you would like to use.
  • AWSRoleARN: The Amazon Resource Name of the role to use when authenticating.
  • AWSPrincipalARN: The ARN of the SAML Identity provider in your AWS account.
  • SSOProperties: The SSO properties specific to each Identity Provider. See the SSOProperties page under Connection String Options for more information.
Leave AWSAccessKey and AWSSecretKey empty.

This will cause the cmdlet to submit the SSO credentials in a request to retrieve temporary authentication credentials. Note that the duration of the temporary credentials may be controlled via the TemporaryTokenDuration property (default 3600 seconds).

See SSO Connections for further details.

Authenticating with MFA

For users and roles that require Multi-factor Authentication, specify the following to authenticate:

  • AuthScheme: Set this to AwsMFA.
  • CredentialsLocation: The location of the settings file where MFA credentials are saved. See the Credentials File Location page under Connection String Options for more information.
  • MFASerialNumber: The serial number of the MFA device if one is being used.
  • MFAToken: The temporary token available from your MFA device.
If you are connecting to AWS (instead of already being connected such as on an EC2 instance), you must additionally specify the following:
  • AWSAccessKey: The access key of the IAM user for whom MFA will be issued.
  • AWSSecretKey: The secret key of the IAM user whom MFA will be issued.
If you are also using an IAM role to authenticate, you must additionally specify the following:
  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the cmdlet to attempt to retrieve credentials for the specified role using MFA.
  • AWSExternalId: Only if required when you assume a role in another account.
This will cause the cmdlet to submit the MFA credentials in a request to retrieve temporary authentication credentials.

Note that the duration of the temporary credentials may be controlled via the TemporaryTokenDuration property (default 3600 seconds).

Authenticating with a Credential File

You may use a credentials file to authenticate. Any configurations related to AccessKey/SecretKey authentication, temporary credentials, role authentication, or MFA will be used. To do so, set the following properties to authenticate:

  • AuthScheme: Set this to AwsCredentialsFile.
  • AWSCredentialsFile: Set this to the location of your credentials file.
  • AWSCredentialsFileProfile: Optionally set this to the name of the profile you would like to use from the specified credentials file. If not specified, the profile with the name default will be used.
See this link for more information.

Connecting to Amazon Athena

Once you have configured the cmdlet for your desired authentication method, specify the following to connect to data:

  • Database: The name of the Amazon Athena database to connect to.
  • S3StagingDirectory: Specifies the location Amazon Athena will use to store the results of a query.
  • AWSRegion: Set this to the region where your Amazon Athena data is hosted.
  • S3StagingDirectory: Set this to a folder in S3 where you would like to store the results of queries.

If Database is not set in the connection, the cmdlet connects to the default database set in Amazon Athena.

Retrieving Data

The Select-AmazonAthena cmdlet provides a native PowerShell interface for retrieving data:

$results = Select-AmazonAthena -Connection $conn -Table "Customers" -Columns @("Name, TotalDue") -Where "CustomerId='12345'"
The Invoke-AmazonAthena cmdlet provides an SQL interface. This cmdlet can be used to execute an SQL query via the Query parameter.

Piping Cmdlet Output

The cmdlets return row objects to the pipeline one row at a time. The following line exports results to a CSV file:

Select-AmazonAthena -Connection $conn -Table Customers -Where "CustomerId = '12345'" | Select -Property * -ExcludeProperty Connection,Table,Columns | Export-Csv -Path c:\myCustomersData.csv -NoTypeInformation

You will notice that we piped the results from Select-AmazonAthena into a Select-Object cmdlet and excluded some properties before piping them into an Export-CSV cmdlet. We do this because the CData Cmdlets append Connection, Table, and Columns information onto each row object in the result set, and we do not necessarily want that information in our CSV file.

However, this makes it easy to pipe the output of one cmdlet to another. The following is an example of converting a result set to JSON:

 
PS C:\> $conn  = Connect-AmazonAthena AWSAccessKey "a123" -AWSSecretKey "s123" -AWSRegion "IRELAND" -Database "sampledb" -S3StagingDirectory "s3://bucket/staging/"
PS C:\> $row = Select-AmazonAthena -Connection $conn -Table "Customers" -Columns (Name, TotalDue) -Where "CustomerId = '12345'" | select -first 1
PS C:\> $row | ConvertTo-Json
{
  "Connection":  {

  },
  "Table":  "Customers",
  "Columns":  [

  ],
  "Name":  "MyName",
  "TotalDue":  "MyTotalDue"
} 

Copyright (c) 2020 CData Software, Inc. - All rights reserved.
Build 20.0.7654