Amazon S3
Version 24.3.9110
Version 24.3.9110
Amazon S3
You can use the Amazon S3 connector from the CData Sync application to capture data from Amazon S3 and move it to any supported destination. To do so, you need to add the connector, authenticate to the connector, and complete your connection.
Add the Amazon S3 Connector
To enable Sync to use data from Amazon S3, you first must add the connector, as follows:
-
Open the Connections page of the Sync dashboard.
-
Click Add Connection to open the Select Connectors page.
-
Click the Sources tab and locate the Amazon S3 row.
-
Click the Configure Connection icon at the end of that row to open the New Connection page. If the Configure Connection icon is not available, click the Download Connector icon to install the Amazon S3 connector. For more information about installing new connectors, see Connections.
Authenticate to Amazon S3
After you add the connector, you need to set the required properties.
-
Connection Name - Enter a connection name of your choice.
-
File Format - Select the file format that you want to use. Sync supports the CSV, PARQUET, and AVRO file formats.
-
URI - Enter the path of your bucket and folder (for example,
s3://BucketName/FolderName
).
CData Sync supports authenticating to Amazon S3 in several ways. Select your authentication method below to proceed to the relevant section that contains the authentication details.
- Active Directory Federation Services
- AWS Root Keys (default)
- AWS IAM Roles
- AWS EC2 Roles
- AWS Multi-Factor Authentication
- AWS Credentials File
- Okta
- PingFederate
- AWS Cognito Basic
- AWS Cognito Secure Remote Password
Active Directory Federation Services
To connect with single sign-on (SSO) via ADFS, specify the following properties:
-
Auth Scheme - Select ADFS.
-
User - Enter the username that you use to authenticate to your ADFS account.
-
Password - Enter the password that you use to authenticate to your ADFS account.
-
SSO Login URL - Enter the login URL that is used by your SSO provider.
AWS Root Keys
To connect with your account root credentials, specify the following properties:
-
Auth Scheme - Select AwsRootKeys.
-
AWS Access Key - Enter your Amazon Web Services (AWS) account access key. You can locate this value on your AWS security credentials page.
-
AWS Secret Key - Enter your AWS account secret key. You can locate this value on your AWS security credentials page.
AWS IAM Roles
To connect with your IAM user credentials, specify the following properties:
-
Auth Scheme - Select AwsIAMRoles.
-
AWS Access Key - Enter your Amazon Web Services (AWS) account access key. You can locate this value on your AWS security credentials page.
-
AWS Secret Key - Enter your AWS account secret key. You can locate this value on your AWS security credentials page.
-
AWS Role ARN - Enter the Amazon Resource Name (ARN) for the role with which you want to authenticate.
-
AWS External Id (optional) - Enter the unique identifier that is required when you assume a role in another account.
AWS EC2 Roles
When you run CData Sync on an EC2 instance, CData Sync can authenticate by using the IAM role that is assigned to the instance. Select AwsEC2Roles for Auth Scheme to use that role. No additional properties are required.
AWS Multi-Factor Authentication
To connect with your multifactor authentication credentials, specify the following properties:
-
Auth Scheme - Select AwsMFA.
-
MFA Serial Number - Enter the serial number for your multifactor authentication (MFA) device, if you are using such a device.
-
MFA Token - Enter the temporary token that is available from your MFA device.
-
Temporary Token Duration (optional) - Enter the amount of time (in seconds) a temporary token will last. The default duration is 3600.
AWS Credentials File
To connect with a credentials file, specify the following properties:
-
Auth Scheme - Select AwsCredentialsFile.
-
AWS Credentials File - Enter the location of your Amazon Web Services (AWS) credentials file.
-
AWS Credentials File Profile (optional) - Enter the name of the AWS profile that you want to use from the credentials file that you specify. If you do not enter a profile name, Sync uses the profile named default.
Okta
To connect with single sign-on (SSO) via Okta, specify the following properties:
-
Auth Scheme – Select OKTA.
-
User - Enter the username that you use to authenticate to your Okta account.
-
Password - Enter the password that you use to authenticate to your Okta account.
-
SSO Login URL - Enter the login URL that is used by your SSO provider.
-
SSO Properties - Enter additional properties that are required to connect to your identity provider. You must use a semicolon-separated list of properties (for example, SSOProperty1=Value1;SSOProperty2=Value2;…).
PingFederate
To connect with single sign-on via PingFederate, specify the following properties:
-
Auth Scheme - Select PingFederate.
-
User - Enter the username that you use to authenticate to your PingFederate account.
-
Password - Enter the password that you use to authenticate to your PingFederate account.
-
SSO Login URL - Enter the login URL that is used by your SSO provider.
-
SSO Properties - Enter a comma-separated list of the single sign-on (SSO) properties that you want to use.
-
SSO Exchange UrI - Enter the Partner Service Identifier URI that is configured in your PingFederate server instance. The URI is available under SP Connections > SP Connection > WS-Trust > Protocol Settings.
AwsCognitoBasic
To connect with your Amazon Web Services (AWS) Cognito credentials, specify the following properties:
-
Auth Scheme - Select AwsCognitoBasic.
-
AWS Cognito Region - Select the region for your user pool.
-
AWS User Pool Id - Enter your Amazon Web Services (AWS) user-pool Id.
-
AWS User Pool Client App Id - Enter your AWS user-pool client application Id.
-
AWS Identity Pool Id - Enter the identity-pool Id of the that is linked with your user pool.
-
AWS User Pool Client App Secret (optional) - Enter the secret for your user-pool client.
AwsCognitoSrp
To connect with your Amazon Web Services (AWS) Cognito credentials, specify the following properties:
-
Auth Scheme - Select AwsCognitoSrp.
-
AWS Cognito Region - Select the region for your user pool.
-
AWS User Pool Id - Enter your Amazon Web Services (AWS) user-pool Id.
-
AWS User Pool Client App Id - Enter your AWS user-pool client application Id.
-
AWS Identity Pool Id - Enter the identity-pool Id of the that is linked with your user pool.
-
AWS User Pool Client App Secret (optional) - Enter the secret for your user-pool client.
Complete Your Connection
To complete your connection:
-
For AWS Region, select the hosting region for Amazon Web Services.
-
Define advanced connection settings on the Advanced tab. (In most cases, though, you should not need these settings.)
-
Click Create & Test to create your connection.
More Information
For more information about interactions between CData Sync and Amazon S3, see Amazon S3 Connector for CData Sync.