Search

REST

Version 26.2.9623

REST

You can use the REST connector from the CData Sync application to capture data from REST and move it to any supported destination. To do so, you need to add the connector, authenticate to the connector, and complete your connection.

Add the REST Connector

To enable Sync to use data from REST, you first must add the connector, as follows:

  1. Open the Connections page of the Sync dashboard.

  2. Click Add Connection to open the Select Connectors page.

  3. Click the Sources tab and locate the REST row.

  4. Click the Configure Connection icon at the end of that row to open the New Connection page. This action opens the Add Connection dialog box.

    Note: If the Configure Connection icon is not available, click the Download Connector icon to install the REST connector.

  5. Enter a name for your connection in the Add Connection dialog box.

  6. Click Add to open the Settings tab for your connector.

For more information about installing new connectors, see Connections.

Authenticate to REST

After you add the connector, you need to set the required properties.

  • Api Key: Enter the API key that identifies the user to IBM Cloud.

  • URI: Enter the Uniform Resource Identifier (URI) for your resource location.

  • Region: Enter the hosting region for your S3-like web services.

  • Oracle Namespace: Enter the Oracle Cloud Object Storage namespace to use.

  • Azure Storage Account: Enter the name of your Azure storage account.

CData Sync supports authenticating to REST in several ways. Select your authentication method below to proceed to the relevant section that contains the authentication details.

Azure Access Key

To connect with an Azure access key, set the following properties:

  • Auth Scheme: Select Access Key.

  • Azure Access Key: Enter the access key that is associated with your storage account.

To retrieve your access key:

  1. Sign in to the Azure portal with the credentials for your root account.

  2. Click Storage accounts and select the storage account that you want to use.

  3. Under Settings, click Access keys. Your storage account name and key are displayed on that page.

AWS Credentials File

To connect with a credentials file, specify the following properties:

  • Auth Scheme: Select AwsCredentialsFile.

  • AWS Credentials File: Enter the location of your Amazon Web Services (AWS) credentials file.

  • (Optional) AWS Credentials File Profile: Enter the name of the AWS profile that you want to use from the credentials file that you specify. If you do not enter a profile name, Sync uses the profile named default.

Azure Active Directory

To connect with an Azure Active Directory (AD) user account, specify the following properties:

  • Auth Scheme: Select AzureAD.

  • Use Lake Formation: Select the Enable checkbox if you want the AWS Lake Formation service to retrieve temporary credentials. These temporary credentials enforce access policies against the user based on the configured IAM role. You can use this service when you authenticate through AzureAD, Okta, ADFS, and PingFederate, while providing a Security Assertion Markup Language (SAML) assertion. By default, the Enable checkbox is not selected.

  • OAuth Client Id: Enter the client Id that you were assigned when you registered your application with an OAuth authorization server.

  • OAuth Client Secret: Enter the client secret that you were assigned when you registered your application with an OAuth authorization server.

Azure Managed Service Identity

To leverage Azure Managed Service Identity (MSI) when CData Sync is running on an Azure virtual machine, select Azure MSI for Auth Scheme. No additional properties are required.

Azure Service Principal

To connect with an Azure service principal and client secret, set the following properties:

  • Auth Scheme: Select AzureServicePrincipal.

  • Azure Tenant: Enter the Microsoft Online tenant to which you want to connect.

  • OAuth Client Id: Enter the client Id that you were assigned when you registered your application with an OAuth authorization server.

  • OAuth Client Secret: Enter the client secret that you were assigned when you registered your application with an OAuth authorization server.

  • (Optional) Scope: Specify the scope of your access to the application.

  • (Optional)OAuth Authorization URL: Enter the OAuth authorization URL for the OAuth service.

  • (Optional) OAuth Access Token URL: Enter the URL from which to retrieve the access token.

  • (Optional) OAuth Refresh Token URL: Enter the URL from which to refresh the OAuth token.

To obtain the OAuth client Id and client secret for your application:

  1. Log in to the Azure portal.

  2. In the left navigation pane, select All services. Then, search for and select App registrations.

  3. Click New registrations.

  4. Enter an application name and select Any Azure AD Directory - Multi Tenant.

  5. After you create the application, copy the application (client) Id value that is displayed in the Overview section. Use this value as the OAuth client Id.

  6. Navigate to the Certificates & Secrets section and select New Client Secret for the application.

  7. Specify the duration and save the client secret. After you save it, the key value is displayed.

  8. Copy this value because it is displayed only once. You will use this value as the OAuth client secret.

  9. On the Authentication tab, make sure to select Access tokens (used for implicit flows).

Azure Service Principal Certificate

To connect with an Azure service principal and client certificate, set the following properties:

  • Auth Scheme: Select AzureServicePrincipalCert.

  • Azure Tenant: Enter the Microsoft Online tenant to which you want to connect.

  • OAuth JWT Cert: Enter your Java web tokens (JWT) certificate store.

  • OAuth JWT Cert Type: Enter the type of key store that contains your JWT Certificate. The default type is PEMKEY_BLOB.

  • (Optional) OAuth Client Id: Enter the client Id that you were assigned when you registered your application with an OAuth authorization server.

  • (Optional) Scope: Specify the scope of your access to the application.

  • (Optional)OAuth Authorization URL: Enter the OAuth authorization URL for the OAuth service.

  • (Optional) OAuth Access Token URL: Enter the URL from which to retrieve the access token.

  • (Optional) OAuth Refresh Token URL: Enter the URL from which to refresh the OAuth token.

  • (Optional) OAuth JWT Cert Password: Enter the password for your OAuth JWT certificate.

  • (Optional) OAuth JWT Cert Subject: Enter the subject of your OAuth JWT certificate.

To obtain the OAuth certificate for your application:

  1. Log in to the Azure portal.

  2. In the left navigation pane, select All services. Then, search for and select App registrations.

  3. Click New registrations.

  4. Enter an application name and select Any Azure AD Directory - Multi Tenant.

  5. After you create the application, copy the application (client) Id value that is displayed in the Overview section. Use this value as the OAuth client Id.

  6. Navigate to the Certificates & Secrets section and select Upload certificate. Then, select the certificate to upload from your local machine.

  7. Specify the duration and save the client secret. After you save it, the key value is displayed.

  8. Copy this value because it is displayed only once. You will use this value as the OAuth client secret.

  9. On the Authentication tab, make sure to select Access tokens (used for implicit flows).

Azure Shared Access Signature

To connect with an Azure shared access signature, set the following properties:

  • Auth Scheme: Select SAS.

  • Azure Shared Access Signature: Enter the shared access signature that is associated with the storage account.

To create an Azure shared access signature:

  1. Sign in to the Azure portal with the credentials for your root account.

  2. Click Storage accounts and select the storage account you want to use.

  3. Under Settings, click Shared Access Signature.

  4. Set the permissions and a date when the token will expire.

  5. Click Generate SAS and copy the token that is generated.

Basic

To connect with your user credentials, set the following properties:

  • Auth Scheme: Select Basic.

  • User: Enter the username that you use to authenticate to your REST account.

  • Password: Enter the password that you use to authenticate to your REST account.

Digest

To connect with your user credentials, set the following properties:

  • Auth Scheme: Select Digest.

  • User: Enter the username that you use to authenticate to your REST account.

  • Password: Enter the password that you use to authenticate to your REST account.

GCP Instance Account

When you run CData Sync on a GCP virtual machine, Sync can authenticate by using the service account that is tied to the virtual machine. Select GCPInstanceAccount for Auth Scheme to use that account. No additional properties are required.

IAM Secret Key

To connect with an IAM secret key, set the following properties:

  • Auth Scheme: Select IAMSecretKey.

  • Access Key: Enter your account access key. This value is available from your security credentials page.

  • Secret Key: Enter your account secret key. This value is available from your security credentials page.

Negotiate

To connect with Kerberos credentials, specify the following properties:

  • Auth Scheme: Select Negotiate.

  • User - Enter the username that you use to authenticate to REST.

  • Password - Enter the password that you use to authenticate to REST.

  • Kerberos KDC: Enter the Kerberos Key Distribution Center (KDC) service that you use to authenticate.

  • Kerberos Realm: Enter the Kerberos 5ealm that you use to authenticate.

  • Kerberos SPN: Enter the service principal name (SPN) for the Kerberos domain controller.

  • (Optional) Kerberos User: Enter the principal name for the Kerberos Domain Controller. The name should be in the form Host/User@Realm.

  • (Optional) Kerberos Keytab File: Enter the path to the keytab file that contains your pairs of the Kerberos principals and encrypted keys.

  • (Optional) Kerberos Service Realm: Enter the Kerberos realm of the service.

  • (Optional) Kerberos Service KDC: Enter the Kerberos Key Distribution Center (KDC) service that you use to authenticate.

  • (Optional) Kerberos Ticket Cache: Enter the full path to an MIT Kerberos credential cache file. Sync uses the specified cache file to obtain the Kerberos ticket that is required to connect to Apache HBase.

OAuth

To connect with OAuth custom credentials, specify the following properties:

  • Auth Scheme: Select OAuth.

  • OAuth Client Id: Enter the client Id that you were assigned when you registered your application with an OAuth authorization server.

  • OAuth Client Secret: Enter the client secret that you were assigned when you registered your application with an OAuth authorization server.

  • (Optional) Scope: Specify the scope of your access to the application.

  • (Optional) OAuth Authorization URL: Enter the OAuth authorization URL for the OAuth service.

  • (Optional) OAuth Access Token URL: Enter the URL from which to retrieve the access token.

  • (Optional) OAuth Refresh Token URL: Enter the URL from which to refresh the OAuth token.

OAuth Client

To connect with an OAuth client, specify the following properties:

  • Auth Scheme: Select OAuthClient.

  • OAuth JWT Cert: Enter your Java web tokens (JWT) certificate store.

  • OAuth JWT Cert Type: Enter the type of key store that contains your JWT Certificate. The default type is PEMKEY_BLOB.

  • OAuth Client Id: Enter the client Id that you were assigned when you registered your application with an OAuth authorization server.

  • OAuth Client Secret: Enter the client secret that you were assigned when you registered your application with an OAuth authorization server.

  • (Optional) Scope: Enter the scope of your access to the application.

  • (Optional) OAuth Authorization URL: Enter the OAuth authorization URL for the OAuth service.

  • (Optional) OAuth Access Token URL: Enter the URL from which to retrieve the access token.

  • (Optional) OAuth Refresh Token URL: Enter the URL from which to refresh the OAuth token.

  • (Optional) Subject Id: Enter the user subject for which the application is requesting delegated access.

  • Subject Type: Select the subject type for the client-credentials authentication. The default type is enterprise.

OAuth JWT

To connect with a REST account, specify the following properties:

  • Auth Scheme: Select OAuthJWT.

  • OAuth Client Id: Enter the client Id that you were assigned when you registered your application with an OAuth authorization server.

  • OAuth Client Secret: Enter the client secret that you were assigned when you registered your application with an OAuth authorization server.

  • Scope: Enter the scope of your access to the application.

  • OAuth JWT Cert: Enter your Java web tokens (JWT) certificate store.

  • OAuth JWT Cert Type: Enter the type of key store that contains your JWT certificate. The default type is PEMKEY_BLOB.

  • OAuth JWT Issuer: Enter the issuer of your Java web token. Typically, the issuer is either the client Id or the email address of the OAuth application.

  • (Optional) OAuth JWT Cert Subject: Enter the subject of your OAuth JWT certificate.

  • (Optional) OAuth JWT Subject: Enter the user subject for which the application is requesting delegated access.

  • (Optional) OAuth JWT Subject Type: Select the subject type (enterprise or user) for the JWT authentication. The default type is enterprise.

  • (Optional)OAuth JWT Public Key Id: Enter the Id of the public key for JWT.

  • (Optional) OAuth JWT Audience: Enter a space-separated list of entities that can use the JWT.

  • (Optional) OAuth JWT Validity Time: Specify (in seconds) how long the JWT should remain valid. The default number of seconds is 3600. Click the up and down arrows to specify the number of seconds, or enter the number directly in the text box.

OAuth Password

To connect with your user credentials, specify the following properties:

  • Auth Scheme: Select OAuthPassword.

  • User: Enter the username that you use to authenticate to your REST account.

  • Password: Enter the password that you use to authenticate to your REST account.

  • OAuth Client Id: Enter the client Id that you were assigned when you registered your application with an OAuth authorization server.

  • OAuth Client Secret: Enter the client secret that you were assigned when you registered your application with an OAuth authorization server.

  • OAuth Version: Select the OAuth version that you want to use. The default value is the latest version.

  • (Optional) Scope: Enter the scope of your access to the application.

  • (Optional) OAuth Authorization URL: Enter the OAuth authorization URL for the OAuth service.

  • (Optional) OAuth Access Token URL: Enter the URL from which to retrieve the access token.

  • (Optional) OAuth Refresh Token URL: Enter the URL from which to refresh the OAuth token.

OAuth PKCE

To connect with the OAuth PKCE extension, set the following properties:

  • Auth Scheme: Select OAuthPKCE.

  • (Optional) OAuth Client Id: Enter the client Id that you were assigned when you registered your application with an OAuth authorization server.

Secure File Transfer Protocol

To connect using Secure File Transfer Protocol, set the following properties:

  • Auth Scheme: Select SFTP*.

  • SSH Auth Mode: Select the authentication mode to use when establishing an SSH tunnel to the service. The default mode is Password.

  • SSH User: Enter the SSH user.

  • (Optional) SSH Password: Enter the SSH password.

Complete Your Connection

To complete your connection:

  1. Specify the following properties:

    • Format: Select the data structuring standard used by the REST source to which you want to connect. The default standard is JSON.

    • Data Model: Select the data model that you want to use to parse documents for your format and to generate the database metadata. The default data model is Document.

    • JSON Format: Select the format of the JSON document. The default value is JSON.

    • (Optional) XPath: Enter the XML path that repeats at the same level within the XML document. You can specify multiple paths in a semicolon-separated list.

    • AWS Region: Select the region that hosts your Amazon Web Services. The default region is NORTHERNVIRGINIA.

    • (Optional) Storage Base URL: Enter the URL of your cloud-storage service provider.

  2. Define advanced connection settings on the Advanced tab. (In most cases, though, you should not need these settings.)

  3. If you authenticate with AzureAD, click Connect to REST to connect to your REST account.

  4. Click Create & Test to create your connection.

More Information

For more information about interactions between CData Sync and REST, see REST Connector for CData Sync.