Version 24.2.9064

• Add the SQL Server Connector
• Authenticate to SQL Server
  • Password
  • NTLM
  • Kerberos
  • AzurePassword
  • Azure Active Directory
  • Azure Managed Service Identity
  • Azure Service Principal
  • Azure Service Principal Certificate
• Complete Your Connection
• Configure Change Data Capture and Change Tracking
  • Enable Change Data Capture for CData Sync
  • Enable Change Tracking for CData Sync
  • Alter Schema
• Support for Incremental Replication
• More Information

You can use the SQL Server connector from the CData Sync application to capture data from SQL Server and move it to any supported destination. To do so, you need to add the connector, authenticate to the connector, and complete your connection.

Add the SQL Server Connector

To enable Sync to use data from SQL Server, you first must add the connector, as follows:

1. Open the Connections page of the Sync dashboard.

2. Click Add Connection to open the Select Connectors page.

3. Click the Sources tab and locate the SQL Server row.

4. Click the Configure Connection icon at the end of that row to open the New Connection page. If the Configure Connection icon is not available, click the Download Connector icon to install the SQL Server connector. For more information about installing new connectors, see Connections.

Authenticate to SQL Server

After you add the connector, you need to set the required properties.

On the New Connection page, enter the connection name of your choice.

CData Sync supports authenticating to SQL Server in several ways. Select your authentication method below to proceed to the relevant section that contains the authentication details.

• Password (default)
• NTLM
• Kerberos
• AzurePassword
• AzureAD
• AzureMSI
• AzureServicePrincipal
• AzureServicePrincipalCert

Password

To connect with your user credentials, specify the following properties:

• Auth Scheme - Select Password.

• User - Enter the username that you use to authenticate to SQL Server.

• Password - Enter the password that you use to authenticate to SQL Server.

NTLM

To connect with your NTLM user credentials, specify the following properties:

• Auth Scheme – Select NTLM.

• User - Enter the username that you use to authenticate to SQL Server.

• Password - Enter the password that you use to authenticate to SQL Server.

• Domain (optional) - Enter the name of the domain for a Windows (NTLM) security login.

• NTLM Version (optional) - Select the NTLM version that you want to use. The default version is 1.

Kerberos

To connect with your Kerberos credentials, specify the following properties:

• Auth Scheme – Select Kerberos.

• User - Enter the username that you use to authenticate to SQL Server.

• Password - Enter the password that you use to authenticate to SQL Server.

• Kerberos KDC - Enter the this to the host name or IP Address of your Kerberos Key Distribution Center (KDC) machine.

• Kerberos Realm - Enter the Kerberos realm that you use to authenticate to Kerberos.

• Kerberos SPN - Enter the service principal name (SPN) for the Kerberos domain controller.

• Kerberos Keytab File (optional) - Enter the full file path to your Kerberos keytab file.

• Kerberos Ticket Cache (optional) - Enter the full file path to an MIT Kerberos credential cache file.

AzurePassword

To connect with your Azure user credentials, specify the following properties:

• Auth Scheme – Select AzurePassword.

• User - Enter the username that you use to authenticate to Azure.

• Password - Enter the password that you use to authenticate to Azure.

Azure Active Directory

To connect with an Azure Active Directory (AD) user account, select Azure AD for Auth Scheme. CData Sync provides an embedded OAuth application with which to connect so no additional properties are required.

Azure Managed Service Identity

To leverage Azure Managed Service Identity (MSI) when CData Sync is running on an Azure virtual machine, select Azure MSI for Auth Scheme. No additional properties are required.

Azure Service Principal

To connect with an Azure service principal and client secret, set the following properties:

• Auth Scheme - Select Azure Service Principal.

• Azure Tenant - Enter the Microsoft Online tenant to which you want to connect.

• OAuth Client Id - Enter the client Id that you were assigned when you registered your application with an OAuth authorization server.

• OAuth Client Secret - Enter the client secret that you were assigned when you registered your application with an OAuth authorization server.

To obtain the OAuth client Id and client secret for your application:

1. Log in to the Azure portal.

2. In the left navigation pane, select All services. Then, search for and select App registrations.

3. Click New registrations.

4. Enter an application name and select Any Azure AD Directory - Multi Tenant. Set the redirect URI to the value that is specified for CallbackURL.

5. After you create the application, copy the application (client) Id value that is displayed in the Overview section. Use this value as the OAuth client Id.

6. Navigate to the Certificates & Secrets section and select New Client Secret for the application.

7. Specify the duration and save the client secret. After you save it, the key value is displayed.

8. Copy this value because it is displayed only once. You will use this value as the OAuth client secret.

9. On the Authentication tab, make sure to select Access tokens (used for implicit flows).

Azure Service Principal Certificate

To connect with an Azure service principal and client certificate, set the following properties:

• Auth Scheme - Select Azure Service Principal Cert.

• Azure Tenant - Enter the Microsoft Online tenant to which you want to connect.

• OAuth Client Id - Enter the client Id that you were assigned when you registered your application with an OAuth authorization server.

• OAuth JWT Cert – Enter your Java web tokens (JWT) certificate store.

• OAuth JWT Cert Type – Enter the type of key store that contains your JWT Certificate. The default type is PEMKEY_BLOB.

• OAuth JWT Cert Password (optional) – Enter the password for your OAuth JWT certificate.

• OAuth JWT Cert Subject (optional) – Enter the subject of your OAuth JWT certificate.

To obtain the OAuth certificate for your application:

1. Log in to the Azure portal.

2. In the left navigation pane, select All services. Then, search for and select App registrations.

3. Click New registrations.

4. Enter an application name and select Any Azure AD Directory - Multi Tenant. Set the redirect URI to the value that is specified for CallbackURL.

5. After you create the application, copy the application (client) Id value that is displayed in the Overview section. Use this value as the OAuth client Id.

6. Navigate to the Certificates & Secrets section and select Upload certificate. Then, select the certificate to upload from your local machine.

7. Specify the duration and save the client secret. After you save it, the key value is displayed.

8. Copy this value because it is displayed only once. You will use this value as the OAuth client secret.

9. On the Authentication tab, make sure to select Access tokens (used for implicit flows).

Complete Your Connection

To complete your connection:

1. For the Database property (optional), enter the default database to which you want to connect when you connect to SQL Server.

2. Define advanced connection settings on the Advanced tab. (In most cases, though, you should not need these settings.)

3. If you authenticate with AzurePassword, AzureAD, AzureMSI, AzureServicePrincipal, or AzureServicePrincipalCert, click Connect to SQL Server to connect to your SQL Server account.

4. Click Create & Test to create your connection.

Configure Change Data Capture and Change Tracking

SQL Server supports two methods for tracking the changes from your source database:

• Change data capture (CDC): Change data capture tracks every change that is applied to a table and records those changes in a shadow history table. Rather than capturing only the primary key (for example, Change Tracking), CDC records the full row data to the history table. This behavior enables CDC to work with tables that do not include the primary key.

• Change tracking: Change tracking provides an efficient tracking mechanism for CData Sync. Once change tracking is configured on your tables, any DML statement that affects rows in the source table causes change-tracking information to be recorded to the change-tracking table for each modified row. For more details about change tracking for SQL Server, see Fine Tuning Data Access.

Note: If both of these methods are enabled on a table, Sync uses CDC.

To use CDC for the SQL Server source in Sync, ensure that you have the following prerequisites in place:

• CDC must be enabled on the SQL Server database (see step 1 in the section Enable Change Data Capture for CData Sync.

• The SQL Server agent must be running.

• You must be a member of the db_owner fixed database role for the database.

For more details about CDC for SQL Server, see Fine Tuning Data Access.

Enable Change Data Capture for CData Sync

1. Enable CDC on your database by submitting the following statements:

    USE [DatabaseName];
    EXEC sys.sp_cdc_enable_db;
    GO 
   

   To enable CDC on an individual table, submit the following statements:

    USE [DatabaseName];
    EXEC sys.sp_cdc_enable_table 
    @source_schema = [SchemaName],
    @source_name   = [TableName],
    @role_name     = NULL
    GO 
   

2. Create a job in Sync with your SQL Server source and select the Change Data Capture option.

Enable Change Tracking for CData Sync

1. Enable change tracking on your database by submitting the following statement:

     ALTER DATABASE [DatabaseName] SET CHANGE_TRACKING=ON (CHANGE_RETENTION=7 DAYS, AUTO_CLEANUP=ON);
   

   The CHANGE_RETENTION parameter specifies the time period for which change-tracking information is kept in your database. As a best practice, set a longer time frame to give Sync time to resolve conflicts and errors. If the last successful job run is outside the retention period, Sync replicates the full table automatically to ensure that no changes are missed.

   To enable change tracking on an individual table, submit the following statement:

    ALTER TABLE [SchemaName].[TableName] ENABLE CHANGE_TRACKING;
   

   Note: To use change tracking, each table must have at least one primary key.

2. Create a job in Sync with your SQL Server source and select the Change Data Capture option.

Alter Schema

When you use change tracking, Sync updates the destination table automatically when changes (like adding a column or changing a data type) are made to the source table structure. When you use CDC, SQL Server does not track new columns automatically. So, you must create a new CDC instance and drop the old instance. This action triggers a full refresh of the table in Sync.

Support for Incremental Replication

The SQL Server source supports incremental replication. Incremental replication reduces the workload tremendously and minimizes bandwidth use and latency of synchronization. Moving data in increments offers great flexibility when you are dealing with slow APIs or daily quotas.

You must configure the incremental check column based on the SQL Server source columns.

For details about how to set up incremental replication, see Incremental Replication.

More Information

For more information about interactions between CData Sync and SQL Server, see SQL Server Connector for CData Sync.