Version 22.0.8483

• Task Settings
  • Source Table Information
  • Destination Table Information
  • Column Mapping
  • Replication Options
  • Write Custom Queries
• Load From Folder

Tasks control the data flow from a source into a destination table. In a standard replicate job, all source tables and views are available to be added as replicate tasks to the job.

To add tables and views as replicate tasks:

1. Click a job.

2. Click Add Tables in the Job Settings section of the Jobs/YourJobName page. This action opens the Add Tables modal.

3. Select specific tables in the modal or select the check box next to Table Name (top of the modal) to select all tables. Then, click Add Selected Tables.

CData Sync automatically builds the replicate tasks and saves your job.

If you previously selected the Sync all tables option in your job, all available tables and views are added automatically as replicate tasks to the job. Each time the job executes, Sync first searches for any new tables or views your source and appends additional replicate tasks to your job so that they are included in the run. Click the Exclude button (on the Tables tab of the Job Settings panel) to remove certain tasks from the job. When you do this, future job executions ignore those tables.

Task Settings

You can configure each task in your job individually by clicking a row on the Tables tab. This action opens the Task Settings modal where you can adjust the task settings that are described in the following sections.

Source Table Information

The Source Table Information section provides an overview of the source table’s capabilities and key features:

• Primary Key: Specifies the column or columns that make up the primary key index for the table.

• Incremental Check Column: Specifies the column that is used by Sync to identify new or modified records during subsequent replication intervals. See Incremental Check Column for more information.

• Captures Deletes: Specifies whether Sync can detect when data has been deleted in the source. See Deletion Captures for more information.

Destination Table Information

The Destination Table Information section gives you full control over where to land your source data.

In this table:

• Schema Name: The schema name is inherited from the Destination Schema advanced job option. For destinations that support multiple schemas, select from a list of available schemas in your database.

• Table Name: By default, Sync creates a new table with the same name as the source table in the target destination where you want to replicate the data. You have the option of renaming that table by clicking Create New Table. Another option is to map your source table into an existing table in the destination. Click Map to Existing Table to select an existing destination table to which to map your source table. Sync attempts to map the source automatically to destination columns. Navigate to Column Mapping to adjust the default mappings.

Column Mapping

The Column Mapping tab gives you more agency over which data to retrieve from your source table and how you want that data to be represented in the target table. See Mapping Columns for information about how to remove or adjust column mappings and apply different transformations to each replicate task.

Replication Options

Replication options control the flow from your source application to your target destination. While you can set replication options in your job that affect the job as a whole, replication options that you set in the query only affect that specific query. By default, each replication option that you set in the job is inherited by each query, and any replication option that you set in the query overrides those values.

Write Custom Queries

The Task Settings automatically update the REPLICATE query as you make changes in the modal. In addition, you can select the Write Custom Query check box to edit the generated query manually. See Custom Querying: REPLICATE Command for REPLICATE command syntax; see Custom Querying: CHECKCACHE Command for CHECKCACHE command syntax. Each of these pages contain common examples, as well.

Load From Folder

If the Load all files into single table option is selected, you need to configure a single LoadFolder task that continuously replicates files from a directory or bucket into a single destination table. When Sync runs the job, it compares the last modified date of each file in source folder to the value stored in the destination table. To help facilitate this task, Sync adds these required system columns to your destination table:

Column Name	Data Type	Primary Key
_filename	Varchar	Yes
_line	Integer	Yes
_lastModified	BigInt	Yes if Modified Action = APPEND

To adjust the settings for the LoadFolder tasks, select the Configure button to open the task settings:

• Modified Action: Specifies whether to append or upsert data in your destination table. When a previously replicated file is modified, the APPEND action appends all rows while the UPSERT action replaces any existing records.

• File Mask: Specifies a pattern to match file names within the folder.

• Schema Name: Specifies a particular schema name or names. For destinations that support multiple schemas, you can select from a list of available schemas in your database.

• Table Name: Specifies the table name. By default, Sync creates a new table with the same name as the job name in the target destination where you want to replicate the data. You can rename that table by clicking Create New Table.

After you create a LoadFolder task, you can choose a sample file from which to build the column mapping from the Column-Mapping tab in the Task Settings modal. See Mapping Columns for more information about how to edit the mappings and apply different transformations to the task.