Advanced Job Options


Advanced Job Options


Advanced Job Options allow you greater control over the data flows. Overriding these defaults can help fine-tune how your data is extracted from your source and loaded into your database.

Replication Options can be set at the Query level or the Job level. Options set at the Query level affect that query only and take higher precedence than those set at the Job Level.

Replication Options

Option Name Default Description
Table Name Prefix None Applies a prefix to all of your destination tables. It only applies to general Replicate commands. For example, Replicate [Accounts] with a prefix of sync_ will replicate your Accounts data to the sync_Accounts table in your destination.
Destination Schema None Specifies the schema to be used in the Destination. Important if the database has multiple schemas defined.
Status Connection Default Specifies a database connection to save your job status. By default, the database is stored in the same directory as App settings.
Alter Schema True If this option is set, the job will add columns and alter data types in the destination table if there is a mismatch between the source table metadata and the destination table metadata.
Auto Truncate Strings False If this option is set, the job will automatically truncate strings based on the destination column size.
Batch Size 1000 Controls the number of records to send to the destination table at one time. Higher BatchSize generally means better overall performance, though a higher Timeout may be needed.
Command Timeout 300 Controls the timeout for destination connections.
Continue On Error True If this option is set, when the job encounters an error, instead of stopping, it will continue on to the next query in the job.
Convert Date Time Values to GMT False This setting converts all local time to GMT before inserting into the database.
Drop Table False If this option is set, the job drops the existing destination database table before caching the new results. Use this option if you want to refresh the entire destination database table, including its schema.
Replicate Interval 180 Paired with ReplicateIntervalUnit, this option controls the time frames to split the data into when retrieving data.
Replicate Interval Unit days Paired with ReplicateInterval, this option controls the time frames to split the data into when retrieving data. Accepted values are ‘minutes,hours,days,weeks,months,years’.
Replicate Start Date None This option sets the minimum date to start replicating from (yyyy-MM-dd).
Deletion Behavior Hard Delete Control how the Job handles deleted data in your source: ‘Skip Delete’ will ignore deleted records, ‘Hard Delete’ will remove deleted records from your destination, ‘Soft Delete’ will leave records in the destination but mark them as deleted. (Not all sources support this feature.)
Truncate Table Data False If this option is set, the job removes existing rows in the destination database table before adding the selected rows. Use this option if you want to refresh the entire destination database table but keep its existing schema.

Additional Options

Several less typical options can be set in the Additional Options input. Use name-value pairs separated by a comma to specify multiple options:

ColumnNameIncludeVowels=true,TransactionSize=100000
Option Name Default Description
BytesPerCharacter 1 Specifies how many bytes to allocate on the cache for each character. You can use this to work around an issue for some destinations where the column size is denoted by byte size and not column size. The column sizes reported will not work in the case that more than one byte is allocated per character.
ColumnNameIncludeVowels True When set to false, vowels will be removed from column names. Useful for shrinking column name sizes to get around identifier limits.
ColumnNameMaxSize -1 Determines the maximum size of column names. -1 represents there is no maximum and defaults to the maximum column size reported by the destination.
ReplaceInvalidDatesWithNull True If this option is set, all date-times reported by the source table that are invalid will be converted to null to avoid replication errors.
ReplicateEndDate None This option sets the maximum date to replicate to (yyyy-MM-dd).
StoresUppercaseIdentifiers False If this option is set, identifiers in the cache are stored with all uppercase characters.
StoresLowercaseIdentifiers False If this option is set, identifiers in the cache are stored with all lowercase characters.
Transaction Size 50000 Controls the number of records held in memory before committing them to the destination table.
LongTextSize None Set’s the maximum column size before a varchar column is converted to varchar(max). For example, setting this to 255 will leave all varchar columns with column size <= 255 as-is and set all column sizes > 255 to varchar(max). Useful for truncation issues.