Search

Transformations Page

Version 24.2.9064

Transformations Page

Transformations take raw data from your source and clean, shape, and organize it into a form that best fits your needs as you proceed through your data pipeline. Transformations allow you to decide how your data should be shaped, giving you only the data you need in the form you need.

The following sections explain the features of the Transformations summary page as well as the following tasks:

Viewing Your Transformations

You can view a list of all your Transformatios from the Transformations page. The table on this page summarizes your transformations, enabling you to manage them from one access point.

The transformation records are displayed in rows, with columns that provide details about each transformation. As shown above, you can specify the number of rows that you want to display on the page by selecting a number from the X records per page list at the bottom of the page.

Sync provides details about your transformations in five columns:

  • NAME - Lists the name of the transformation, the destination connectors, and the transformation type (either SQL or dbt Cloud).

  • LABEL - Displays labels that you assign to your transformation. You add labels in either of these ways:

    • Select . . . > Add Labels at the end of the transformation row.

    • Click Add Labels in the Labels section on the Overview tab in the transformation settings.

  • SCHEDULE - Specifies the run schedule for your transformation. For more details, see Enabling and Disabling the Scheduler.

  • LAST RUN - Specifies the last date and time that the transformation ran, how long the run took, and the number of rows that were affected.

  • NEXT RUN - Specifies the next date and time that the transformation will run.

If you click a transformation name, Sync opens the Settings page for that transformation.

You can organize your transformations list in ascending or descending order for the NAME, LAST RUN, or NEXT RUN columns. To order the columns, simply click the up (▲) and down (▼) arrows next to the column by which you want to sort.

In addition, you can filter the table by transformation type, status, destination, and label. After you set filters, you can clear all filter options by clicking Reset. Sync also enables you to save filters that you define so that you can use them later.

To save a filter:

  1. Choose your filter options.

  2. Select Saved Filters > Add Saved Filter.

    This selection opens the Add Saved Filter dialog box.

  3. Enter a name for your filter. Then, click Add Saved Filter.

After you save the filter, the filter name appears in the Saved Filter list. Then, you can choose that filter from the list any time that you want to apply it.

Adding a New Transformation

Adding a transformation requires preconfigured destination connections or dbt jobs (depending on the type of transformation you want to create). After you define a supported destination connection to your database or you set up a dbt job, follow these steps to create a new transformation:

  1. Click Transformations in the navigation bar to open the Transformations page.

  2. Click Add Transformation (upper right corner). This action opens the Add Transformation dialog box where you can enter transformation information.

  3. Enter a transformation name. Then, select a transformation type (either SQL or dbt Cloud) and either a destination (for SQL) or a dbt job (for dbt Cloud).

  4. Select a transformation type.

  5. Click Add Transformation to create your new transformation.

From your specific transformation page (Transformations > YourTransformationName), you can set specific features, including setting triggers for run schedules, adding and managing queries (for SQL queries only), mapping columns, and more.

Exporting an SQL Transformation

In Sync, you can export only SQL transformations. You can export these transformations from the Transformations page to a location on your machine.

To export an SQL transformation, select … > Export from the end of the row of the transformation that you want to export.

The resulting SYNCJOB file contains the details of the exported transformation, including the destination, tasks, and additional settings.

Sync exports the transformation to your Downloads folder. From there, you can move it to any other location on your machine.

Importing an SQL Transformation

In Sync, you can import only SQL transformations. You can import these transformations from your machine to the Transformations table.

To import an SQL transformation:

  1. Select Add Transformation > Import Transformation from the Transformations page. This action opens the Import Transformation dialog box.

  2. Click Choose File to open an explorer window where you can select the transformation that you want to import. (The transformation file must be a SYNCJOB file, as shown below.) Once you select your file, the dialog box expands, as shown below.

  3. Keep the original transformation name or modify it, as necessary. Then, choose whether to create a new destination or to select an existing destination.

  4. Click Import to import the transformation into the Transformations table.

Enabling and Disabling the Scheduler

You can enable or disable the scheduler via these two methods:

  • the Schedule column on the main Transformations page (for all transformations). This option enables you enable or disable a schedule that is already defined.

  • the Trigger option on the Overview tab of the settings page (for an individual transformation). This option enables you to set a specific schedule by which to run your transformation. You can also enable or disable the scheduler from this location.

On the main Transformations page, you enable or disable the scheduler for all transformations by toggling the Schedule button in the SCHEDULE column.

Starting and Stopping Transformations

You can start a transformation by positioning your cursor over the end of the transformation row and clicking the Run Transformation (▷) icon.

When a transformation is running, you cancel that run by clicking the Stop Transformation (■) icon.

More Information

For more detailed information about Sync transformations and how they work, see Transformations.