Version 23.4.8839

• Creating Connectors in the Flow
• Connecting Connectors in the Flow
• Configuring Connectors in the Flow
  • Mappings
• Best Practices
  • Interacting with the Local File System
    • File Pickup
    • File Drop-Off
  • Partner-Specific File Transfer Connectors
  • Connector Naming Conventions
  • XML for Data Transformation
  • Organize Flows into Workspaces

To access the CData Arc Flow designer, navigate to the Flows section in the Administration console.

A Workspace canvas appears. If you have pre-existing connectors, those connectors are shown on the canvas. From here, you can add and configure new connectors, connect and rearrange existing connectors, and configure existing connectors.

• Left-click and hold on empty space in the canvas to drag the designer into position for a better view.

• Use the slider in the lower-right corner of the canvas to zoom in or out on the selected view, or to reset the zoom.

Note: You can also use a mouse to zoom if the mouse has a scroll wheel.

Creating Connectors in the Flow

To add a new connector to the flow, click the Connectors tab. Find the connector you need by using the Search bar or by expanding the connector types until you find the one you need. Drag your connector onto the canvas.

When you drag a connector onto the canvas you are prompted to name your new connector. Be sure to choose a name that clearly describes the operation the connector performs. See Connector Naming Conventions for some best practice recommendations.

Connecting Connectors in the Flow

To rearrange connectors in the designer, drag them to move around the canvas and rearrange them into your desired configuration.

To connect connectors, select the bright blue dot on the right side of a connector and drag it to the gray dot on the left side of the second connector, as shown in the following image:

This creates a logical relationship between the two connectors so that when the first connector generates a message or file, it is passed directly to the second connector for processing.

To connect a connector to a different connector, select the bright blue dot on the left side of the second connector, and move it to an unconnected connector. To disconnect a connector, drag the blue dot from the connected connector to an empty space on the canvas.

Configuring Connectors in the Flow

To configure a connector, click the connector on the canvas to open the connector configuration wizard, as shown in the following image:

On the series of tabs that appear, you can configure and test all of the connector settings. When you finish making changes, select the gray x on the left side of the panel to close the connector configuration wizard and return that space to the canvas.

Mappings

Some connectors (for example, CData, Salesforce, QuickBooks, and others) model data sources like a database, and each of those connectors follow a common model. The Actions setting for those connectors allow you to perform various actions such as Select, Upsert, Lookup, Lookup Stored Procedure, and Execute Stored Procedure in your flow. See Select an Action for an example of how to use these actions, and links to detailed descriptions of each one.

Note: Depending on which action you require, these connectors can be at the beginning, in the middle, or at the end of your flow.

Best Practices

Most flows have a connector at which data enters the flow and a connector at which data exits the flow. For some cyclical flows, the entry and exit point might be the same connector.

For example, data might enter the flow at an AS2 connector when an external trading partner sends a business document over AS2. The data from this business document might need to be imported into a back-end system like a database. In this case, the AS2 connector and the Database connector are the entry and exit points for the flow.

When creating a new flow, it is often easiest to start with the entry and exit connectors. Then, “work inwards” by adding more connectors to the flow that fill in the steps between the entry and exit point (for example, you might need to add connectors that transform an inbound AS2 payload into a database insert).

Interacting with the Local File System

CData recommends using the File connector when configuring a flow that interacts with the local file system. This is especially important for folders that are shared with external monitoring processes. Guidelines for file pickup and file drop-off are outlined below.

File Pickup

To monitor a directory for files to be processed in Arc, add a File connector at the beginning of your flow as shown below:

Follow these steps to configure the connector:

1. Open the Settings tab.

2. Set the Path property to the directory that you want to monitor. You can use macros to dynamically resolve paths based on date and time; hover over the Path field for more information.

3. Configure the File Mask property with the file pattern that you want to retrieve.

4. To avoid reprocessing files that have already been picked up, configure one of these two options based on what works best for your flow:

   • In the Receive section, check the box for Delete files (after received). This option deletes files after they have been processed.
   • In the Caching section, check one or both caching options to maintain a file size and/or timestamp record of files that have been processed. The application skips cached files but does not delete them.

5. Open the Automation tab.

6. Disable Send automation.

7. Enable Receive automation and set the desired polling interval for the directory.

   

File Drop-Off

To drop off processed files to the local file system, add a File connector to the end of your flow as shown below:

Follow these steps to configure the connector:

1. Open the Settings tab.

2. Set the Path property to the directory where you want to drop off files. You can use macros to dynamically resolve paths based on date and time; hover over the Path field for more information.

3. Open the Advanced tab.

4. Use the Overwrite Option to specify how to handle files that already exist in the destination path.

Partner-Specific File Transfer Connectors

Connectors that send or receive files over the network (for example, AS2, AS4, FTP, SFTP, OFTP, and others) are bi-directional, but are configured for a single trading partner. In other words, a single AS2 connector could send and receive AS2 messages with Amazon, but it could not also send and receive files with Walmart.

Many MFT (Managed File Transfer) connectors require setting up a profile to exchange files. The Profiles Page contains a tab for configuring application-wide settings for each supported protocol. For example, the information configured in the AS2 profile is used in all AS2 connectors, and partner-specific AS2 connection details are configured in each AS2 connector.

Connector Naming Conventions

Connectors should always be given a descriptive name that communicates the role of the connector in the flow.

• The name of an MFT connector (for example, AS2, AS4, FTP, SFTP, OFTP, and others) should indicate the remote party with whom files are exchanged, and the protocol over which the exchange occurs (for example: Walmart_Production_AS2).
• The name of an EDI connector (such as X12 and EDIFACT) should indicate the name of the partner with whom the EDI document is exchanged, the direction of translation, and the format of the document (for example: Walmart_X12_Outbound).
• The name of a Data Mapping connector (such as XML Map) should indicate the format/schemas that the connector is mapping from and mapping to (for example: Map_850_to_Orders).
• The name of a Back-End Integration connector (for example, Database, MySQL, CData) should indicate the specific back-end system and the table in that back-end system if applicable (for example: PostgreSQL_Orders_table).

If you use the Connector Description field on a connector, you can hover your cursor over the information icon to see your description of that connector. If you do not see an information icon, the connector does not have any text in the field.

XML for Data Transformation

Arc uses XML as an intermediary format for data manipulation and transformation. Many connectors translate files of various formats into XML and vice versa (for example, the X12 connector, EDIFACT connector, CSV connector, and others). Additionally, Arc uses XML to model back-end system input and output (such as SELECTS and INSERTS into database-like systems).

Since most source and destination formats can be represented as XML, data transformation flows often have the following structure:

• A connector that pulls or receives data in its original (source) format
• A connector that converts this source format into XML (if it is not already XML)
• An XML Map connector that transforms this source XML into a new structure representing the target (destination) format
• A connector that converts this transformed XML into the destination format (if a non-XML format is required)
• A connector that sends the finalized file to its intended destination

The XML Map Connector is the engine that performs the critical work in these data transformation flows. Becoming comfortable with data transformation is often a matter of familiarizing yourself with the XML Map connector’s visual designer.

Organize Flows into Workspaces

You can configure multiple separate flows on the same flows canvas without risk of cross-contamination, but dividing flows into different Workspaces can help reduce clutter and maintain organization.

Use the cogwheel in the top-right of the Flows page to create new workspaces, import or export workspace configuration settings, and delete the current workspace (except for the Default workspace). Only the connectors from the current workspace are displayed at any given time.

Connectors in different workspaces cannot be connected to each other, and the data and configuration files specific to these connectors are held in separate folder structures.