Search

X12 Connector

Version 22.0.8473

X12 Connector

The X12 Connector can generate X12 files from XML and also generate XML from X12 files.

Overview

When receiving X12 documents, X12 Connectors validate X12 interchange headers and convert the X12 document into XML. This is useful as a staging step, as XML is the primary format that CData Arc uses to manipulate data within a flow. The X12 Connector automatically reads the input file to determine the appropriate X12 schema, then parses the document according to this schema.

When generating X12 documents, X12 Connectors convert XML into X12 document syntax and apply the appropriate X12 interchange headers. This is useful as the final step for creating an X12 document, after the XML data has been fetched and transformed elsewhere in the flow.

Note that interchange header validation can be disabled by setting the Usage Indicator connector setting to Test Data.

An X12 Connector can also automatically generate acknowledgments for incoming X12 documents. For more information, please see the Acknowledgments section.

Connector Configuration

This section contains all of the configurable connector properties.

Settings Tab

Translation Configuration

Settings related to how the connector will attempt to translate input files.

  • Connector Id The static name of the connector. All connector-specific files are held in a folder by the same name within the Data Directory.
  • Connector Description An optional field to provide free-form description of the connector and its role in the flow.
  • Translation Type Whether the connector is converting an X12 document into XML or XML data into an X12 document.

Interchange Settings

Settings related to the interchange headers of X12 documents. When generating X12 documents, these settings are applied as interchange headers in the resulting document. When parsing X12 documents, the interchange settings are used to validate the incoming document.

  • Authorization Qualifier (ISA01) Qualifies the Authorization Id setting by providing context for the value.
  • Authorization Id (ISA02) Identifies the authorization level associated with the interchange.
  • Security Qualifier (ISA03) Qualifies the Security Password setting by providing context for the value.
  • Security Password (ISA04) Represents security information about the interchange.
  • Sender Id Qualifier (ISA05) Qualifies the Sender Identifier by providing context to the value.
  • Sender Identifier (ISA06) Identifies the sending party in the X12 communication (when generating an X12 document, this should be your identifier).
  • Receiver Id Qualifier (ISA07) Qualifies the Receiver Identifier by providing context to the value.
  • Receiver Identifier (ISA08) Identifies the receiving party in the X12 communication (when generating an X12 document, this should be your partner’s identifier).
  • Control Standards Identifier (ISA11) Identifies the X12 standard. The only valid value is ‘U’.
  • Control Version Number (ISA12) Identifies the control version for the interchange.
  • Usage Indicator (ISA15) Whether the data exchanged represents test data, production data, or general information. If this is set to Test Data, interchange header validation will not be performed.

Functional Group Settings

Settings related to the functional group headers of X12 documents. These optional identifiers may help group similar interchanges together or facilitate sub-addressing within an organization.

  • Sender Identifier (GS02) Identifies the sending party in the X12 communication.
  • Application Recipient Identifier (GS03) Identifies the receiving party in the X12 communication.
  • Date Format (GS04) How the datestamp should be formatted.
  • Time Format (GS05) How the timestamp should be formatted.
  • Responsible Agency Code (GS07) Identifies the issuer of the X12 standard.
  • Identifier Code (GS08) Identifies the version, release, and industry of the X12 standard.

Acknowledgments

Settings related to generating and requesting acknowledgments.

  • TA1 Acknowledgment Whether to send TA1 interchange acknowledgments.
  • Functional ACK expected Whether a functional ACK should be returned (when receiving) and requested (when sending). A functional acknowledgment serves as an indication of acceptance or rejection of the received interchange and transaction set.

Automation

Settings related to the automatic processing of files by the connector.

  • Send A toggle that instructs the connector to automatically send files when they are ready.
  • Retry Interval The interval the connector will wait before retrying a failed send.
  • Max Attempts The number of attempts the connector will make to send the message. Setting this value to 1 instructs the connector to only make the initial send attempt without retrying. The connector waits the duration specified by Retry Interval between each attempt.

Advanced Tab

EDI Delimiters

Settings that specify which characters separate elements, segments, etc.

  • Data Element Separator The character that separates individual data elements within the document.
  • Component Element Separator The character that separates elements within a composite data structure in the document.
  • Segment Terminator The character that indicates the end of a segment within the document.
  • Suffix Appended to the Segment Terminator to distinguish segments. Settings related to the allocation of resources to the connector.
  • Max Workers The maximum number of worker threads that will be consumed from the threadpool to process files on this connector. If set, overrides the default setting from the Profile tab.
  • Max Files The maximum number of files that will be processed by the connector each time worker threads are assigned to the connector. If set, overrides the default setting from the Profile tab.

Tracked Headers

You can enable tracking for ISA06, ISA08, and Transaction Types by selecting the corresponding checkboxes. When enabled, these values are added as headers to processed messages. These headers are required when running EDI Reports.

Other Settings

Settings not included in the previous categories.

  • Batch Transactions Whether multiple transactions should be grouped together in a single output file. If false, each transaction found in the interchange will result in a separate output document.
  • Element Ref By ID When translating X12 into XML, whether reference ID’s will be used to name XML elements, for example:
    <_143>value</_143>
    If false, the Reference designator will be used to name XML elements:
    <BEG03>value</BEG03>
  • Expand Qualifier Values When translating X12 into XML, whether elements containing an EDI qualifier will have child elements ‘Code’ and ‘Value’ to express the qualifier code and value. For example:
    <N101>
     <Code>ST</Code>
     <Value>Ship To</Value>
    </N101>
  • Generate Description As When translating X12 into XML, descriptions of the X12 segments and elements will be provided as context for the X12 data. This context can be added as an XML comment or included within the XML elements as XML attributes. Use this setting to determine which of these approaches, or neither, is used.
  • Return inbound functional acknowledgments By default, the connector will route acknowledgments (997, 999) to the connector specified by the Route To Connector setting (or the ACK connections established in the Flows canvas). When this setting is enabled, the connector will also write these ACKs as output in the Receive folder so that they can be processed through an Arc Flow like any other EDI document.
  • Use HIPAA Schemas When enabled, the connector will use a separate set of EDI schemas with HIPAA constraints applied when translating and validating documents. For EDI documents other than healthcare documents subject to HIPAA, this setting will be ignored.
  • Treat HIPAA 278 as a Request Two separate schemas can be used for 278 documents depending on whether these should be interpreted as Requests or Responses. When enabled, the Request schema will be used, otherwise the Response schema will be used.
  • Nest Master-Detail Loops When enabled, the connector will detect EDI structures (e.g. HLLoops) that have hierarchical relationships embedded in the EDI data, and will generate XML with these hierarchical relationships represented as Parent-Child relationships. For more information, please see the Master-Detail Hierarchy: Translating HLLoops section.
  • SNIP Validation When enabled, the connector will perform the first three levels of SNIP Validation for HIPAA compliance: (1) the syntactical integrity of the document, (2) the presence of required segments and appropriate repetitions for repeated segments, and (3) the correct mathematical relationship between claim line items and claim totals. Higher levels of SNIP Validation (4+) may require a Validate Connector or custom script.
  • Send Filter A glob pattern filter to determine which files in the Send folder will be processed by the connector (e.g. *.edi). Negative patterns may be used to indicate files that should not be processed by the connector (e.g. -*.tmp). Multiple patterns may be separated by commas, with later filters taking priority except when an exact match is found.
  • Local File Scheme A filemask for determining local file names as they are downloaded by the connector. The following macros may be used to reference contextual information:
    %ConnectorId%, %Filename%, %FilenameNoExt%, %Ext%, %ShortDate%, %LongDate%, %RegexFilename:%, %DateFormat:%.
    As an example: %FilenameNoExt%_%ShortDate%%Ext%
  • Parent Connector The connector from which settings should be inherited, unless explicitly overwritten within the existing connector configuration. Must be set to a connector of the same type as the current connector.
  • Strict Schema Validation Whether the connector should Ignore, Warn, or Fail when the following are detected: Repeat counts above the allowed number; missing required elements/segments; invalid qualifier and code values; disallowed element lengths; invalid element values.
  • Log Subfolder Scheme Instructs the connector to group files in the Logs folder according to the selected interval. For example, the Weekly option instructs the connector to create a new subfolder each week and store all logs for the week in that folder. The blank setting tells the connector to save all logs directly in the Logs folder. For connectors that process many transactions, using subfolders can help keep logs organized and improve performance.
  • Log Messages Whether the log entry for a processed file will include a copy of the file itself.
  • Save to Sent Folder Whether files processed by the connector should be copied to the Sent folder for the connector.

Miscellaneous

Settings for specific use cases.

  • Other Settings Allows configuration of hidden connector settings in a semicolon-separated list, like setting1=value1;setting2=value2. Normal connector use cases and functionality should not require use of these settings.

Converting Formats

The following sections detail the process of converting X12 to XML and vice versa.

X12 to XML

Setting the Translation Type to ‘X12 to XML’ will instruct the connector to parse incoming X12 documents into XML. The connector first reads all of the header information for the Interchange and Functional Group sections of the document and validates them against the configured connector settings (unless Test Indicator is enabled). The connector then parses out the specific X12 schema used in the document and loads the schema from the ‘x12_schemas’ folder on disk (additional schema files can be downloaded from our website for free). Using the schema, the connector generates XML representing the X12 structure of the document, populates the XML with the values from the document, and provides context to each value either as XML comments or as attributes within the XML elements (depending on the value of Generate Descriptions As).

To see this process with a set of test X12 documents, navigate to the Input tab of an X12 Connector (with Translation Type set to EDI-to-XML) and select More > Create Test Files. X12 documents for an Invoice, Purchase Order, Purchase Order Acknowledgment, and Shipping Notice will be automatically generated and placed in the Input directory. After these test files are processed by the connector, navigate to the Output tab to see the resulting XML.

Once X12 documents are converted to XML, the data can be transformed and manipulated in many ways. Commonly, X12 data needs to be stored in a database or other back-end application system. Since Arc uses XML to represent Inserts into these back-end systems, storing the X12 data becomes a matter of simply mapping one XML structure onto another. This is typically done with the visual designer-driven XML Map Connector.

XML to X12

Setting the Translation Type to ‘XML to X12’ will instruct the connector to generate an X12 document out of an XML representation of the document. After the connector has constructed the X12 message out of the data parsed from the XML, it will add Functional Group and Interchange headers according to the configured connector settings.

To see this process with a set of test XML files, navigate to the Input tab of an X12 Connector (with Translation Type set to XML-to-EDI) and select More > Create Test Files. XML files representing an Invoice, Purchase Order, Purchase Order Acknowledgment, and Shipping Notice will be automatically generated and placed in the Input directory. After these test files are processed by the connector, navigate to the Output tab to see the resulting X12 document.

X12 Acknowledgments

The following sections detail the three types of X12 acknowledgments (ACKs), and how Arc expects, processes, and generates ACKs.

Interchange ACKs, 997s, and 999s

A TA1 is the commonly accepted form of Interchange Acknowledgment. It is an indication that an interchange has taken place between the two parties, though not necessarily that any individual message has been exchanged. These serve as a receipt to the sender indicating that the EDI message was successfully received, but it does not specify whether there were any issues while processing the content of the message.

The 997 (Version 4010), 997 (Version 5010), or 999 acknowledgements are considered functional ACKs - acknowledgements that an individual message (such as an Invoice or Purchase Order) has been accepted. These are generated in response to a bidirectional agreement by both parties.

Expecting Acknowledgements

X12 Connectors, while operating in XML-to-EDI mode, can be configured so that both interchange and functional acknowledgments can be requested for an interchange. When either TA1 Expected or Functional ACK expected are enabled in the Settings tab, the connector will maintain a ‘Pending ACK’ status for a transmission until the appropriate ACKs have been returned and processed. Thus, the connector status can be used to determine whether the recipient has confirmed that they received the interchange.

The following diagram shows the logical flow involved when expecting an ACK from an Invoice document:

Receive X12 ACK

In the above illustration, an X12 Connector operating in XML-to-EDI mode generates the document to be exchanged (1) and holds it in a Pending ACK state while the document is transmitted to the trading partner. The trading partner processes the transmission according to their business logic and creates acknowledgments in accordance with the configured exchange parameters (2). When the acknowledgments are returned, they are processed according to the information in the next section (3).

Processing Acknowledgments

In a typical flow, X12 ACKs will arrive at an X12 Connector operating in EDI-to-XML mode. This X12 Connector can be configured to automatically route any received acknowledgments to the X12 Connector that originally generated the document being acknowledged. Routing ACKs between X12 Connectors is configured visually in the Flows canvas by dragging the gray dot at the bottom edge of one X12 Connector (the one in EDI-to-XML mode) onto the other X12 Connector (which is in XML-to-EDI mode).

Once the X12 Connector in XML-to-EDI mode receives the routed ACK, it pairs the ACK to the original message and resolves its state from ‘Pending ACK’ to ‘Sent’.

Generating Acknowledgements

When an X12 Connector in EDI-to-XML mode receives a message and generates the corresponding XML, it can automatically generate acknowledgments for the received message. To accomplish this, enable TA1 Acknowledgment and/or Functional ACK expected within the connector’s Settings tab. These acknowledgments must be routed to another X12 Connector (in XML-to-EDI mode) to finalize the ACK. The connector to which the ACK is routed will apply Interchange headings and pass the ACK along to the next connector in the flow like any other X12 message. To route the ACK appropriately, drag the gray dot at the bottom of edge of the X12 Connector in EDI-to-XML mode onto the other X12 Connector (which is in XML-to-EDI mode).

The below diagram shows the logical flow involved when creating an ACK for a received Invoice document:

Create X12 ACK

As shown in the above illustration, after a trading partner sends a message expecting an acknowledgment (1), the X12 Connector in EDI-to-XML mode that parses the document automatically generates an ACK (2). This ACK is an XML file containing the transactional information relevant for the transaction. Then, this XML acknowledgment is routed back to an X12 Connector in XML-to-EDI mode (3) so that the Interchange headers (and any other EDI party agreement settings) are applied before the ACK is sent back to the trading partner.

Master-Detail Hierarchy: Translating HLLoops

In EDI documents, most hierarchical relationships are represented by the order of EDI segments. Some EDI structures, like HLLoops (which are found in ‘Advance Shipment Notifications’, or X12 856s), do not follow this convention and instead have the hierarchical relationships embedded in the EDI element data itself. This can make it difficult to preserve these hierarchical relationships when converting the EDI data to XML.

The X12 Connector supports preserving hierarchical relationships in HLLoops via the Nest Master-Detail Loops setting in the Advanced tab. When enabled, the connector will parse the elements within the HLLoop segments to determine which segments “belong to” other segments in a hierarchical relationship. These hierarchical relationships are reflected in the output XML as Parent-Child relationships; in other words, this setting converts hierarchy implied by the EDI content into hierarchy represented by the XML structure.

This section will briefly explain both how hierarchy is encoded in HLLoop data, and how this hierarchy is converted into XML when Nest Master-Detail Loops is enabled.

HLLoop Hierarchy

All HL segments have two values that help establish hierarchy:

  • an ID value, which identifies the current HL segment (this value is stored in HL01, or the first element in an HL segment)
  • a parent-ID value, which identifies the current HL segment’s hierarchical parent (this value is stored in HL02, or the second element in an HL segment)

For example, say that an HL segment has an ID value of ‘2’. If the next HL segment should “belong to” this prior segment in a hierarchical relationship, then the next HL segment’s parent-ID should also be ‘2’.

If an HL segment’s parent-ID is ‘0’, this means that the segment does not have a parent (i.e. it is at the top level of the hierarchy).

HLLoop Example

To further clarify the hierarchical relationships in HLLoop data, take the following EDI snippet as sample input (only the lines beginning with ‘HL’ are relevant to determining the data hierarchy; the remaining segments are included to more closely represent actual EDI data):

HL*1*0*S      // First HL loop, the shipment loop
TD3*RR*SEAU*1234567
HL*2*1*O      // Second HL loop, the order loop
PRF*PO111111
HL*3*2*P      // Third HL loop, the first package loop
MAN*UC*PACKAGE1
HL*4*3*I      // Fourth HL loop, the first item loop
MAN*UC*ITEM_1
HL*5*3*I      // Fifth HL loop, the second item loop
MAN*UC*ITEM_2
HL*6*2*P      // Sixth HL loop, the second package loop
MAN*UC*PACKAGE_2
HL*7*6*I      // Seventh HL loop, the third item loop
MAN*UC*ITEM_1
HL*8*6*I      // Eighth HL loop, the fourth item loop
MAN*UC*ITEM_3

In this data, the second HL segment “belongs to” the first HL segment in a hierarchical relationship. This can be determined by looking at the HL01 and HL02 values for these segments:

  • The HL01 (ID) value for the first HL segment is ‘1’
  • The HL02 (parent-ID) value for the second HL segment is also ‘1’

The same process of matching parent-ID values to ID values can be used to determine all of the hierarchical relationships in the data.

Converting HLLoop Hierarchy to XML

When Nest Master-Detail Loops is enabled, the X12 Connector handles the conversion of HL hierarchy to XML hierarchy automatically. The connector parses the HL01 and HL02 element values from the EDI input and ensures that XML elements are appropriately nested (indented) inside the elements representing their parents.

To see how the HL hierarchy is converted into XML hierarchy, the following XML output is generated from the above sample HL data when Nest Master-Detail Loops is enabled:

<HLLoop1_S>
  <HL><HL01>1</HL01><HL02>0</HL02><HL03>S</HL03></HL>
  <TD3><TD301>RR</TD301><TD302>SEAU</TD302><TD303>1234567</TD303></TD3>
  <HLLoop1_O>
    <HL><HL01>2</HL01><HL02>1</HL02><HL03>O</HL03></HL>
    <PRF><PRF01>PO111111</PRF01></PRF>
    <HLLoop1_P>
      <HL><HL01>3</HL01><HL02>2</HL02><HL03>P</HL03></HL>
      <MAN><MAN01>UC</MAN01><MAN02>PACKAGE1</MAN02></MAN>
      <HLLoop1_I>
        <HL><HL01>4</HL01><HL02>3</HL02><HL03>I</HL03></HL>
        <MAN><MAN01>UC</MAN01><MAN02>ITEM_1</MAN02></MAN>
      </HLLoop1_I>
      <HLLoop1_I>
        <HL><HL01>5</HL01><HL02>3</HL02><HL03>I</HL03></HL>
        <MAN><MAN01>UC</MAN01><MAN02>ITEM_2</MAN02></MAN>
      </HLLoop1_I>
    </HLLoop1_P>
    <HLLoop1_P>
      <HL><HL01>6</HL01><HL02>2</HL02><HL03>P</HL03></HL>
      <MAN><MAN01>UC</MAN01><MAN02>PACKAGE2</MAN02></MAN>
      <HLLoop1_I>
        <HL><HL01>7</HL01><HL02>6</HL02><HL03>I</HL03></HL>
        <MAN><MAN01>UC</MAN01><MAN02>ITEM_1</MAN02></MAN>
      </HLLoop1_I>
      <HLLoop1_I>
        <HL><HL01>8</HL01><HL02>6</HL02><HL03>I</HL03></HL>
        <MAN><MAN01>UC</MAN01><MAN02>ITEM_3</MAN02></MAN>
      </HLLoop1_I>
    </HLLoop1_P>
  </HLLoop1_O>
</HLLoop1_S>

It is difficult to parse the difference at a glance, but the key thing to notice is that HLLoop segments are not contained within other segments (i.e. have a Parent-Child relationship in the XML) in accordance with the HL01 and HL02 values from the original data.

X12 Operations

In addition to the Operations provided with Arc, connectors may provide operations that extend functionality into ArcScript. Operations specific to the functionality of the X12 Connector are listed below.

x12Scan

Scans header values from the headers of an X12 document.

Required Parameters

  • file: The path to the X12 file.

Optional Parameters

  • format: The file format. The default value is ‘X12’.

Output Attributes

  • InterchangeSenderIdQualifier: The interchange qualifier of the sender Id (ISA05).
  • InterchangeSenderId: The interchange sender Id (ISA06).
  • InterchangeReceiverIdQualifier: The interchange qualifier of the receiver Id (ISA07).
  • InterchangeReceiverId: The interchange receiver Id (ISA08).
  • InterchangeControlNumber: The value that uniquely identifies the EDI interchange (ISA13).
  • FunctionalGroupSenderId: The functional group sender Id (GS02).
  • FunctionalGroupReceiverId: The functional group receiver Id (GS03).
  • DocumentType: The document type or transaction set Identifier code (ST01).
  • StandardVersion The major version of the X12 Standard used in this interchange (e.g. 00401, 00501).

Common Errors

Below is a list of common errors, their causes, and the recommended solution. Please contact [email protected] for more information.

ERROR:

“Schema file not found ([schema major version] [schema document type])”

Cause

When Arc parses an X12 file, it first scans the file for the EDI version and document type to determine which schema to use during parsing. The application includes an extensive set of X12 schemas in JSON format. They are stored in the schemas folder in the application directory (next to the data folder).

This error indicates that the application could not find a schema file matching the major version and document type reported by the file.

If either the [schema major version] value or [schema document type] value are missing in the error message, this indicates that EDI document does not contain the major version or document type.

Otherwise, this indicates that the appropriate document schema is not included in the set of schema definitions automatically included with Arc.

Resolution

If the EDI file does not contain the schema major version or document type, contact the trading partner (or other source of the EDI document) and inform them that these values must be included in the message header.

Otherwise, find the appropriate schema version from our free download page here. Download the schema folder and place it in the following path:

[application_directory]\schemas\x12_schemas\[major_version]\

ERROR:

“The segment at line [line number] with tag [segment name] is not in a valid position in the specified schema ([schema major version] [schema document type]). This may indicate that the input file contains segments that are out of order.”

Cause

When Arc parses an X12 file, it first scans the file for the EDI version and document type to determine which schema to use during parsing. The application includes an extensive set of X12 schemas in JSON format, stored in the ‘www\app_data’ folder of the root installation directory.

This error indicates that the parser encountered an EDI segment in the file that does not match the order of segments defined in the relevant document schema. This may indicate a few different issues:

  • The EDI file was generated incorrectly by the trading partner
  • The EDI file was generated using a different schema version than what is indicated in the file
  • The EDI file was generated using a custom schema

Resolution

If the EDI file was generated using a custom schema, simply place this custom schema file (in JSON format) in the folder where Arc reads EDI schemas. The specific folder depends on the major version of the EDI schema:

[installation_directory]\www\app_data\x12_schemas\[major_version]\

If the EDI file was not generated using a custom schema, there are two approaches to resolving the discrepancy between the EDI file generated by the partner and the EDI schema used to parse it.

  1. Manually edit the appropriate EDI schema file (found in the filepath above) to adjust the segment order so that it matches the partner’s EDI files. The error message indicates the line in the file where the segments diverge from the schema definition. Trace through the JSON definition of segment order until the divergence is found, and re-order the segments in the JSON to match the file provided by the partner.
  2. Contact the trading partner about the EDI file. Confirm that the file was generated according to the major version that is reported by the file, and explain that the order of the segments in the file was not recognized.

The approach to use depends on the familiarity with EDI segments for the particular document type (to correctly edit the JSON schema according to the partner’s file) and the partner’s flexibility (to adjust the EDI file in response to informing them about the error).