XML Group Connector
Version 25.2.9314
Version 25.2.9314
XML Group Connector
The XML Group connector processes collections of related XML records, grouping them by columns for efficient data management and analysis.
Overview
The XML Group connector processes collections of related XML records by aggregating, validating, and transforming structured data for seamless integration with various systems. It enables efficient handling of hierarchical XML data, ensuring consistency and interoperability across applications such as data exchange, reporting, and automated workflows.
Connector Configuration
This section contains all of the configurable connector properties.
Group Config Tab
Settings related to configuring the group.
Sample File
Use the Sample File toggle to specify a sample file that allows the connector to provide more context about the document. See Using Sample Files for more information.
- Sample File Select an existing sample file, or upload a new one.
Loop Settings
Settings related to the XPath the connector should use to represent a single record in the input document, as well as the XML element to use to hold each group of related records.
- Element Name The name of the XML element to use for the grouping in the output file.
- Record XPath Which XPath from the input should represent a single record.
Group Settings
Settings that define which elements in each record to use to determine the key for each group. See Using Group Settings for more information.
- Element Name Tells the connector what to name the XML element to use for grouping in the output file. This is usually the same name as the element in the input file that the Key XPath points to, but you can use a different name.
- Key XPath The XPath from the input file of the element to use as the key (what you want to group by). The Key XPath is relative to the Record XPath.
- Add Key XPath Expand this to define additional keys.
Summary Settings
Use the Summary Settings toggle to see the settings that define how to summarize the records in each group. See Using Summary Settings for more information.
- Element Name The name of the XPath to use to summarize the records.
- Function The function to use as the aggregate. Choose from MIN, MAX, SUM, AVG, COUNT, and CONCATENATE.
- XPath The relative XPath to the element to aggregate.
- Add Aggregate XPath Expand this to add more aggregate XPaths.
Settings Tab
Configuration
Settings related to establishing the connection.
- Connector Id The static, unique identifier for the connector.
- Connector Type Displays the connector name and a description of what it does.
- Connector Description An optional field to provide a free-form description of the connector and its role in the flow.
Message
Message settings determine how the connector searches for messages and manages them after processing. You can save messages to your Sent folder or you can group them based on a Sent folder scheme, as described below.
- Save to Sent Folder Check this to copy files processed by the connector to the Sent folder for the connector.
- Sent Folder Scheme Instructs the connector to group files in the Sent folder according to the selected interval. For example, the Weekly option instructs the connector to create a new subfolder each week and store all sent files for the week in that folder. The blank setting instructs the connector to save all files directly in the Sent folder. For connectors that process many transactions, using subfolders can help keep files organized and improve performance.
Logging
- Log Level The verbosity of logs generated by the connector. When you request support, set this to Debug.
- Log Subfolder Scheme Instructs the connector to group files in the Logs folder according to the selected interval. For example, the Weekly option instructs the connector to create a new subfolder each week and store all logs for the week in that folder. The blank setting tells the connector to save all logs directly in the Logs folder. For connectors that process many transactions, using subfolders helps keep logs organized and improves performance.
- Log Messages Check this to have the log entry for a processed file include a copy of the file itself. If you disable this, you might not be able to download a copy of the file from the Input or Output tabs.
Miscellaneous
Miscellaneous settings are for specific use cases.
- Other Settings Enables you to configure hidden connector settings in a semicolon-separated list (for example,
setting1=value1;setting2=value2
). Normal connector use cases and functionality should not require the use of these settings.
Automation Tab
Automation Settings
Settings related to the automatic processing of files by the connector.
- Send Check this to have the connector automatically send groups when they are ready.
Performance
Settings related to the allocation of resources to the connector.
- Max Workers The maximum number of worker threads consumed from the threadpool to process files on this connector. If set, this overrides the default setting on the Settings > Automation page.
- Max Files The maximum number of files sent by each thread assigned to the connector. If set, this overrides the default setting on the Settings > Automation page.
Alerts Tab
Settings related to configuring alerts and Service Level Agreements (SLAs).
Connector Email Settings
Before you can execute SLAs, you need to set up email alerts for notifications. Clicking Configure Alerts opens a new browser window to the Settings page where you can set up system-wide alerts. See Alerts for more information.
Service Level Agreement (SLA) Settings
SLAs enable you to configure the volume you expect connectors in your flow to send or receive, and to set the time frame in which you expect that volume to be met. CData Arc sends emails to warn the user when an SLA is not met, and marks the SLA as At Risk, which means that if the SLA is not met soon, it will be marked as Violated. This gives the user an opportunity to step in and determine the reasons the SLA is not being met, and to take appropriate actions. If the SLA is still not met at the end of the at-risk time period, the SLA is marked as violated, and the user is notified again.
To define an SLA, click Add Expected Volume Criteria.
- If your connector has separate send and receive actions, use the radio buttons to specify which direction the SLA pertains to.
- Set Expect at least to the minimum number of transactions (the volume) you expect to be processed, then use the Every fields to specify the time frame.
- By default, the SLA is in effect every day. To change that, uncheck Everyday then check the boxes for the days of the week you want.
- Use And set status to ‘At Risk’ to indicate when the SLA should be marked as at risk.
- By default, notifications are not sent until an SLA is in violation. To change that, check Send an ‘At Risk’ notification.
The following example shows an SLA configured for a connector that expects to receive 1000 files every day Monday-Friday. An at-risk notification is sent 1 hour before the end of the time period if the 1000 files have not been received.
Using Sample Files
Toggle Sample File on to upload a sample input XML that mimics the XML input that is going to be passed into the connector.
When enabled, it provides XPath helpers for each of the XPath fields in the UI, so it becomes easier to see the document structure and select the correct XPath. You can only click eligible XML elements, based on what the XPath is representing for the connector.
For example, in Loop Settings, you can only select parent elements because Arc expects that those elements are what represent a record.
Then, in Group Settings, the visible elements are the child elements of the Loop Settings parent element because Arc expects the key element to be a child of the parent record.
Using Group Settings
Group settings define the elements in each record to use to determine the key for each group. The key is usually an Id
value such as CustomerID
or InvoiceID
. You can have multiple keys, which appear as children of the element name and xpath set in Loop Settings.
- Element Name tells the connector what to name the XML element to use for grouping in the output file. This is usually the same name as the element in the input file that the Key XPath points to, but you can use a different name.
- Key XPath is the XPath from the input file of the element to use as the key (what you want to group by). The Key XPath is relative to the Record XPath.
For example, in the following XML, the <MaterialCode>
element is the key. This lets you can group all occurrences of ALPHA
, BRAVO
and CHARLIE
together in the output, instead of having multiples of each.
<Items>
<Invoice>
<InvoiceDetail>
<MaterialCode>ALPHA</MaterialCode>
<MaterialCodeDescription>Alpha Materials</MaterialCodeDescription>
<NetAmount>1000.00</NetAmount>
</InvoiceDetail>
<InvoiceDetail>
<MaterialCode>ALPHA</MaterialCode>
<MaterialCodeDescription>Alpha Materials</MaterialCodeDescription>
<NetAmount>2000.00</NetAmount>
</InvoiceDetail>
<InvoiceDetail>
<MaterialCode>BRAVO</MaterialCode>
<MaterialCodeDescription>Bravo Materials</MaterialCodeDescription>
<NetAmount>100.00</NetAmount>
</InvoiceDetail>
<InvoiceDetail>
<MaterialCode>CHARLIE</MaterialCode>
<MaterialCodeDescription>Charlie Materials</MaterialCodeDescription>
<NetAmount>500.00</NetAmount>
</InvoiceDetail>
<InvoiceDetail>
<MaterialCode>CHARLIE</MaterialCode>
<MaterialCodeDescription>Charlie Materials</MaterialCodeDescription>
<NetAmount>100.00</NetAmount>
</InvoiceDetail>
</Invoice>
</Items>
Using Summary Settings
Summary settings control how the connector should aggregate data. You can define an XPath containing a value that you want to perform an action operator on (MIN, MAX, SUM, AVG, COUNT, CONCATENATE), and the connector then performs that operator on all records in a group. A common example is to SUM the price of each record.
The following example uses the Loop Settings and Group Settings described in the previous images, along with the following Summary Settings:
Using the XML in the previous section results in an output XML with a InvoiceDetailGroup
for each key element (the MaterialCode
). Inside each InvoiceDetailGroup
is the MaterialCode
key from the Group Settings, and the calculated NetAmountSUM
from the Summary Settings. Finally, the original InvoiceDetail
elements and their children are included:
<Items>
<Invoice>
<InvoiceDetailGroup>
<MaterialCode>ALPHA</MaterialCode>
<NetAmountSUM>3000</NetAmountSUM>
<InvoiceDetail>
<MaterialCode>ALPHA</MaterialCode>
<MaterialCodeDescription>Alpha Materials</MaterialCodeDescription>
<NetAmount>1000.00</NetAmount>
</InvoiceDetail>
<InvoiceDetail>
<MaterialCode>ALPHA</MaterialCode>
<MaterialCodeDescription>Alpha Materials</MaterialCodeDescription>
<NetAmount>2000.00</NetAmount>
</InvoiceDetail>
</InvoiceDetailGroup>
<InvoiceDetailGroup>
<MaterialCode>BRAVO</MaterialCode>
<NetAmountSUM>100.00</NetAmountSUM>
<InvoiceDetail>
<MaterialCode>BRAVO</MaterialCode>
<MaterialCodeDescription>Bravo Materials</MaterialCodeDescription>
<NetAmount>100.00</NetAmount>
</InvoiceDetail>
</InvoiceDetailGroup>
<InvoiceDetailGroup>
<MaterialCode>CHARLIE</MaterialCode>
<NetAmountSUM>600</NetAmountSUM>
<InvoiceDetail>
<MaterialCode>CHARLIE</MaterialCode>
<MaterialCodeDescription>Charlie Materials</MaterialCodeDescription>
<NetAmount>500.00</NetAmount>
</InvoiceDetail>
<InvoiceDetail>
<MaterialCode>CHARLIE</MaterialCode>
<MaterialCodeDescription>Charlie Materials</MaterialCodeDescription>
<NetAmount>100.00</NetAmount>
</InvoiceDetail>
</InvoiceDetailGroup>
</Invoice>
</Items>
Because you already have the associated records as nested children of the key, instead of in a flat structure, this XML is easier to work with when it goes to an XML Map connector.
Macros
Using macros in file naming strategies can enhance organizational efficiency and contextual understanding of data. By incorporating macros into filenames, you can dynamically include relevant information such as identifiers, timestamps, and header information, providing valuable context to each file. This helps ensure that filenames reflect details important to your organization.
CData Arc supports these macros, which all use the following syntax: %Macro%
.
Macro | Description |
---|---|
ConnectorID | Evaluates to the ConnectorID of the connector. |
Ext | Evaluates to the file extension of the file currently being processed by the connector. |
Filename | Evaluates to the filename (extension included) of the file currently being processed by the connector. |
FilenameNoExt | Evaluates to the filename (without the extension) of the file currently being processed by the connector. |
MessageId | Evaluates to the MessageId of the message being output by the connector. |
RegexFilename:pattern | Applies a RegEx pattern to the filename of the file currently being processed by the connector. |
Header:headername | Evaluates to the value of a targeted header (headername ) on the current message being processed by the connector. |
LongDate | Evaluates to the current datetime of the system in long-handed format (for example, Wednesday, January 24, 2024). |
ShortDate | Evaluates to the current datetime of the system in a yyyy-MM-dd format (for example, 2024-01-24). |
DateFormat:format | Evaluates to the current datetime of the system in the specified format (format ). See Sample Date Formats for the available datetime formats |
Vault:vaultitem | Evaluates to the value of the specified vault item. |
Examples
Some macros, such as %Ext% and %ShortDate%, do not require an argument, but others do. All macros that take an argument use the following syntax: %Macro:argument%
Here are some examples of the macros that take an argument:
- %Header:headername%: Where
headername
is the name of a header on a message. - %Header:mycustomheader% resolves to the value of the
mycustomheader
header set on the input message. - %Header:ponum% resolves to the value of the
ponum
header set on the input message. - %RegexFilename:pattern%: Where
pattern
is a regex pattern. For example,%RegexFilename:^([\w][A-Za-z]+)%
matches and resolves to the first word in the filename and is case insensitive (test_file.xml
resolves totest
). - %Vault:vaultitem%: Where
vaultitem
is the name of an item in the vault. For example,%Vault:companyname%
resolves to the value of thecompanyname
item stored in the vault. - %DateFormat:format%: Where
format
is an accepted date format (see Sample Date Formats for details). For example,%DateFormat:yyyy-MM-dd-HH-mm-ss-fff%
resolves to the date and timestamp on the file.
You can also create more sophisticated macros, as shown in the following examples:
- Combining multiple macros in one filename:
%DateFormat:yyyy-MM-dd-HH-mm-ss-fff%%EXT%
- Including text outside of the macro:
MyFile_%DateFormat:yyyy-MM-dd-HH-mm-ss-fff%
- Including text within the macro:
%DateFormat:'DateProcessed-'yyyy-MM-dd_'TimeProcessed-'HH-mm-ss%