RSS Connector

Version 23.4.8839


RSS Connector


RSS Connectors support downloading remote RSS and Atom feeds.

Overview

RSS connectors expose a simple interface to automatically poll for downloadable content from a specific RSS or Atom feed. The connector can be configured to only process new and updated content, and can perform both HTTP and TLS authentication to the remote service.

Connector Settings

Settings Tab

Configuration

Settings related to the core configuration of the connector.

  • Connector Id The static, unique identifier for the connector.
  • Connector Type Displays the connector name and a description of what it does.
  • Connector Description An optional field to provide a free-form description of the connector and its role in the flow.
  • Feed URL The HTTP or HTTPS endpoint from where RSS content should be downloaded.
  • Feed Type Whether the remote URL is an RSS or Atom feed.
  • Use If-Modified-Since When enabled, the connector includes an If-Modified-Since header in the outgoing request that equals the latest publication date and time from the previous download attempt. This causes the server to filter responses based on what content is new or has been modified recently.
  • Process Only New Items When enabled, the connector stores the last modified date from previous downloads, and filters out results that are not new or modified since that date. The result is the same as Use If-Modified-Since, but the filtering is done on the client-side rather than sending a filter condition to the server.

Authentication

Settings related to authenticating with the RSS service.

  • Authorization Type The type of authentication to use with the RSS service. See the Authentication section for details.

TLS Server Authentication

Settings related to verifying the TLS server’s identity.

  • TLS Server Certificate The public key certificate used to verify the identity of a TLS server. You can upload a certificate, leave the field blank to allow the underlying OS/JVM to perform certificate validation, or set it to Any Certificate to trust the target server’s identity. Use Any Certificate with caution: the certificate verifies that you are connecting to the intended server.

Automation Tab

Automation Settings

Settings related to the automatic processing of files by the connector.

  • Receive Whether the connector should automatically send RSS download requests according to a specified interval.
  • Receive Interval The interval between automatic requests.
  • Minutes Past the Hour The minutes offset for an hourly schedule. Only applicable when the interval setting above is set to Hourly. For example, if this value is set to 5, the automation service downloads at 1:05, 2:05, 3:05, etc.
  • Time The time of day that the attempt should occur. Only applicable when the interval setting above is set to Daily, Weekly, or Monthly.
  • Day The day on which the attempt should occur. Only applicable when the interval setting above is set to Weekly or Monthly.
  • Minutes The number of minutes to wait before attempting the download. Only applicable when the interval setting above is set to Minute.
  • Cron Expression A five-position string representing a cron expression that determines when the attempt should occur. Only applicable when the interval setting above is set to Advanced.

Alerts Tab

Settings related to configuring alerts and Service Level Agreements (SLAs).

Connector Email Settings

Before you can execute SLAs, you need to set up email alerts for notifications. Clicking Configure Alerts opens a new browser window to the Settings page where you can set up system-wide alerts. See Alerts for more information.

Service Level Agreement (SLA) Settings

SLAs enable you to configure the volume you expect connectors in your flow to send or receive, and to set the time frame in which you expect that volume to be met. CData Arc sends emails to warn the user when an SLA is not met, and marks the SLA as At Risk, which means that if the SLA is not met soon, it will be marked as Violated. This gives the user an opportunity to step in and determine the reasons the SLA is not being met, and to take appropriate actions. If the SLA is still not met at the end of the at-risk time period, the SLA is marked as violated, and the user is notified again.

To define an SLA, click Add Expected Volume Criteria.

  • If your connector has separate send and receive actions, use the radio buttons to specify which direction the SLA pertains to.
  • Set Expect at least to the minimum number of transactions (the volume) you expect to be processed, then use the Every fields to specify the time frame.
  • By default, the SLA is in effect every day. To change that, uncheck Everyday then check the boxes for the days of the week you want.
  • Use And set status to ‘At Risk’ to indicate when the SLA should be marked as at risk.
  • By default, notifications are not sent until an SLA is in violation. To change that, check Send an ‘At Risk’ notification.

The following example shows an SLA configured for a connector that expects to receive 1000 files every day Monday-Friday. An at-risk notification is sent 1 hour before the end of the time period if the 1000 files have not been received.

Advanced Tab

Proxy Settings

Settings for using a proxy with the connector.

  • Use Global A toggle that instructs the connector to use the proxy settings configured under the Settings tab of Arc.
  • Proxy Type The type of proxy to use. Select None to not use a proxy. Otherwise, select Tunnel, SOCKS4, SOCKS5, or HTTP.
  • Proxy Host The proxy server. Formatting depends on the Proxy Type selected.
  • Proxy Port The port to use when connecting to the proxy.
  • Proxy User The username to use when connecting to the proxy.
  • Proxy Password The password for the associated username.
  • Authentication Scheme The protocol to use when connecting to the proxy. Options include Basic, Digest, Proprietary, and NTLM.

Advanced Settings

Settings not included in the previous categories.

  • Include Channel Whether to include data about the RSS channel in the output.
  • Processing Delay The amount of time (in seconds) by which the processing of files placed in the Input folder is delayed. This is a legacy setting. Best practice is to use a File connector to manage local file systems instead of this setting.
  • TLS Enabled Protocols The list of TLS/SSL protocols supported when establishing outgoing connections. Best practice is to only use TLS protocols. Some obsolete operating systems do not support TLS 1.2.
  • Skip Processed Items When enabled, the connector maintains a checksum of each item downloaded, and does not download the same content more than once.
  • Timeout The duration in seconds to wait for a response from the RSS server before throwing a timeout error.
  • Use ETag Servers might include a checksum of the RSS content as the ETag (entity tag). Enable this setting to have the connector include the ETag from the previous response as the If-None-Match header in the outgoing request.

Message

Message settings determine how the connector searches for messages and manages them after processing. You can save messages to your Sent folder or you can group them based on a Sent folder scheme, as described below.

  • Save to Sent Folder A toggle that instructs the connector to keep a copy of sent messages in the Sent folder.
  • Sent Folder Scheme Instructs the connector to group files in the Sent folder according to the selected interval. For example, the Weekly option instructs the connector to create a new subfolder each week and store all sent files for the week in that folder. The blank setting instructs the connector to save all files directly in the Sent folder. For connectors that process many transactions, using subfolders can help keep files organized and improve performance.

Logging

  • Log Level The verbosity of logs generated by the connector. When you request support, set this to Debug.
  • Log Subfolder Scheme Instructs the connector to group files in the Logs folder according to the selected interval. For example, the Weekly option instructs the connector to create a new subfolder each week and store all logs for the week in that folder. The blank setting tells the connector to save all logs directly in the Logs folder. For connectors that process many transactions, using subfolders helps keep logs organized and improves performance.
  • Log Messages Check this to have the log entry for a processed file include a copy of the file itself. If you disable this, you might not be able to download a copy of the file from the Input or Output tabs.

Miscellaneous

Miscellaneous settings are for specific use cases.

  • Other Settings Enables you to configure hidden connector settings in a semicolon-separated list (for example, setting1=value1;setting2=value2). Normal connector use cases and functionality should not require the use of these settings.

Establishing a Connection

A valid target Feed URL is required to establish a connection to any RSS service. Some services might also require authentication or a set of custom headers in order to consume the service.

The connector silently supports both HTTP and HTTPS target URLs. If the target URL is an HTTPS URL, set the TLS Server Certificate to the public key certificate identifying the server. To implicitly trust the target endpoint, set the field to Any Certificate.

Authentication

The RSS connector supports HTTP authentication in the following formats:

  • Basic (plain text)
  • Digest (encrypted)
  • NTLM
  • Negotiate
  • Proprietary

The connector provides the credentials specified in Username and Password with each request to the remote RSS service.