Version 22.0.8473

• Overview
• Connector Configuration
  • Settings Tab
    • Host Configuration
    • Client Authentication
    • Server Authentication
    • Upload
    • Download
  • Automation Tab
    • Automation Settings
    • Performance
  • Advanced Tab
    • Server Commands
    • Proxy Settings
    • Other Settings
    • Miscellaneous
• Establishing a Connection
• Uploading Files
• Downloading Files
• Subfolder Headers

The SFTP Connector transfers files to and from a remote SFTP server.

Overview

Each SFTP Connector connects uploads to a specified folder and downloads from a set of folders on the remote SFTP server.

Files that reach the SFTP Connector in the CData Arc flow are uploaded to the upload target folder, and files found in the download target folder(s) are downloaded and entered into the Arc flow. The connector supports upload and download filters to only send and receive specified file names and/or file extensions.

Connector Configuration

This section contains all of the configurable connector properties.

Settings Tab

Host Configuration

Settings related to establishing the connection to the remote SFTP host.

• Connector Id The static name of the connector. All connector-specific files are held in a folder by the same name within the Data Directory.
• Connector Description An optional field to provide free-form description of the connector and its role in the flow.
• Remote Host The hostname or IP address of the SFTP server.
• Remote Port The port on which to connect to the SFTP server.

Client Authentication

Settings related to authenticating to the SFTP server.

• Authentication Mode The type of authentication to use with the SFTP server.
• User The username credential for logging in to the SFTP server.
• Password The password credential for logging in to the SFTP server.
• Client Certificate The certificate with a private key to use during public key authentication.
• Certificate Password The password required to access the Client Certificate.

Server Authentication

Settings related to verifying the SFTP server’s identity.

• Server Thumbprint The thumbprint of the certificate that identifies the SFTP server. This can also be set to a certificate file containing the server’s public key. When testing the connection, the connector will prompt to accept/trust the thumbprint presented by the server.

Upload

Settings related to uploading files to the SFTP server.

• Remote Path Files processed by the connector will be uploaded to this specified remote folder.

Download

Settings related to downloading from the SFTP server.

• Remote Path The remote folder from which files will be downloaded. Multiple paths can be specified in a comma-delimited list.
• File Mask A glob pattern that determines which files within the Remote Path should be downloaded (e.g. *.txt).
• Delete files (after received) Whether files that are successfully downloaded should be deleted from the SFTP server afterwards.

Automation Tab

Automation Settings

Settings related to the automatic processing of files by the connector.

• Upload Whether files arriving at the connector will automatically be uploaded.
• Retry Interval The amount of time before a failed upload is retried.
• Max Attempts The maximum number of times the connector will process the input file. Success is measured based on a successful server acknowledgement. If this is set to 0, the connect will retry the file indefinitely.
• Download Whether the connector should automatically poll the remote download path(s) for files to download.
• Download Interval The interval between automatic download attempts.
• Minutes The number of minutes to wait before downloading. Only applicable when Download Interval is set to Minute.
• Minutes Past the Hour The minutes offset for an hourly schedule. Only applicable when Download Interval is set to Hourly. For example, if this value is set to 5, the automation service will download at 1:05, 2:05, 3:05, etc.
• Time The time within a given day that the download should occur. Only applicable when Download Interval is set to Daily, or Weekly, or Monthly.
• Day The day on which the download should occur. Only applicable when Download Interval is set to Weekly or Monthly.
• Cron Expression An arbitrary string representing a cron expression that determines when the download should occur. Only applicable when Download Interval is set to Advanced.

Performance

Settings related to the allocation of resources to the connector.

• Max Workers The maximum number of worker threads that will be consumed from the threadpool to process files on this connector. If set, overrides the default setting from the Profile tab.
• Max Files The maximum number of files that will be processed by the connector each time worker threads are assigned to the connector. If set, overrides the default setting from the Profile tab.

Advanced Tab

Server Commands

An set of arbitrary SFTP commands to execute on the remote server at various times during connector execution: directly after connecting, after downloading each file, after uploading each file, before downloading each file, and before uploading each file.

Proxy Settings

A collection of settings that identify and authenticate to the proxy through which the SFTP connection should be routed.

Other Settings

Settings not included in the previous categories.

• Download Excluded Extensions A comma-delimited list of file extensions that will not be downloaded by the connector.
• Download Excluded Prefixes A comma-delimited list of prefixes that instructs the connector not to download filenames that begin with a specified prefix.
• Enable File Size Comparison Whether to cache downloaded file names and sizes; if True then files will only be downloaded if they have not been downloaded before or have changed in size.
• Enable Timestamp Comparison Whether to cache downloaded file names and last-modified timestamps; if True then files will only be downloaded if they have not been downloaded before or have been modified since they were downloaded.
• Receive Filter A glob pattern filter to determine which files should be downloaded from the remote storage (e.g. *.txt). Negative patterns may be used to indicate files that should not be downloaded (e.g. -*.tmp). This setting should be used when multiple File Mask patterns are desired. Multiple patterns may be separated by commas, with later filters taking priority except when an exact match is found.
• Send Filter A glob pattern filter to determine which files in the Send folder will be uploaded by the connector (e.g. *.txt). Negative patterns may be used to indicate files that should not be uploaded (e.g. -*.tmp). Multiple patterns may be separated by commas, with later filters taking priority except when an exact match is found.
• Local File Scheme A filemask for determining local file names as they are downloaded by the connector. The following macros may be used to reference contextual information:
  %ConnectorId%, %Filename%, %FilenameNoExt%, %Ext%, %ShortDate%, %LongDate%, %RegexFilename:%, %DateFormat:%.
  As an example: %FilenameNoExt%_%ShortDate%%Ext%
• Log Level The verbosity of logs generated by the connector. When requesting support, it is recommended to set this to Debug.
• Max Download Files The maximum number of files that will be downloaded from the SFTP server in one polling interval. If set to -1, no limit is applied.
• Monitor File Growth If enabled, the connector will store the file sizes of remote files between polling intervals, and only download files that do not change in size. This prevents the connector from downloading files that are still in the process of being written on the server.
• Parent Connector The connector from which settings should be inherited, unless explicitly overwritten within the existing connector configuration. Must be set to a connector of the same type as the current connector.
• Recurse Subdirectories Whether to download files in subfolders of the target remote path.
• Resume Downloads If enabled, when an error occurs during a file download, the connector will store the portion of the file that was downloaded and attempt to download the remainder of the file on the next download attempt.
• Resume Uploads If enabled, when an error occurs during a file upload, the connector will note the number of bytes that were transferred and attempt to upload the remainder of the file on the next upload attempt.
• SSH Encryption Algorithms A comma-delimited list of algorithms to use during SSH encryption. By default, all algorithms are enabled. The following algorithms are supported:
  
  aes256-ctr, aes256-cbc, aes192-ctr, aes192-cbc, aes128-ctr, aes128-cbc, 3des-ctr, 3des-cbc, cast128-cbc, blowfish-cbc, arcfour, arcfour128, arcfour256, [email protected], [email protected], [email protected]
• Temp Download Extension Temporary file extension to append to file downloads while the transfer is in progress.
• Temp Upload Extension Temporary file extension to append to file uploads while the transfer is in progress.
• Temp Upload Path Temporary path on the remote SFTP server where files will be uploaded. The file will be moved into the permanent upload path after the transfer is complete.
• Temp Upload Prefix Temporary file prefix to prepend to file uploads while the transfer is in progress.
• Timeout The duration the connector will wait for a connection response before throwing a timeout error.
• Log Subfolder Scheme Instructs the connector to group files in the Logs folder according to the selected interval. For example, the Weekly option instructs the connector to create a new subfolder each week and store all logs for the week in that folder. The blank setting tells the connector to save all logs directly in the Logs folder. For connectors that process many transactions, using subfolders can help keep logs organized and improve performance.
• Log Messages Whether the log entry for a processed file will include a copy of the file itself.
• Save to Sent Folder Whether files processed by the connector should be copied to the Sent folder for the connector.

Miscellaneous

Settings for specific use cases.

• Other Settings Allows configuration of hidden connector settings in a semicolon-separated list, like setting1=value1;setting2=value2. Normal connector use cases and functionality should not require use of these settings.

Establishing a Connection

The following settings are required to connect to a remote SFTP server:

• Remote Host
• Remote Port
• Authentication Mode
• User
• Password or Client Certificate/Certificate Password
• Server Thumbprint

When testing the connection to the server, the connector will prompt to accept/trust the thumbprint of the server’s public key.

Uploading Files

The SFTP Connector uploads files from the Input/Send Folder that match the Send Filter set in the Advanced tab. Files are automatically uploaded to the SFTP server if Upload Automation is enabled in the connector.

Remote Path should be set to the folder on the remote SFTP server where files should be uploaded. The application changes directories to the specified folder after logging in to the server.

If any files could not be uploaded, the connector attempts to send them later after Retry Interval minutes have elapsed. This process continues until the Max Attempts setting has been reached, after which the connector will raise an error and the file will no longer be sent automatically.

Downloading Files

Downloaded files are displayed in the Output/Receive Folder of the connector, or passed along to the next connector in the flow. Files are automatically downloaded from the SFTP server if Download Automation is enabled in the connector, and can be manually downloaded by clicking the Receive button in the Output tab.

Remote Path should be set to the folder on the remote SFTP server from which files should be downloaded. The application changes directories to the specified folder after logging in to the server. The File Mask property filters files such that only filenames matching the glob pattern are downloaded from the server.

The Delete files (after download) setting can be used to delete the files on the remote server after successfully downloading them. Within the Output tab, click the Receive to test these settings and retrieve the matching files from the server.

Subfolder Headers

If Recurse Subdirectories is set to True when receiving a file from the Remote Path in the SFTP connector:

• The message for the file that is received will contain a Subfolder header in its metadata.
• This header contains the subfolder, relative to the Remote Path in the SFTP connector, that the file was received from.
• This subfolder header will be supported by other connectors that support a Subfolder header in Send operations.

When sending files, if a Subfolder header is present on the message that is sent to the Upload Path in the SFTP connector:

• The file will be placed in the subfolder, relative to the Upload Path in the SFTP connector, that is specified in the subfolder header.
• The subfolder will be created if permissions are available.