Adding a Connection to MongoDB

1. In the application console, navigate to the Connections page.
2. At the Add Connections panel, select the icon for the connection you want to add.
3. If the MongoDB icon is not available, click the Add More icon to download and install the MongoDB connector from the CData site.

For required properties, see the Settings tab.

For connection properties that are not typically required, see the Advanced tab.

Connecting to MongoDB

Set the following connection properties to connect to a single MongoDB instance:

• Server: Set this to the name or address of the server your MongoDB instance is running on. You can specify the port here or in Port.
• Database: Set this to the database you want to read from and write to.

Connecting to MongoDB Using DNS Seed Lists

• Server: Set this to "mongodb+srv://"" + the name of the server your MongoDB instance is running on. You can specify the port here or in Port.
• Database: Set this to the database you want to read from and write to.
• DNSServer: Set this to the hostname of a DNSServer that can resolve the necessary DNS entries.

Connecting to Replica Sets

To connect to a replica set, set the following in addition to the preceding connection properties:

• ReplicaSet: Set this to a comma-separated list of secondary servers in the replica set, specified by address and port.
• SlaveOK: Set this to true if you want to read from secondary (slave) servers.
• ReadPreference: Set this to fine-tune how the Sync App reads from secondary servers.

Securing MongoDB Connections

You can set UseSSL to negotiate SSL/TLS encryption when you connect.

Authenticating MongoDB Connections

Supported AuthScheme types (MONGODB-CR,SCRAM-SHA-1,SCRAM-SHA-256,PLAIN,GSSAPI) are challenge-response authentication and LDAP.

Challenge-Response

In challenge-response authentication, the User and Password properties correspond to a username and password stored in a MongoDB database. If you want to connect to data from one database and authenticate to another database, set both Database and AuthDatabase.

LDAP

To use LDAP authentication, set AuthDatabase to "$external" and set AuthScheme to PLAIN. This value specifies the SASL PLAIN mechanism; note that this mechanism transmits credentials over plaintext, so it is not suitable for use without TLS/SSL on untrusted networks.

X.509 Certificates

Set AuthScheme to X509 to use X.509 certificate authentication.

Connecting to an Amazon DocumentDB Cluster

Before you can connect to Amazon DocumentDB, you will first need to, ensure your Amazon DocumentDB cluster and the EC2 instance containing the mongo shell are currently running.

Next, configure an SSH tunnel to the EC2 instance as follows.

1. From the AWS management console, select Services -> Database -> Amazon DocumentDB. From the DocumentDB management page, select Clusters, then click your cluster.
2. Under the Connect section, note the --host value and its port found in the sample connection string.
3. Navigate to Services -> Compute -> EC2. Select Running instances.
4. Select your instance, then click the Connect button.
5. Under the Example section, note the value identifying the instance and user, shown in the form <ami_username>@<Public DNS>
6. In your preferred SSH client, establish a connection to your EC2 instance using the Host Name from the EC2 instance's Connect page (username@publicDNS) and Port 22.
7. Provide your EC2 instance's private key file (in Putty, you will need to convert the keys from .pem to .ppk) for authentication.
8. Configure an SSH tunnel using the port and host name from the DocumentDB cluster page.
9. Establish the connection to the EC2 virtual machine.

Specify the following to connect to the DocumentDB cluster.

• Server: Set this to the machine name which is hosting the SSH tunnel.
• Port: Set this to the port the SSH tunnel is hosted on.
• User: Set this to the master username used to provision the DocumentDB cluster.
• Password: Set this to the master password set when provisioning the DocumentDB cluster.
• UseSSL: Set this to true.
• UseFindAPI Set this to true.

Connecting to CosmosDB with the MongoDB API

To obtain the connection string needed to connect to a Cosmos DB account using the MongoDB API, log in to the Azure Portal, select Azure Cosmos DB, and select your account. In the Settings section, click Connection String and set the following values.

• Server: Set this to the Host value, the FQDN of the server provisioned for your account. You can also specify the port here or in Port.
• Port: Set this to the port.
• Database: Set this to the database you want to read from and write to.
• User: Set this to the database user.
• Password: Set this to the user's password.

When you connect to Atlas, ObjectRocket, or another database-as-a-service provider, there typically are a few variations on the procedure outlined in Establishing a Connection. The following sections show how to obtain the necessary connection properties for several popular services.

Atlas

You can authenticate to MongoDB Atlas with a MongoDB user or an LDAP user. The following sections show how to map Atlas connection strings to Sync App connection properties. To obtain the Atlas connection string, follow the steps below:

1. In the Clusters view, click Connect for the cluster you want to connect to.
2. Click Connect Your Application.
3. Select either driver option to display a connection string.

Prerequisites

In addition to creating a MongoDB user and/or setting up LDAP, your Atlas project's white-list must include the IP address of the machine the Sync App is connecting from. To add an IP address to the white-list, select the Security tab in the Clusters view and then click IP Whitelist -> Add IP Address.

MongoDB User Credentials

Below is an example connection string providing a MongoDB user's credentials.

mongodb://USERNAME:[email protected]:27017,cluster0-shard-00-01.mongodb.net:27017,cluster0-shard-00-02.mongodb.net:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin

• AuthScheme: SCRAM-SHA-1 for MongoDB versions 3.0, 3.2, 3.4, and 3.6.
• Server: The first server in the replica set. Or, you can specify a primary or secondary server here (the Sync App queries the servers in Server and ReplicaSet to find the primary).
  

  cluster0-shard-00-00.mongodb.net

• Port: The port the server is running on (27017 is the default).
• ReplicaSet: The other servers in the replica set. Server and ReplicaSet together specify all instances in the MongoDB replica set. Specify both the server name and port in ReplicaSet.
  

  mycluster0-shard-00-01.mongodb.net:27017,mycluster0-shard-00-02.mongodb.net:27017

• SlaveOK: true to allow reading from secondary (slave) servers in the replica set.
• AuthDatabase: "admin" to connect to MongoDB Atlas. All MongoDB users for Atlas are associated with the admin database, their authentication database.
• Database: The database you want to read from and write to.

• User: The username of a MongoDB user you added to your MongoDB project.

• Password: The password of the MongoDB user.

• UseSSL: true. Atlas requires TLS/SSL.

LDAP

The following list shows the MongoDB Atlas requirements for authenticating with an LDAP user. Below is an example command to connect with the mongo client:

mongo "mongodb://cluster0-shard-00-00.mongodb.net:27017,cluster0-shard-00-01.mongodb.net:27017,cluster0-shard-00-02.mongodb.net:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=$external" --authenticationMechanism PLAIN --username cn=rob,cn=Users,dc=atlas-ldaps-01,dc=myteam,dc=com 

• Server: The first server in the replica set. Or, you can specify another primary or secondary server here (the Sync App queries the servers in Server and ReplicaSet to find the primary). For example:
  

  cluster0-shard-00-00.mongodb.net

• Port: The port the server is running on (27017 is the default).
• ReplicaSet: The other servers in the replica set. Server and ReplicaSet together specify all instances in the MongoDB replica set. Below is an example value:
  

  mycluster0-shard-00-01.mongodb.net:27017,mycluster0-shard-00-02.mongodb.net:27017

• SlaveOK: true to allow reading from secondary (slave) servers in the replica set.

• AuthScheme: PLAIN in LDAP authentication.

• Database: The database you want to read from and write to.

• AuthDatabase: "$external" to authenticate with an LDAP user.

• User: The full Distinguished Name (DN) of a user in your LDAP server as the Atlas username. For example:
  

  cn=rob,cn=Users,dc=atlas-ldaps-01,dc=myteam,dc=com

• Password: The password of the LDAP user.

• UseSSL: true. Atlas requires TLS/SSL.

ObjectRocket

To connect to ObjectRocket, you authenticate with the credentials for a database user. You can obtain the necessary connection properties from the control panel: On the Instances page, select your instance and then select the Connect menu to display a MongoDB connection string.

Prerequisites

In addition to adding a user for your database, you also need to allow access to the IP address for the machine the Sync App is connecting from. You can configure this by selecting your instance on the Instances page and then clicking Add ACL.

MongoDB User

mongodb://YOUR_USERNAME:[email protected]:52826,abc123-d4-2.mongo.objectrocket.com:52826,abc123-d4-1.mongo.objectrocket.com:52826/YOUR_DATABASE_NAME?replicaSet=89c04c5db2cf403097d8f2e8ca871a1c

• Server: The first server in the replica set. Click Replica Set to obtain the server names. Or, you can specify another primary or secondary server here (the Sync App queries the servers in Server and ReplicaSet to find the primary).
  

  abc123-d4-0.mongo.objectrocket.com

• Port: The port the server is running on (27017 is the default).
• ReplicaSet: The other servers in the replica set. Server and ReplicaSet together specify all instances in the MongoDB replica set. Below is an example value:
  

  abc123-d4-2.mongo.objectrocket.com:52826,abc123-d4-1.mongo.objectrocket.com:52826

• Database: The database you want to read from and write to. Note that this is also the authentication database for the user you are connecting with; database users cannot interact with other databases outside their database in ObjectRocket.
• User: The username of a MongoDB user you defined for the Database.
• Password: The password for the database user.
• UseSSL: true to enable TLS/SSL.

MongoDB is a schemaless, document database that provides high performance, availability, and scalability. These features are not necessarily incompatible with a standards-compliant query language like SQL-92. In this section we will show various schemes that the Sync App offers to bridge the gap with relational SQL and a document database.

Working with MongoDB Objects as Tables

The Sync App models the schemaless MongoDB objects into relational tables and translates SQL queries into MongoDB queries to get the requested data. See Query Mapping for more details on how various MongoDB operations are represented as SQL.

Discovering Schemas Automatically

The Automatic Schema Discovery scheme automatically finds the data types in a MongoDB object by scanning a configured number of rows of the object. You can use RowScanDepth, FlattenArrays, and FlattenObjects to control the relational representation of the collections in MongoDB. You can also write Free-Form Queries not tied to the schema.

The Sync App automatically infers a relational schema by inspecting a series of MongoDB documents in a collection. You can use the RowScanDepth property to define the number of documents the Sync App will scan to do so. The columns identified during the discovery process depend on the FlattenArrays and FlattenObjects properties.

Flattening Objects

If FlattenObjects is set, all nested objects will be flattened into a series of columns. For example, consider the following document:

{
  id: 12,
  name: "Lohia Manufacturers Inc.",
  address: {street: "Main Street", city: "Chapel Hill", state: "NC"},
  offices: ["Chapel Hill", "London", "New York"],
  annual_revenue: 35,600,000
}

If FlattenObjects is not set, then the address.street, address.city, and address.state columns will not be broken apart. The address column of type string will instead represent the entire object. Its value would be {street: "Main Street", city: "Chapel Hill", state: "NC"}. See JSON Functions for more details on working with JSON aggregates.

Flattening Arrays

The FlattenArrays property can be used to flatten array values into columns of their own. This is only recommended for arrays that are expected to be short, for example the coordinates below:

"coord": [ -73.856077, 40.848447 ]

It is best to leave other unbounded arrays as they are and piece out the data for them as needed using JSON Functions.

As discussed in Automatic Schema Discovery, intuited table schemas enable SQL access to unstructured MongoDB data. JSON Functions enable you to use standard JSON functions to summarize MongoDB data and extract values from any nested structures. Custom Schema Definitions enable you to define static tables and give you more granular control over the relational view of your data; for example, you can write schemas defining parent/child tables or fact/dimension tables. However, you are not limited to these schemes.

After connecting you can query any nested structure without flattening the data. Any relations that you can access with FlattenArrays and FlattenObjects can also be accessed with an ad hoc SQL query.

Let's consider an example document from the following Restaurant data set:

 
{
  "address": {
    "building": "1007",
    "coord": [
      -73.856077,
      40.848447
    ],
    "street": "Morris Park Ave",
    "zipcode": "10462"
  },
  "borough": "Bronx",
  "cuisine": "Bakery",
  "grades": [
    {
      "grade": "A",
      "score": 2,
      "date": {
        "$date": "1393804800000"
      }
    },
    {
      "date": {
        "$date": "1378857600000"
      },
      "grade": "B",
      "score": 6
    },
    {
      "score": 10,
      "date": {
        "$date": "1358985600000"
      },
      "grade": "C"
    }
  ],
  "name": "Morris Park Bake Shop",
  "restaurant_id": "30075445"
} 

SELECT [address.building], [grades.1.grade] FROM restaurants WHERE restaurant_id = '30075445'

It is possible to retrieve an array of documents as if it were a separate table. Take the following JSON structure from the restaurants collection for example:

{
  "_id" : ObjectId("568c37b748ddf53c5ed98932"),
  "address" : {
    "building" : "1007",
    "coord" : [-73.856077, 40.848447],
    "street" : "Morris Park Ave",
    "zipcode" : "10462"
  },
  "borough" : "Bronx",
  "cuisine" : "Bakery",
  "grades" : [{
      "date" : ISODate("2014-03-03T00:00:00Z"),
      "grade" : "A",
      "score" : 2
    }, {
      "date" : ISODate("2013-09-11T00:00:00Z"),
      "grade" : "A",
      "score" : 6
    }, {
      "date" : ISODate("2013-01-24T00:00:00Z"),
      "grade" : "A",
      "score" : 10
    }, {
      "date" : ISODate("2011-11-23T00:00:00Z"),
      "grade" : "A",
      "score" : 9
    }, {
      "date" : ISODate("2011-03-10T00:00:00Z"),
      "grade" : "B",
      "score" : 14
    }],
  "name" : "Morris Park Bake Shop",
  "restaurant_id" : "30075445"
}

SELECT * FROM [restaurants.grades]

You may also want to include information from the base restaurants table. You can do this with a join. Flattened arrays can only be joined with the root document. The Sync App expects the left part of the join is the array document you want to flatten vertically. Set SupportEnhancedSQL to false to join nested MongoDB documents -- this type of query is supported through the MongoDB API.

SELECT [restaurants].[restaurant_id], [restaurants.grades].* FROM [restaurants.grades] JOIN [restaurants] WHERE [restaurants].name = 'Morris Park Bake Shop'

It's also possible to build queries targeting arrays within other arrays.

Consider this sample Inventory collection:

{
	"_id": {
		"$oid": "xxxxxxxxxxxxxxxxxxxxxx"
	},
	"Company Branch": "Main Branch",
	"ItemList": [
		{
			"item": "journal",
			"instock": [
				{
					"warehouse": "A",
					"qty": 15
				},
				{
					"warehouse": "B",
					"qty": 45
				}
			]
		},
		{
			"item": "paper",
			"instock": [
				{
					"warehouse": "A",
					"qty": 50
				},
				{
					"warehouse": "B",
					"qty": 5
				}
			]
		}
	]
}

Insert data into the nested arrays using the syntax of <parent array>.<index>.<child array>, as follows:

INSERT INTO [Inventory.ItemList] (p_id, item, [instock.0.warehouse], [instock.0.qty], [instock.0.price]) VALUES ('xxxxxxxxxxxxxxxxxxxxxx', 'NoteBook', 'B', 20, '5$')

The Inventory collection after executing the INSERT statement:

{
	"_id": {
		"$oid": "xxxxxxxxxxxxxxxxxxxxxx"
	},
	"Company Branch": "Main Branch",
	"ItemList": [
		{
			"item": "journal",
			"instock": [
				{
					"warehouse": "A",
					"qty": 15
				},
				{
					"warehouse": "B",
					"qty": 45
				}
			]
		},
		{
			"item": "paper",
			"instock": [
				{
					"warehouse": "A",
					"qty": 50
				},
				{
					"warehouse": "B",
					"qty": 5
				}
			]
		},
		{
			"item": "NoteBook",
			"instock": [
				{
					"warehouse": "B",
					"qty": 20,
					"price": "5$"
				}
			]
		}
	]
}

The Sync App can return JSON structures as column values. The Sync App enables you to use standard SQL functions to work with these JSON structures. The examples in this section use the following array:

[
     { "grade": "A", "score": 2 },
     { "grade": "A", "score": 6 },
     { "grade": "A", "score": 10 },
     { "grade": "A", "score": 9 },
     { "grade": "B", "score": 14 }
]

JSON_EXTRACT

SELECT Name, JSON_EXTRACT(grades,'[0].grade') AS Grade, JSON_EXTRACT(grades,'[0].score') AS Score FROM Students;

JSON_COUNT

SELECT Name, JSON_COUNT(grades,'[x]') AS NumberOfGrades FROM Students;

JSON_SUM

SELECT Name, JSON_SUM(score,'[x].score') AS TotalScore FROM Students;

JSON_MIN

SELECT Name, JSON_MIN(score,'[x].score') AS LowestScore FROM Students;

JSON_MAX

SELECT Name, JSON_MAX(score,'[x].score') AS HighestScore FROM Students;

DOCUMENT

The DOCUMENT function can be used to retrieve the entire document as a JSON string. See the following query and its result as an example:

SELECT DOCUMENT(*) FROM Customers;

{ "id": 12, "name": "Lohia Manufacturers Inc.", "address": { "street": "Main Street", "city": "Chapel Hill", "state": "NC"}, "offices": [ "Chapel Hill", "London", "New York" ], "annual_revenue": 35,600,000 }

The Sync App maps SQL queries into the corresponding MongoDB queries. A detailed description of all the transformations is out of scope, but we will describe some of the common elements that are used. The Sync App takes advantage of MongoDB features such as the aggregation framework to compute the desired results.

SELECT Queries

SELECT * FROM Users

db.users.find()

SELECT user_id, status 
FROM Users

db.users.find(
  {}, 
  { user_id: 1, status: 1, _id: 0 }
)

SELECT * 
FROM Users 
WHERE status = 'A'

db.users.find( 
  { status: "A" }
)

SELECT * 
FROM Users 
WHERE status = 'A' OR age=50

db.users.find(
  { $or: [ { status: "A" }, 
           { age: 50 } ] }
)

SELECT * 
FROM Users 
WHERE name LIKE 'A%'

db.users.find(
  {name: /^a/}
)

SELECT * FROM Users 
WHERE status = 'A'
ORDER BY user_id ASC

db.users.find( { status: "A" }.sort( { user_id: 1 } )

SELECT * 
FROM Users 
WHERE status = 'A' 
ORDER BY user_id DESC

db.users.find( {status: "A" }.sort( {user_id: -1} )

Aggregate Queries

SELECT Count(*) As Count 
FROM Orders

db.orders.aggregate( [ 
  { 
    $group: { 
      _id: null, 
      count: { $sum: 1 } 
    } 
  } 
] )

SELECT Sum(price) As Total 
FROM Orders

db.orders.aggregate( [ 
  { 
    $group: { 
      _id: null, 
      total: { $sum: "$price" } 
    }
  } 
] )

SELECT cust_id, Sum(price) As total 
FROM Orders 
GROUP BY cust_id 
ORDER BY total

db.orders.aggregate( [ 
  { 
    $group: { 
      _id: "$cust_id", 
      total: { $sum: "$price" } 
    } 
  } ,
  { $sort: {total: 1 } }
] )

SELECT cust_id, ord_date, Sum(price) As total 
FROM Orders 
GROUP BY cust_id, ord_date 
HAVING total > 250

db.orders.aggregate( [ 
  { 
    $group: { 
      _id: { 
        cust_id: "$cust_id", 
        ord_date: { 
          month: { $month: "$ord_date" }, 
          day: { $dayOfMonth: "$ord_date" }, 
          year: { $year: "$ord_date"} 
        } 
      }, 
      total: { $sum: "$price" } 
    }
  }, 
  { $match: { total: { $gt: 250 } } } 
] )

INSERT Statements

INSERT INTO users (user_id, age, status, [address.city], [address.postalcode]) 
VALUES ('bcd001', 45, 'A', 'Chapel Hill', 27517)

db.users.insert( 
  { user_id: "bcd001", age: 45, status: "A", address:{ city:"Chapel Hill", postalCode:27514} }
) 

INSERT INTO t1 ("c1") VALUES (('a1', 'a2', 'a3'))

db.users.insert({"c1": ['a1', 'a2', 'a3']})

INSERT INTO t1 ("c1") VALUES (())

db.users.insert({"c1": []})

INSERT INTO t1 ("a.b.c.c1") VALUES (('a1', 'a2', 'a3'))

db.users.insert("a":{"b":{"c":{"c1":['a1','a2', 'a3']}}})

Update Statements

UPDATE users 
SET status = 'C', [address.postalcode] = 90210
WHERE age > 25

db.users.update( 
  { age: { $gt: 25 } }, 
  { $set: { status: "C", address.postalCode: 90210 }, 
  { multi: true }
) 

Delete Statements

DELETE FROM users WHERE status = 'D'

db.users.remove( { status: "D" } )

You can extend the table schemas created with Automatic Schema Discovery by saving them into schema files. The schema files have a simple format that makes the schemas to edit.

Generating Schema Files

Set GenerateSchemaFiles to "OnStart" to persist schemas for all tables when you connect. You can also generate table schemas as needed: Set GenerateSchemaFiles to "OnUse" and execute a SELECT query to the table.

For example, consider a schema for the restaurants data set. This is a sample data set provided by MongoDB. To download the data set, follow the Getting Started with MongoDB guide.

Below is an example document from the collection:

{
  "address":{
    "building":"461",
      "coord":[
        -74.138492,
        40.631136
      ],
      "street":"Port Richmond Ave",
      "zipcode":"10302"
   },
   "borough":"Staten Island",
   "cuisine":"Other",
   "name":"Indian Oven",
   "restaurant_id":"50018994"
}

Importing the MongoDB Restaurant Data Set

You can use the mongoimport utility to import the data set:

mongoimport --db test --collection restaurants --drop --file dataset.json

Customizing a Schema

When GenerateSchemaFiles is set, the Sync App saves schemas into the folder specified by the Location property. You can then change column behavior in the resulting schema.

The following schema uses the other:bsonpath property to define where in the collection to retrieve the data for a particular column. Using this model you can flatten arbitrary levels of hierarchy.

The collection attribute specifies the collection to parse. The collection attribute gives you the flexibility to use multiple schemas for the same collection. If collection is not specified, the filename determines the collection that is parsed.

Below are the column definitions and the collection to extract the column values from. In Custom Schema Example, you will find the complete schema.

<rsb:script xmlns:rsb="http://www.rssbus.com/ns/rsbscript/2">  

  <rsb:info title="StaticRestaurants" description="Custom Schema for the MongoDB restaurants data set.">  
    <!-- Column definitions -->
    <attr  name="borough"   xs:type="string" other:bsonpath="$.borough"              />
    <attr  name="cuisine"   xs:type="string" other:bsonpath="$.cuisine"              />
    <attr  name="building"  xs:type="string" other:bsonpath="$.address.building"     />
    <attr  name="street"    xs:type="string" other:bsonpath="$.address.street"       />
    <attr  name="latitude"  xs:type="double" other:bsonpath="$.address.coord.0"      />
    <attr  name="longitude" xs:type="double" other:bsonpath="$.address.coord.1"      />
  </rsb:info>  

  <rsb:set attr="collection" value="restaurants"/>

</rsb:script>

This section contains an example of a complete schema that has been automatically generated by GenerateSchemaFiles. Set the Location property to the file directory that will contain the schema file. The schema consists of the following parts:

• The info section enables a relational view of a MongoDB object. For more details, see Custom Schema Definitions.

• The collection attribute specifies the name of the collection to be parsed. The collection attribute can be used to define multiple schemas for the same collection. If collection is not specified, the filename determines the collection that is parsed.

<rsb:script xmlns:rsb="http://www.rssbus.com/ns/rsbscript/2">  

  <rsb:info title="StaticRestaurants" description="Automatic GenerateSchemaFile">  
    <!-- Column definitions -->
    <attr  name="borough"           xs:type="string"  other:bsonpath="$.borough"          />
    <attr  name="cuisine"           xs:type="string"  other:bsonpath="$.cuisine"          />
    <attr  name="address_building"  xs:type="string"  other:bsonpath="$.address.building" />
    <attr  name="address_street"    xs:type="string"  other:bsonpath="$.address.street"   />
    <attr  name="address_coord_0"   xs:type="double"  other:bsonpath="$.address.coord.0"  />
    <attr  name="address_coord_1"   xs:type="double"  other:bsonpath="$.address.coord.1"  />
  </rsb:info>  

  <rsb:set attr="collection" value="restaurants"/>
</rsb:script>

Data Type Mappings

The Sync App maps types from the data source to the corresponding data type available in the schema. The table below documents these mappings.

Stored procedures are function-like interfaces that extend the functionality of the Sync App beyond simple SELECT/INSERT/UPDATE/DELETE operations with MongoDB.

Stored procedures accept a list of parameters, perform their intended function, and then return any relevant response data from MongoDB, along with an indication of whether the procedure succeeded or failed.

MongoDB Connector for CData Sync Stored Procedures

Inserts a JSON document into a MongoDB collection without modification, preserving its original structure.

Input

Result Set Columns

Generates a schema file based on the structure of a specified MongoDB collection, defining fields and data types.

CreateSchema

Creates a local schema file (.rsd) from an existing table or view in the data model.

The schema file is created in the directory set in the Location connection property when this procedure is executed. You can edit the file to include or exclude columns, rename columns, or adjust column datatypes.

The Sync App checks the Location to determine if the names of any .rsd files match a table or view in the data model. If there is a duplicate, the schema file will take precedence over the default instance of this table in the data model. If a schema file is present in Location that does not match an existing table or view, a new table or view entry is added to the data model of the Sync App.

Input

Result Set Columns

Creates a schema definition for a MongoDB collection, mapping document structure to a tabular format.

Input

Result Set Columns

Executes a pass-through query to retrieve specific documents from a MongoDB collection, allowing for advanced filtering and projection.

Input

Result Set Columns

Retrieves an entire MongoDB document as a JSON-formatted string, maintaining its native structure.

Input

Result Set Columns

This section details a selection of advanced features of the MongoDB Sync App.

User Defined Views

The Sync App supports the use of user defined views, virtual tables whose contents are decided by a pre-configured user defined query. These views are useful when you cannot directly control queries being issued to the drivers. For an overview of creating and configuring custom views, see User Defined Views .

SSL Configuration

Use SSL Configuration to adjust how Sync App handles TLS/SSL certificate negotiations. You can choose from various certificate formats;. For further information, see the SSLServerCert property under "Connection String Options" .

Firewall and Proxy

Configure the Sync App for compliance with Firewall and Proxy, including Windows proxies. You can also set up tunnel connections.

Query Processing

For further information, see Query Processing.

Logging

Customizing the SSL Configuration

To enable TLS, set UseSSL to True.

With this configuration, the Sync App attempts to negotiate TLS with the server. The server certificate is validated against the default system trusted certificate store. You can override how the certificate gets validated using the SSLServerCert connection property.

To specify another certificate, see the SSLServerCert connection property.

Client SSL Certificates

The MongoDB Sync App also supports setting client certificates. Set the following to connect using a client certificate.

• SSLClientCert: The name of the certificate store for the client certificate.
• SSLClientCertType: The type of key store containing the TLS/SSL client certificate.
• SSLClientCertPassword: The password for the TLS/SSL client certificate.
• SSLClientCertSubject: The subject of the TLS/SSL client certificate.

Connecting Through a Firewall or Proxy

Set the following properties:

• To use a proxy-based firewall, set FirewallType, FirewallServer, and FirewallPort.
• To tunnel the connection, set FirewallType to TUNNEL.
• To authenticate, specify FirewallUser and FirewallPassword.
• To authenticate to a SOCKS proxy, additionally set FirewallType to SOCKS5.

The connection string properties are the various options that can be used to establish a connection. This section provides a complete list of the options you can configure in the connection string for this provider. Click the links for further details.

Authentication

Kerberos

SSL

SSH

Firewall

Logging

Schema

Miscellaneous

This section provides a complete list of the Authentication properties you can configure in the connection string for this provider.

The authentication mechanism that MongoDB will use to authenticate the connection.

Remarks

Accepted values are MONGODB-CR, SCRAM-SHA-1, SCRAM-SHA-256, GSSAPI, PLAIN, and NONE. The following authentication types correspond to the authentication values.

Authenticating with Challenge-Response

Generally, this property does not need to be set for this authentication type, as the Sync App uses different challenge-response mechanisms by default to authenticate a user to different versions of MongoDB.

• MongoDB 2: MongoDB 2 uses MONGODB-CR to authenticate.
• MongoDB 3.x: MongoDB 3 uses SCRAM-SHA-1 by default; new users you create in MongoDB 3 use this authentication method. However, MongoDB 3 servers will continue to use MONGODB-CR to authenticate users created in MongoDB 2.6.
• MongoDB 4.x: MongoDB 4 uses SCRAM-SHA-1 by default and does not support the deprecated MongoDB MONGODB-CR authentication mechanism.

Authenticating with LDAP

Set AuthScheme to PLAIN to use LDAP authentication. This value specifies the SASL PLAIN mechanism; note that this mechanism transmits credentials over plain-text, so it is not suitable for use without TLS/SSL on untrusted networks.

Authenticating with Kerberos

Set AuthScheme to GSSAPI to use Kerberos authentication. Additionally configure the following properties as configured for the MongoDB environment:

Authenticating with X.509 Authentication

Set AuthScheme to X509 to use X.509 certificate authentication.

The host name or IP address of the server hosting the MongoDB database.

Remarks

The host name or IP address of the server hosting the MongoDB database. If you choose to connect using DNS seed lists, set this option to "mongodb+srv://" + the name of the server your MongoDB instance is running on.

If connecting through MongoDB Atlas, set the Server connection property to the shard value of the primary cluster (ex: cluster0-shard-00-00-test.mongodb.net). More information about sharding can be found here: MongoDB Sharding.

The port for the MongoDB database.

Remarks

The port for the MongoDB database.

Specifies the authenticating user's user ID.

Remarks

The authenticating server requires both User and Password to validate the user's identity.

Specifies the authenticating user's password.

Remarks

The authenticating server requires both User and Password to validate the user's identity.

The name of the MongoDB database.

Remarks

The name of the MongoDB database.

This field sets whether SSL is enabled.

Remarks

This field sets whether the Sync App will attempt to negotiate TLS/SSL connections to the server. By default, the Sync App checks the server's certificate against the system's trusted certificate store. To specify another certificate, set SSLServerCert.

The name of the MongoDB database for authentication.

Remarks

The name of the MongoDB database for authentication. Only needed if the authentication database is different from the database to retrieve data from.

This property allows you to specify multiple servers in addition to the one configured in Server and Port . Specify both a server name and port; separate servers with a comma.

Remarks

This property allows you to specify the other servers in the replica set in addition to the one configured in Server and Port. You must specify all servers in the replica set using ReplicaSet, Server, and Port.

Specify both a server name and port in ReplicaSet; separate servers with a comma. For example:

Server=localhost;Port=27017;ReplicaSet=localhost:27018,localhost:27019;

To find the primary server, the Sync App queries the servers in ReplicaSet and the server specified by Server and Port.

Note that only the primary server in a replica set is writable. Secondaries can be readable if the SlaveOK setting allows it. To configure a strategy executing SELECT queries to secondaries, see ReadPreference.

Specify the DNS server when resolving MongoDB seed list.

Remarks

Specify the DNS server when resolving MongoDB seed list.

This section provides a complete list of the Kerberos properties you can configure in the connection string for this provider.

Identifies the Kerberos Key Distribution Center (KDC) service used to authenticate the user. (SPNEGO or Windows authentication only).

Remarks

The Kerberos properties are used when using SPNEGO or Windows Authentication. The Sync App requests session tickets and temporary session keys from the Kerberos KDC service, which is usually co-located with the domain controller.

If KerberosKDC is not specified, the Sync App tries to detect these properties automatically from the following locations:

• KRB5 Config File (krb5.ini/krb5.conf): If the KRB5_CONFIG environment variable is set and the file exists, the Sync App obtains the KDC from the specified file. If it is not found there, the Sync App tries to read from the default MIT location based on the OS: C:\ProgramData\MIT\Kerberos5\krb5.ini (Windows) or /etc/krb5.conf (Linux).
• Domain Name and Host: If the Kerberos Realm and Kerberos KDC cannot be inferred from another location, the Sync App infers them from the configured domain name and host.

Identifies the Kerberos Realm used to authenticate the user.

Remarks

A realm is a logical network, similar to a domain, that defines a group of systems under the same master KDC. Some realms are hierarchical, where one realm is a superset of the other realm, but usually realms are nonhierarchical (or “direct”) and the mapping between the two realms must be defined. Kerberos cross-realm authentication enables authentication across realms. Each realm only needs to have a principal entry for the other realm in its KDC.

The Kerberos properties are used when using SPNEGO or Windows Authentication. The Sync App requests session tickets and temporary session keys from the Kerberos KDC service, which is usually co-located with the domain controller. The Kerberos Realm can be configured by an administrator to be any string, but it is usually based on the domain name.

If Kerberos Realm is not specified, the Sync App will attempt to detect these properties automatically from the following locations:

• KRB5 Config File (krb5.ini/krb5.conf): If the KRB5_CONFIG environment variable is set and the file exists, the Sync App will obtain the default realm from the specified file. Otherwise, it will attempt to read from the default MIT location based on the OS: C:\ProgramData\MIT\Kerberos5\krb5.ini (Windows) or /etc/krb5.conf (Linux)
• Domain Name and Host: If the Kerberos Realm and Kerberos KDC could not be inferred from another location, the Sync App will infer them from the user-configured domain name and host. This might work in some Windows environments.

Identifies the service principal name (SPN) for the Kerberos Domain Controller.

Remarks

If the SPN on the Kerberos Domain Controller is not the same as the URL that you are authenticating to, use this property to set the SPN to the KDC's URL.

Confirms the principal name for the Kerberos Domain Controller, which uses the format host/user@realm.

Remarks

If there is a Kerberos principal, that Kerberos principal name should always be used to authenticate to the database.

Identifies the Keytab file containing your pairs of Kerberos principals and encrypted keys.

Remarks

A keytab (short for “key table”) stores long-term keys for one or more principals. In most cases, end users authenticate to the KDC using their client secret (password). However, in situations where authentication or re-authentication happen using automated scripts and applications, it may be more efficient to use a keytab, which sends passwords to the KDC in encrypted form, automatically.

Keytabs are normally represented by files in a standard format, and named using the format type:value. Usually type is FILE and value is the absolute pathname of the file. The other possible value for type is MEMORY, which indicates a temporary keytab stored in the memory of the current process.

A keytab contains one or more entries, where each entry consists of a timestamp (indicating when the entry was written to the keytab), a principal name, a key version number, an encryption type, and the encryption key itself. They can be generated using kutil.

For example:

[admin@myhost]# ktutil

ktutil: addent -password -p starlord/[email protected] -k 1 -e aes256-cts-hmac-sha1-96
Password for starlord/myhost.galaxy.com:

ktutil: addent -password -p starlord/[email protected] -k 1 -e aes128-cts-hmac-sha1-96
Password for starlord/myhost.galaxy.com:

ktutil: addent -password -p starlord/[email protected] -k 1 -e des3-cbc-sha1
Password for starlord/myhost.galaxy.com:

ktutil: wkt /path/to/starlord.keytab

Note: You must create principals for all authentication methods (encryption types) you want to support.

To display a keytab, use klist -k.

Identifies the service's Kerberos realm. (Cross-realm authentication only).

Remarks

The KerberosServiceRealm is used to specify a service's KerberosRealm when using cross-realm Kerberos authentication.

In most cases, a single realm and KDC machine are used to perform the Kerberos authentication, which means that this property would not be required. However, the property is available for complex setups where a different realm and KDC machine are used to obtain an authentication ticket (AS request) and a service ticket (TGS request).

Identifies the service's Kerberos Key Distribution Center (KDC).

Remarks

The KerberosServiceKDC is used to specify the service Kerberos KDC when using cross-realm Kerberos authentication.

In most cases, a single realm and KDC machine are used to perform the Kerberos authentication, which means that this property would not be required. However, the property is available for complex setups where a different realm and KDC machine are used to obtain an authentication ticket (AS request) and a service ticket (TGS request).

Specifies the full file path to an MIT Kerberos credential cache file.

Remarks

Set this property if you want to use a credential cache file that was created using the MIT Kerberos Ticket Manager or kinit command.

This section provides a complete list of the SSL properties you can configure in the connection string for this provider.

Specifies the TLS/SSL client certificate store for SSL Client Authentication (2-way SSL). This property works in conjunction with other SSL-related properties to establish a secure connection.

Remarks

This property specifies the client certificate store for SSL Client Authentication. Use this property alongside SSLClientCertType, which defines the type of the certificate store, and SSLClientCertPassword, which specifies the password for password-protected stores. When SSLClientCert is set and SSLClientCertSubject is configured, the driver searches for a certificate matching the specified subject.

Certificate store designations vary by platform. On Windows, certificate stores are identified by names such as MY (personal certificates), while in Java, the certificate store is typically a file containing certificates and optional private keys.

The following are designations of the most common User and Machine certificate stores in Windows:

For PFXFile types, set this property to the filename. For PFXBlob types, set this property to the binary contents of the file in PKCS12 format.

Specifies the type of key store containing the TLS/SSL client certificate for SSL Client Authentication. Choose from a variety of key store formats depending on your platform and certificate source.

Remarks

This property determines the format and location of the key store used to provide the client certificate. Supported values include platform-specific and universal key store formats. The available values and their usage are:

Specifes the password required to access the TLS/SSL client certificate store. Use this property if the selected certificate store type requires a password for access.

Remarks

This property provides the password needed to open a password-protected certificate store. This property is necessary when using certificate stores that require a password for decryption, as is often recommended for PFX or JKS type stores.

If the certificate store type does not require a password, for example USER or MACHINE on Windows, this property can be left blank. Ensure that the password matches the one associated with the specified certificate store to avoid authentication errors.

Specifes the subject of the TLS/SSL client certificate to locate it in the certificate store. Use a comma-separated list of distinguished name fields, such as CN=www.server.com, C=US. The wildcard * selects the first certificate in the store.

Remarks

This property determines which client certificate to load based on its subject. The Sync App searches for a certificate that exactly matches the specified subject. If no exact match is found, the Sync App looks for certificates containing the value of the subject. If no match is found, no certificate is selected.

The subject should follow the standard format of a comma-separated list of distinguished name fields and values. For example, CN=www.server.com, OU=Test, C=US. Common fields include the following:

Note: If any field contains special characters, such as commas, the value must be quoted. For example: CN="Example, Inc.", C=US.

Specifies the certificate to be accepted from the server when connecting using TLS/SSL.

Remarks

If you are using a TLS/SSL connection, use this property to specify the TLS/SSL certificate to be accepted from the server. If you specify a value for this property, all other certificates that are not trusted by the machine are rejected.

This property can take the following forms:

Note: It is possible to use '*' to signify that all certificates should be accepted, but due to security concerns this is not recommended.

This section provides a complete list of the SSH properties you can configure in the connection string for this provider.

The authentication method used when establishing an SSH Tunnel to the service.

Remarks

• None: No authentication is performed. The current SSHUser value is ignored, and the connection is logged in as anonymous.
• Password: The Sync App uses the values of SSHUser and SSHPassword to authenticate the user.
• Public_Key: The Sync App uses the values of SSHUser and SSHClientCert to authenticate the user. SSHClientCert must have a private key available for this authentication method to succeed.

A certificate to be used for authenticating the SSHUser.

Remarks

SSHClientCert must contain a valid private key in order to use public key authentication. A public key is optional, if one is not included then the Sync App generates it from the private key. The Sync App sends the public key to the server and the connection is allowed if the user has authorized the public key.

The SSHClientCertType field specifies the type of the key store specified by SSHClientCert. If the store is password protected, specify the password in SSHClientCertPassword.

Some types of key stores are containers which may include multiple keys. By default the Sync App will select the first key in the store, but you can specify a specific key using SSHClientCertSubject.

The password of the SSHClientCert key if it has one.

Remarks

This property is required for SSH tunneling when using certificate-based authentication. If the SSH certificate is in a password-protected key store, provide the password using this property to access the certificate.

The subject of the SSH client certificate.

Remarks

When loading a certificate the subject is used to locate the certificate in the store.

If an exact match is not found, the store is searched for subjects containing the value of the property.

If a match is still not found, the property is set to an empty string, and no certificate is selected.

The special value "*" picks the first certificate in the certificate store.

The certificate subject is a comma separated list of distinguished name fields and values. For instance "CN=www.server.com, OU=test, C=US, [email protected]". Common fields and their meanings are displayed below.

If a field value contains a comma it must be quoted.

The type of SSHClientCert private key.

Remarks

This property can take one of the following values:

The SSH server.

Remarks

The SSH server.

The SSH port.

Remarks

The SSH port.

The SSH user.

Remarks

The SSH user.

The SSH password.

Remarks

The SSH password.

The SSH server fingerprint.

Remarks

The SSH server fingerprint.

Whether to tunnel the MongoDB connection over SSH. Use SSH.

Remarks

By default the Sync App will attempt to connect directly to MongoDB. When this option is enabled, the Sync App will instead establish an SSH connection with the SSHServer and tunnel the connection to MongoDB through it.

This section provides a complete list of the Firewall properties you can configure in the connection string for this provider.

Specifies the protocol the provider uses to tunnel traffic through a proxy-based firewall.

Remarks

A proxy-based firewall (or proxy firewall) is a network security device that acts as an intermediary between user requests and the resources they access. The proxy accepts the request of an authenticated user, tunnels through the firewall, and transmits the request to the appropriate server.

Because the proxy evaluates and transfers data backets on behalf of the requesting users, the users never connect directly with the servers, only with the proxy.

The following table provides port number information for each of the supported protocols.

Identifies the IP address, DNS name, or host name of a proxy used to traverse a firewall and relay user queries to network resources.

Remarks

A proxy-based firewall (or proxy firewall) is a network security device that acts as an intermediary between user requests and the resources they access. The proxy accepts the request of an authenticated user, tunnels through the firewall, and transmits the request to the appropriate server.

Because the proxy evaluates and transfers data backets on behalf of the requesting users, the users never connect directly with the servers, only with the proxy.

Specifies the TCP port to be used for a proxy-based firewall.

Remarks

A proxy-based firewall (or proxy firewall) is a network security device that acts as an intermediary between user requests and the resources they access. The proxy accepts the request of an authenticated user, tunnels through the firewall, and transmits the request to the appropriate server.

Because the proxy evaluates and transfers data backets on behalf of the requesting users, the users never connect directly with the servers, only with the proxy.

Identifies the user ID of the account authenticating to a proxy-based firewall.

Remarks

A proxy-based firewall (or proxy firewall) is a network security device that acts as an intermediary between user requests and the resources they access. The proxy accepts the request of an authenticated user, tunnels through the firewall, and transmits the request to the appropriate server.

Because the proxy evaluates and transfers data backets on behalf of the requesting users, the users never connect directly with the servers, only with the proxy.

Specifies the password of the user account authenticating to a proxy-based firewall.

Remarks

A proxy-based firewall (or proxy firewall) is a network security device that acts as an intermediary between user requests and the resources they access. The proxy accepts the request of an authenticated user, tunnels through the firewall, and transmits the request to the appropriate server.

Because the proxy evaluates and transfers data backets on behalf of the requesting users, the users never connect directly with the servers, only with the proxy.

This section provides a complete list of the Logging properties you can configure in the connection string for this provider.

Specifies the core modules to include in the log file. Use a semicolon-separated list of module names. By default, all modules are logged.

Remarks

The Sync App writes details about each operation it performs into the logfile specified by the Logfile connection property.

Each of these logged operations are assigned to a themed category called a module, and each module has a corresponding short code used to labels individual Sync App operations as belonging to that module.

When this connection property is set to a semicolon-separated list of module codes, only operations belonging to the specified modules are written to the logfile. Note that this only affects which operations are logged moving forward and doesn't retroactively alter the existing contents of the logfile. For example: INFO;EXEC;SSL;META;

By default, logged operations from all modules are included.

You can explicitly exclude a module by prefixing it with a "-". For example: -HTTP

To apply filters to submodules, identify them with the syntax <module name>.<submodule name>. For example, the following value causes the Sync App to only log actions belonging to the HTTP module, and further refines it to exclude actions belonging to the Res submodule of the HTTP module: HTTP;-HTTP.Res

Note that the logfile filtering triggered by the Verbosity connection property takes precedence over the filtering imposed by this connection property. This means that operations of a higher verbosity level than the level specified in the Verbosity connection property are not printed in the logfile, even if they belong to one of the modules specified in this connection property.

The available modules and submodules are:

This section provides a complete list of the Schema properties you can configure in the connection string for this provider.

Specifies the location of a directory containing schema files that define tables, views, and stored procedures. Depending on your service's requirements, this may be expressed as either an absolute path or a relative path.

Remarks

The Location property is only needed if you want to either customize definitions (for example, change a column name, ignore a column, etc.) or extend the data model with new tables, views, or stored procedures.

If left unspecified, the default location is %APPDATA%\\CData\\MongoDB Data Provider\\Schema, where %APPDATA% is set to the user's configuration directory:

Optional setting that restricts the schemas reported to a subset of all available schemas. For example, BrowsableSchemas=SchemaA,SchemaB,SchemaC .

Remarks

Listing all available database schemas can take extra time, thus degrading performance. Providing a list of schemas in the connection string saves time and improves performance.

Optional setting that restricts the tables reported to a subset of all available tables. For example, Tables=TableA,TableB,TableC .

Remarks

Listing all available tables from some databases can take extra time, thus degrading performance. Providing a list of tables in the connection string saves time and improves performance.

If there are lots of tables available and you already know which ones you want to work with, you can use this property to restrict your viewing to only those tables. To do this, specify the tables you want in a comma-separated list. Each table should be a valid SQL identifier with any special characters escaped using square brackets, double-quotes or backticks. For example, Tables=TableA,[TableB/WithSlash],WithCatalog.WithSchema.`TableC With Space`.

Note: If you are connecting to a data source with multiple schemas or catalogs, you must specify each table you want to view by its fully qualified name. This avoids ambiguity between tables that may exist in multiple catalogs or schemas.

Optional setting that restricts the views reported to a subset of the available tables. For example, Views=ViewA,ViewB,ViewC .

Remarks

Listing all available views from some databases can take extra time, thus degrading performance. Providing a list of views in the connection string saves time and improves performance.

If there are lots of views available and you already know which ones you want to work with, you can use this property to restrict your viewing to only those views. To do this, specify the views you want in a comma-separated list. Each view should be a valid SQL identifier with any special characters escaped using square brackets, double-quotes or backticks. For example, Views=ViewA,[ViewB/WithSlash],WithCatalog.WithSchema.`ViewC With Space`.

Note: If you are connecting to a data source with multiple schemas or catalogs, you must specify each view you want to examine by its fully qualified name. This avoids ambiguity between views that may exist in multiple catalogs or schemas.

This section provides a complete list of the Miscellaneous properties you can configure in the connection string for this provider.

A comprehensive list detailing the mappings of column names for the built-in fields used in MongoDB.

Remarks

This property allows users to input a list of MongoDB column names, separated by commas, and maps these built-in columns to newly defined names. If this property is defined, it directs the Sync App to utilize a predefined set of mappings between MongoDB's document fields and the SQL columns.

The remappable built-in columns are "_index", "P_id", "_id" and "parent_id".

For example:

_index=BuiltInIndex,P_id=Root_Id,_id=My_Id,parent_id=My_Parent_id

Remapping these columns is important, particularly in addressing common issues such as "column names must be unique" errors. These conflicts often occur when the Sync App encounters extra columns labeled "_index", "P_id", "_id" or "parent_id" in addition to the standard built-in columns.

This property is useful for modifying reserved names, offering flexibility in database design, and avoiding conflicts.

Specifies the compression method used for network communication between the client and the MongoDB server.

Remarks

This property enables compression and decompression of messages between the application and MongoDB, thereby reducing the total amount of data transmitted over the network.

• When Compression is set to None, it is not enabled. This means that the data can remain in its original, uncompressed form.
  
• When Compression is set to Snappy, it indicates that network traffic between the MongoDB client and server will be compressed using the Snappy algorithm. Both the client and the server must support Snappy compression, which is enabled by default in most modern versions of the service.
  
• When Compression is set to Zlib, the Sync App activates network compression using the zlib algorithm for communication between the client and server.

This property helps improve performance when working with large MongoDB documents or tables.

By default, the provider will not automatically discover the metadata for a child table as its own distinct table. To enable this functionality, set DataModel to Relational .

Remarks

When setting DataModel to Relational, the discovery of child tables extends to root-level elements and those found within top-level array elements. Additionally, the provider exposes _id and parent_id columns to enable JOIN operations between parent and child tables. The _id column acts as a primary key for the flattened table, while the parent_id column identifies the parent document.

Determines the format of datetime values returned by the Document function. This property only takes effect when StrictMode=true.

Remarks

• Canonical: According to MongoDB standards, for dates before the year 1970 or after the year 9999, the format will always be millis.
• ISO8601: The format will always be ISO8601.
• UnixTS: The format will always be millis.

This property specifies whether nested array elements are flattened into individual columns. By default, nested arrays are returned as JSON strings. Set this property to the number of elements to extract from nested arrays.

Remarks

By default, nested arrays are returned as strings of JSON. The FlattenArrays property can be used to flatten the elements of nested arrays into columns of their own. This is only recommended for arrays that are expected to be short.

Set FlattenArrays to the number of elements you want to return from nested arrays. The specified elements are returned as columns. The zero-based index is concatenated to the column name. Other elements are ignored.

For example, you can return an arbitrary number of elements from an array of strings:

["FLOW-MATIC","LISP","COBOL"]

Setting FlattenArrays to -1 will flatten all the elements of nested arrays.

This property specifies whether the attributes of objects are flattened into separate columns.

Remarks

Set FlattenObjects to true to flatten the properties of objects into individual columns. If set to false, nested properties can remain nested and can be returned as JSON strings.
The Sync App generates the column name by concatenating the property name with the object name, separated by a dot.

For example, you can flatten the nested objects below at connection time:

[
     { "grade": "A", "score": 2 },
     { "grade": "A", "score": 6 },
     { "grade": "A", "score": 10 },
     { "grade": "A", "score": 9 },
     { "grade": "B", "score": 14 }
]

Indicates the user preference as to when schemas should be generated and saved.

Remarks

GenerateSchemaFiles enables you to save the table definitions identified by Automatic Schema Discovery. This property outputs schemas to .rsd files in the path specified by Location.

Available settings are the following:

• Never: A schema file will never be generated.
• OnUse: A schema file will be generated the first time a table is referenced, provided the schema file for the table does not already exist.
• OnStart: A schema file will be generated at connection time for any tables that do not currently have a schema file.
• OnCreate: A schema file will be generated by when running a CREATE TABLE SQL query.

Generate Schemas with SQL

When you set GenerateSchemaFiles to OnUse, the Sync App generates schemas as you execute SELECT queries. Schemas are generated for each table referenced in the query.

When you set GenerateSchemaFiles to OnCreate, schemas are only generated when a CREATE TABLE query is executed.

Generate Schemas on Connection

Another way to use this property is to obtain schemas for every table in your database when you connect. To do so, set GenerateSchemaFiles to OnStart and connect.

Alternatives to Static Schemas

If your data structures are volatile, consider setting GenerateSchemaFiles to Never and using dynamic schemas. See Automatic Schema Discovery for more information about dynamic schemas.

Editing Schemas

Schema files have a simple format that makes them easy to modify. See Custom Schema Definitions for more information.

Specifies the maximum number of rows returned for queries that do not include either aggregation or GROUP BY.

Remarks

The default value for this property, -1, means that no row limit is enforced unless the query explicitly includes a LIMIT clause. (When a query includes a LIMIT clause, the value specified in the query takes precedence over the MaxRows setting.)

Setting MaxRows to a whole number greater than 0 ensures that queries do not return excessively large result sets by default.

This property is useful for optimizing performance and preventing excessive resource consumption when executing queries that could otherwise return very large datasets.

The server typically terminates idle cursors after 30 minutes of inactivity to prevent excessive memory usage. Set this option to true to avoid automatic timeouts and keep your cursors active.

Remarks

By default, the MongoDB server automatically closes idle cursors associated with the session after 30 minutes of inactivity to free up resources. The session refreshes with each new document batch request. If processing takes longer than 30 minutes, the session can expire and close. When NoCursorTimeout is set to true, the cursor can not time out due to inactivity. It remains open until it is explicitly closed by the application or the cursor has exhausted all results.

This property is useful in controlling whether a cursor automatically times out after a period of inactivity.

Specifies advanced connection properties for specialized scenarios. Use this property only under the guidance of our Support team to address specific issues.

Remarks

This property allows advanced users to configure hidden properties for specialized situations, with the advice of our Support team. These settings are not required for normal use cases but can address unique requirements or provide additional functionality. To define multiple properties, use a semicolon-separated list.

Note: It is strongly recommended to set these properties only when advised by the Support team to address specific scenarios or issues.

Integration and Formatting

Specifies the maximum number of records per page the provider returns when requesting data from MongoDB.

Remarks

When processing a query, instead of requesting all of the queried data at once from MongoDB, the Sync App can request the queried data in pieces called pages.

This connection property determines the maximum number of results that the Sync App requests per page.

Note: Setting large page sizes may improve overall query execution time, but doing so causes the Sync App to use more memory when executing queries and risks triggering a timeout.

Specifies the pseudocolumns to expose as table columns, expressed as a string in the format 'TableName=ColumnName;TableName=ColumnName'.

Remarks

This property allows you to define which pseudocolumns the Sync App exposes as table columns.

To specify individual pseudocolumns, use the following format:

Table1=Column1;Table1=Column2;Table2=Column3

To include all pseudocolumns for all tables use:

*=*

Determines whether queries are sent directly to MongoDB without modification.

Remarks

When QueryPassthrough is set to true, the specified query can be passed to MongoDB as-is. Currently, only these shell commands are supported:

• db.myCollection.find() returns all fields for all records in the collection.
• db.myCollection.find({ query }) returns all fields for all records in the collection matching the query.
• db.myCollection.find({ query }, { projection }) returns the fields in the projection, for all records matching the query.
• All of the above forms accept a .json() suffix. This returns a single column containing the matching documents as JSON instead of individual fields.

When QueryPassthrough is set to false, the query is not sent directly to MongoDB. Instead, Sync App parses the query and converts it to MongoDB's native query language (MQL).

Set this to a strategy for reading from a replica set. Accepted values are primary, primaryPreferred, secondary, secondaryPreferred, and nearest.

Remarks

This property enables you to execute queries to a member in a replica set other other than the primary member. Accepted values are the following:

• primary: All SELECT queries are executed against the primary server.
• primaryPreferred: If the primary server is not available, SELECT queries are executed to a secondary server.
• secondary: All SELECT queries are executed to the secondary servers.
• secondaryPreferred: SELECT queries are executed to a secondary server if one is available. Otherwise, the queries are executed to the primary server.
• nearest: SELECT queries are executed to the server with the least latency.

When to Use ReadPreference

When this property is set, query results may not reflect the latest changes if a write operation has not yet been replicated to a secondary machine. You can use ReadPreference to accomplish the following, with some risk that the Sync App will return stale data:

• Configure failover queries: If the primary server is unavailable, you can set this property to "primaryPreferred" to continue to execute queries online.
• Execute faster queries to geographically distributed replica sets: If your deployment uses multiple data centers, setting ReadPreference to "nearest" can result in faster queries, as the Sync App executes SELECT queries to whichever replica set member has the lowest latency.

When directing the Sync App to execute SELECT statements to a secondary server, SlaveOK must also be set. Otherwise, the Sync App will return an error response.

This property is used to identify and interact with one or more members of a replica set that are linked to specific tags.

Remarks

To use the ReadPreferenceTags property, it is necessary to configure the ReadPreference to a value other than the default 'primary' value. The required format consists of a list of semicolon-separated tag sets, where each tag set includes key-value pairs separated by commas.

For example:

• tag1:val1,tag2:val2;: Find members with both tag values. If none are found, find any eligible member.
• tag1:val1;tag2:val2;: Find members with the specified tag1, otherwise find members with the specified tag2. If none are found find any eligible member.
• tag1:val1: Find only members with the specified tag.
• ;: (semicolon only) Find any eligible member. If left empty, any eligible member is targeted.

The maximum number of rows to scan to look for the columns available in a table.

Remarks

The columns in a table must be determined by scanning table rows. This value determines the maximum number of rows that will be scanned.

Setting a high value may decrease performance. Setting a low value may prevent the data type from being determined properly, especially when there is null data.

Setting this property to a value of -1 causes the Sync App to perform a full scan.

Specifies the type of service the provider can interact with.

Remarks

The ServiceKind property informs the Sync App of the type of MongoDB service to which it is connecting. This can affect how the connection operates or which features are accessible. Typical values include MongoDB for standard MongoDB deployments. This is the default option.

This property is useful for tools that support various MongoDB services.

Determines the provider's capability to read data from secondary (slave) servers. It controls whether the provider can access and retrieve information from these backup systems.

Remarks

The SlaveOK property allows read operations on secondary servers in a replica set. This connection property is deprecated. The recommended option is ReadPreference for version 4.2 or above.

When set to true, it enables reading from secondary replica set servers, in addition to the primary server. This property is useful for configuring how the driver queries secondary servers using the ReadPreference setting.

Specifies the maximum time, in seconds, that the provider waits for a server response before throwing a timeout error.

Remarks

The timeout applies to each individual communication with the server rather than the entire query or operation. For example, a query could continue running beyond 60 seconds if each paging call completes within the timeout limit.

Timeout is set to 60 seconds by default. To disable timeouts, set this property to 0.

Disabling the timeout allows operations to run indefinitely until they succeed or fail due to other conditions such as server-side timeouts, network interruptions, or resource limits on the server.

Note: Use this property cautiously to avoid long-running operations that could degrade performance or result in unresponsive behavior.

Specifies the method for detecting metadata discovery.

Remarks

A detailed list of enumerated options outlines the methods the provider uses to examine the data. This property helps identify different fields and their data types in each document collection.

Specifies the strategy that can be used when executing an update statement.

Remarks

When updating a target document with fields that are replaced or merged, an update statement is executed. If the default value is set to Default, the Sync App replaces the entire original document with a new one. However, if the value is set to Merge, only specific fields in the target document can be updated.

This property helps trigger the system to identify which document needs modifications.

For example, if you have a collection 'classySample' as below.

{
  "_id": "1",
  "message": {
    "component_items": [{"locked": true}],
    "id":1
  }
}

UPDATE [classySample] SET [message.component_items.0.locked] = false  WHERE [message.id] = 1

In the query above, the 'message' document will be replaced with new document constructed with SET clause, the collection after updating looks like

{
  "_id": "1",
  "message": {
    "component_items": [
      {
        "locked": false
      }
    ]
  }
}

But when using Merge, only the 'locked' field in 'component_items' will be updated, the collection becomes

{
    "_id": "1",
    "message": {
        "component_items": [
            {
                "locked": false
            }
        ],
        "id": 1
    }
}

Specifies whether MongoDB queries using the method db.collection.find(), allow retrieval of documents from a specific collection based on defined criteria.

Remarks

When UseFindAPI is set to true, the Sync App uses the new Find Command API instead of the older OP_QUERY interface. Therefore, this must be set to true in order to query DocumentDB clusters using db.collection.find(). If set to false, the Sync App can revert to the legacy find operation, such as OP_QUERY, particularly when working with older versions of MongoDB servers.

This property is useful for filtering, sorting, and manipulating data in MongoDB's flexible document structure.

Specifies a filepath to a JSON configuration file that defines custom views. The provider automatically detects and uses the views specified in this file.

Remarks

UserDefinedViews allows you to define and manage custom views through a JSON-formatted configuration file called UserDefinedViews.json. These views are automatically recognized by the Sync App and enable you to execute custom SQL queries as if they were standard database views. The JSON file defines each view as a root element with a child element called "query", which contains the SQL query for the view.

For example:

{
	"MyView": {
		"query": "SELECT * FROM [CData].[Sample].Customers WHERE MyColumn = 'value'"
	},
	"MyView2": {
		"query": "SELECT * FROM MyTable WHERE Id IN (1,2,3)"
	}
}

You can use this property to define multiple views in a single file and specify the filepath. For example:

UserDefinedViews=C:\Path\To\UserDefinedViews.json

For further information, see User Defined Views.

Determines the level of acknowledgment requested for write operations in MongoDB, applicable to standalone mongod, replica sets, or sharded clusters.

Remarks

The WriteConcern property in MongoDB defines the acknowledgment level required for write operations, determining how confident MongoDB must be about the success of a write before confirming it. The default value is { w: 1 }, meaning the primary node must acknowledge the write operation before returning success to the client.

This property is useful for balancing data safety and performance.

Determines whether write operations can be recorded in the on-disk journal before being acknowledged as successful.

Remarks

The WriteConcernJournaled property in MongoDB controls whether write operations must be written to the on-disk journal before being acknowledged as successful.

When set to True, MongoDB acknowledges a write operation only after the data has been committed to the on-disk journal. If the option is set to false, a write operation is acknowledged without waiting for journaling.

The WriteConcernTimeout property specifies the maximum time (in milliseconds) that the server should wait for a write concern to be acknowledged before returning an error.

Remarks

This property specifies the level of acknowledgment requested from MongoDB for write operations, such as INSERT, UPDATE, and DELETE. If a timeout is set for a write operation in MongoDB, it can wait to confirm the write on secondary nodes. If it times out, a write concern error occurs, but the write can still succeed on the primary node.

Sets whether the object type for inserted or updated objects is determined from the existing column metadata or the input value type.

Remarks

Sets whether the object type for inserted or updated objects is determined from the existing column metadata or the input value type. When the default value Metadata is used, the Sync App uses the data type as determined by the TypeDetectionScheme for objects pushed to MongoDB. When the value is set to RawValue, the type of the object in the INSERT determines what type is used for MongoDB.

For example, if you have a field 'c1' in MongoDB defined as String type, the metadata returns the column as String as well. In the following query, the resulting field in MongoDB is therefore defined as String when using WriteScheme=Metadata. But when using RawValue, the inserting field type is Date instead since the FROM_UNIXTIME() function returns an actual Date object:

INSERT INTO Table1 (c1) VALUES (FROM_UNIXTIME(1636910867039, 0))

Inserting an empty array

INSERT INTO t1 ("c1") VALUES (())

This returns an empty array:

"c1":[]

LZMA from 7Zip LZMA SDK

LZMA SDK is placed in the public domain.

Anyone is free to copy, modify, publish, use, compile, sell, or distribute the original LZMA SDK code, either in source code form or as a compiled binary, for any purpose, commercial or non-commercial, and by any means.

LZMA2 from XZ SDK

Version 1.9 and older are in the public domain.

Xamarin.Forms

Xamarin SDK

The MIT License (MIT)

Copyright (c) .NET Foundation Contributors

All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

NSIS 3.10

Copyright (C) 1999-2025 Contributors THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS COMMON PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.

1. DEFINITIONS

"Contribution" means:

a) in the case of the initial Contributor, the initial code and documentation distributed under this Agreement, and b) in the case of each subsequent Contributor:

i) changes to the Program, and

ii) additions to the Program;

where such changes and/or additions to the Program originate from and are distributed by that particular Contributor. A Contribution 'originates' from a Contributor if it was added to the Program by such Contributor itself or anyone acting on such Contributor's behalf. Contributions do not include additions to the Program which: (i) are separate modules of software distributed in conjunction with the Program under their own license agreement, and (ii) are not derivative works of the Program.

"Contributor" means any person or entity that distributes the Program.

"Licensed Patents " mean patent claims licensable by a Contributor which are necessarily infringed by the use or sale of its Contribution alone or when combined with the Program.

"Program" means the Contributions distributed in accordance with this Agreement.

"Recipient" means anyone who receives the Program under this Agreement, including all Contributors.

2. GRANT OF RIGHTS

a) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, distribute and sublicense the Contribution of such Contributor, if any, and such derivative works, in source code and object code form.

b) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free patent license under Licensed Patents to make, use, sell, offer to sell, import and otherwise transfer the Contribution of such Contributor, if any, in source code and object code form. This patent license shall apply to the combination of the Contribution and the Program if, at the time the Contribution is added by the Contributor, such addition of the Contribution causes such combination to be covered by the Licensed Patents. The patent license shall not apply to any other combinations which include the Contribution. No hardware per se is licensed hereunder.

c) Recipient understands that although each Contributor grants the licenses to its Contributions set forth herein, no assurances are provided by any Contributor that the Program does not infringe the patent or other intellectual property rights of any other entity. Each Contributor disclaims any liability to Recipient for claims brought by any other entity based on infringement of intellectual property rights or otherwise. As a condition to exercising the rights and licenses granted hereunder, each Recipient hereby assumes sole responsibility to secure any other intellectual property rights needed, if any. For example, if a third party patent license is required to allow Recipient to distribute the Program, it is Recipient's responsibility to acquire that license before distributing the Program.

d) Each Contributor represents that to its knowledge it has sufficient copyright rights in its Contribution, if any, to grant the copyright license set forth in this Agreement.

3. REQUIREMENTS

A Contributor may choose to distribute the Program in object code form under its own license agreement, provided that:

a) it complies with the terms and conditions of this Agreement; and

b) its license agreement:

i) effectively disclaims on behalf of all Contributors all warranties and conditions, express and implied, including warranties or conditions of title and non-infringement, and implied warranties or conditions of merchantability and fitness for a particular purpose;

ii) effectively excludes on behalf of all Contributors all liability for damages, including direct, indirect, special, incidental and consequential damages, such as lost profits;

iii) states that any provisions which differ from this Agreement are offered by that Contributor alone and not by any other party; and

iv) states that source code for the Program is available from such Contributor, and informs licensees how to obtain it in a reasonable manner on or through a medium customarily used for software exchange.

When the Program is made available in source code form:

a) it must be made available under this Agreement; and

b) a copy of this Agreement must be included with each copy of the Program.

Contributors may not remove or alter any copyright notices contained within the Program.

Each Contributor must identify itself as the originator of its Contribution, if any, in a manner that reasonably allows subsequent Recipients to identify the originator of the Contribution.

4. COMMERCIAL DISTRIBUTION

Commercial distributors of software may accept certain responsibilities with respect to end users, business partners and the like. While this license is intended to facilitate the commercial use of the Program, the Contributor who includes the Program in a commercial product offering should do so in a manner which does not create potential liability for other Contributors. Therefore, if a Contributor includes the Program in a commercial product offering, such Contributor ("Commercial Contributor") hereby agrees to defend and indemnify every other Contributor ("Indemnified Contributor") against any losses, damages and costs (collectively "Losses") arising from claims, lawsuits and other legal actions brought by a third party against the Indemnified Contributor to the extent caused by the acts or omissions of such Commercial Contributor in connection with its distribution of the Program in a commercial product offering. The obligations in this section do not apply to any claims or Losses relating to any actual or alleged intellectual property infringement. In order to qualify, an Indemnified Contributor must: a) promptly notify the Commercial Contributor in writing of such claim, and b) allow the Commercial Contributor to control, and cooperate with the Commercial Contributor in, the defense and any related settlement negotiations. The Indemnified Contributor may participate in any such claim at its own expense.

For example, a Contributor might include the Program in a commercial product offering, Product X. That Contributor is then a Commercial Contributor. If that Commercial Contributor then makes performance claims, or offers warranties related to Product X, those performance claims and warranties are such Commercial Contributor's responsibility alone. Under this section, the Commercial Contributor would have to defend claims against the other Contributors related to those performance claims and warranties, and if a court requires any other Contributor to pay any damages as a result, the Commercial Contributor must pay those damages.

5. NO WARRANTY

EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Each Recipient is solely responsible for determining the appropriateness of using and distributing the Program and assumes all risks associated with its exercise of rights under this Agreement, including but not limited to the risks and costs of program errors, compliance with applicable laws, damage to or loss of data, programs or equipment, and unavailability or interruption of operations.

6. DISCLAIMER OF LIABILITY

EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

7. GENERAL

If any provision of this Agreement is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this Agreement, and without further action by the parties hereto, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable.

If Recipient institutes patent litigation against a Contributor with respect to a patent applicable to software (including a cross-claim or counterclaim in a lawsuit), then any patent licenses granted by that Contributor to such Recipient under this Agreement shall terminate as of the date such litigation is filed. In addition, if Recipient institutes patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Program itself (excluding combinations of the Program with other software or hardware) infringes such Recipient's patent(s), then such Recipient's rights granted under Section 2(b) shall terminate as of the date such litigation is filed.

All Recipient's rights under this Agreement shall terminate if it fails to comply with any of the material terms or conditions of this Agreement and does not cure such failure in a reasonable period of time after becoming aware of such noncompliance. If all Recipient's rights under this Agreement terminate, Recipient agrees to cease use and distribution of the Program as soon as reasonably practicable. However, Recipient's obligations under this Agreement and any licenses granted by Recipient relating to the Program shall continue and survive.

Everyone is permitted to copy and distribute copies of this Agreement, but in order to avoid inconsistency the Agreement is copyrighted and may only be modified in the following manner. The Agreement Steward reserves the right to publish new versions (including revisions) of this Agreement from time to time. No one other than the Agreement Steward has the right to modify this Agreement. IBM is the initial Agreement Steward. IBM may assign the responsibility to serve as the Agreement Steward to a suitable separate entity. Each new version of the Agreement will be given a distinguishing version number. The Program (including Contributions) may always be distributed subject to the version of the Agreement under which it was received. In addition, after a new version of the Agreement is published, Contributor may elect to distribute the Program (including its Contributions) under the new version. Except as expressly stated in Sections 2(a) and 2(b) above, Recipient receives no rights or licenses to the intellectual property of any Contributor under this Agreement, whether expressly, by implication, estoppel or otherwise. All rights in the Program not expressly granted under this Agreement are reserved.

This Agreement is governed by the laws of the State of New York and the intellectual property laws of the United States of America. No party to this Agreement will bring a legal action under this Agreement more than one year after the cause of action arose. Each party waives its rights to a jury trial in any resulting litigation.