This page provides a guide to Establishing a Connection to Cloudant in CData Cloud, as well as information on the available resources, and a reference to the available connection properties.

Connecting to Cloudant

Establishing a Connection shows how to authenticate to Cloudant and configure any necessary connection properties to create a database in CData Cloud

Accessing Data from CData Cloud Services

Accessing data from Cloudant through the available standard services and CData Cloud administration is documented in further details in the CData Cloud Documentation.

Connect to Cloudant by selecting the corresponding icon in the Database tab. Required properties are listed under Settings. The Advanced tab lists connection properties that are not typically required.

Authenticating to Cloudant

• Auto (default): the Cloud decides how to authenticate you, based on the other connection properties you have set.
• Basic: Basic username/password authentication. Set AuthScheme to Basic.
• OAuth: Performs authentication based on the OAuth standard. Set AuthScheme to OAuth and supply the values for the Cloudant instance APIKey and URL.

IBM Cloudant Legacy

To create an IBM Cloudant service credential:

1. Log in to the IBM Cloud dashboard.
2. Navigate to the Menu icon > Resource List, and open your IBM Cloudant service instance.
3. In the menu, click Service credentials.
4. Click New credential. Cloudant displays a Add new credential window.
5. Enter a name for the new credential.
6. Click Add. Your credentials are added to the Service credentials table.
7. Click Actions > View credentials.
8. Extract the values for User and Password from the JSON file.

Use the values you just obtained to set the User and Password.

Authenticating to a Local Instance

To authenticate to your local instance, set these parameters:

1. Url: The Url of your local instance. For example: http://localhost:8006
2. User: Your username.
3. Password: Your password.

Fine-Tuning Data Access

You can use the following properties to gain control over column behavior in Automatic Schema Discovery.

• RowScanDepth: This property determines the number of rows that will be scanned to detect column data types when generating table metadata.
• TypeDetectionScheme: This property allows more control over the strategy implemented by the RowScanDepth property.

Cloudant is a schemaless document database that provides high performance, availability, and scalability. These features are not necessarily incompatible with a standards-compliant query language like SQL-92. The following sections show various schemes that the Cloud offers to bridge the gap with relational SQL and a document database.

The Cloud models the schemaless Cloudant objects into relational tables and translates SQL queries into Cloudant queries to get the requested data. See Query Mapping for more details on how various Cloudant operations are represented as SQL.

The Automatic Schema Discovery scheme automatically finds the data types in a Cloudant object by scanning a configured number of rows of the object. You can use RowScanDepth, FlattenArrays, and FlattenObjects to control the relational representation of the collections in Cloudant.

The schema of the table is necessary to report metadata, but you can write Free-Form Queries not tied to the schema to select, insert, update, or delete data from columns that do not exist in the schema.

The Cloud supports listing Cloudant views and retrieving data from them. This is controlled by the ListViews property which is set to false by default. Set this property to true for Cloudant views to be displayed when listing tables.

The Cloud automatically infers a relational schema by inspecting a series of Cloudant documents in a collection. You can use the RowScanDepth property to define the number of documents the Cloud will scan to do so. The columns identified during the discovery process depend on the FlattenArrays and FlattenObjects properties.

Flattening Objects

If FlattenObjects is set, all nested objects will be flattened into a series of columns. For example, consider the following document:

{
  id: 12,
  name: "Lohia Manufacturers Inc.",
  address: {street: "Main Street", city: "Chapel Hill", state: "NC"},
  offices: ["Chapel Hill", "London", "New York"],
  annual_revenue: 35,600,000
}

If FlattenObjects is not set, then the address.street, address.city, and address.state columns will not be broken apart. The address column of type string will instead represent the entire object. Its value would be {street: "Main Street", city: "Chapel Hill", state: "NC"}. See JSON Functions for more details on working with JSON aggregates.

You can change the separator character in the column name from a dot by setting SeparatorCharacter.

Flattening Arrays

The FlattenArrays property can be used to flatten array values into columns of their own. This is only recommended for arrays that are expected to be short, for example the coordinates below:

"coord": [ -73.856077, 40.848447 ]

It is best to leave other unbounded arrays as they are and piece out the data for them as needed using JSON Functions.

As discussed in Automatic Schema Discovery, intuited table schemas enable SQL access to unstructured Cloudant data. JSON Functions enable you to use standard JSON functions to summarize Cloudant data and extract values from any nested structures. However, you are not limited to these schemes.

After connecting you can query any nested structure without flattening the data. Any relations that you can access with FlattenArrays and FlattenObjects can also be accessed with an ad hoc SQL query.

Let's consider an example document from the following Restaurant data set:

 
{
  "address": {
    "building": "1007",
    "coord": [
      -73.856077,
      40.848447
    ],
    "street": "Morris Park Ave",
    "zipcode": "10462"
  },
  "borough": "Bronx",
  "cuisine": "Bakery",
  "grades": [
    {
      "grade": "A",
      "score": 2,
      "date": {
        "$date": "1393804800000"
      }
    },
    {
      "date": {
        "$date": "1378857600000"
      },
      "grade": "B",
      "score": 6
    },
    {
      "score": 10,
      "date": {
        "$date": "1358985600000"
      },
      "grade": "C"
    }
  ],
  "name": "Morris Park Bake Shop",
  "restaurant_id": "30075445"
} 

SELECT [address.building], [grades.1.grade] FROM restaurants WHERE restaurant_id = '30075445'

It is possible to retrieve an array of documents as if it were a separate table. Take the following JSON structure from the restaurants collection for example:

{
  "_id" : ObjectId("568c37b748ddf53c5ed98932"),
  "address" : {
    "building" : "1007",
    "coord" : [-73.856077, 40.848447],
    "street" : "Morris Park Ave",
    "zipcode" : "10462"
  },
  "borough" : "Bronx",
  "cuisine" : "Bakery",
  "grades" : [{
      "date" : ISODate("2014-03-03T00:00:00Z"),
      "grade" : "A",
      "score" : 2
    }, {
      "date" : ISODate("2013-09-11T00:00:00Z"),
      "grade" : "A",
      "score" : 6
    }, {
      "date" : ISODate("2013-01-24T00:00:00Z"),
      "grade" : "A",
      "score" : 10
    }, {
      "date" : ISODate("2011-11-23T00:00:00Z"),
      "grade" : "A",
      "score" : 9
    }, {
      "date" : ISODate("2011-03-10T00:00:00Z"),
      "grade" : "B",
      "score" : 14
    }],
  "name" : "Morris Park Bake Shop",
  "restaurant_id" : "30075445"
}

SELECT * FROM [restaurants.grades]

You may also want to include information from the base restaurants table. You can do this with a join. Flattened arrays can only be joined with the root document. The Cloud expects the left part of the join is the array document you want to flatten vertically. Disable SupportEnhancedSQL to join nested Cloudant documents -- this type of query is supported through the Cloudant API.

SELECT [restaurants].[restaurant_id], [restaurants.grades].* FROM [restaurants.grades] JOIN [restaurants] WHERE [restaurants].name = 'Morris Park Bake Shop'

The Cloud can return JSON structures as column values. The Cloud enables you to use standard SQL functions to work with these JSON structures. The examples in this section use the following array:

[
     { "grade": "A", "score": 2 },
     { "grade": "A", "score": 6 },
     { "grade": "A", "score": 10 },
     { "grade": "A", "score": 9 },
     { "grade": "B", "score": 14 }
]

JSON_EXTRACT

SELECT Name, JSON_EXTRACT(grades,'[0].grade') AS Grade, JSON_EXTRACT(grades,'[0].score') AS Score FROM Students;

JSON_COUNT

SELECT Name, JSON_COUNT(grades,'[x]') AS NumberOfGrades FROM Students;

JSON_SUM

SELECT Name, JSON_SUM(score,'[x].score') AS TotalScore FROM Students;

JSON_MIN

SELECT Name, JSON_MIN(score,'[x].score') AS LowestScore FROM Students;

JSON_MAX

SELECT Name, JSON_MAX(score,'[x].score') AS HighestScore FROM Students;

DOCUMENT

The DOCUMENT function can be used to retrieve the entire document as a JSON string. See the following query and its result as an example:

SELECT DOCUMENT(*) FROM Customers;

{ "id": 12, "name": "Lohia Manufacturers Inc.", "address": { "street": "Main Street", "city": "Chapel Hill", "state": "NC"}, "offices": [ "Chapel Hill", "London", "New York" ], "annual_revenue": 35,600,000 }

The Cloud maps SQL queries into the corresponding Cloudant queries. The Cloud uses the Selector syntax to compute the desired results.

A detailed description of all the transformations is out of scope, but we will describe some of the common elements that are used.

SELECT Queries

The SELECT statement is mapped to the GET and POST methods. Below are example queries and the corresponding request payloads.

SELECT *
FROM sales

SELECT _id, value
FROM sales

{
  "fields": [
    "_id",
    "value"
  ],
  "skip": 0,
  "selector": {
    "_id": {
      "$exists": true
    }
  }
}

SELECT * 
FROM sales 
WHERE value=175033291697

{
  "selector": {
    "value": {
      "$eq": 175033291697
    }
  },
  "skip": 0
}

SELECT * 
FROM sales 
WHERE value = 175033291697 OR month='June'

{
  "selector": {
    "$or": [
      {
        "value": {
          "$eq": 175033291697
        }
      },
      {
        "month": {
          "$eq": "June"
        }
      }
    ]
  },
  "skip": 0
}

SELECT * 
FROM sales
WHERE name LIKE 'A%'

{
  "selector": {
    "name": {
      "$regex": "A%"
    }
  },
  "skip": 0
}

SELECT * FROM sales
WHERE month='June'
ORDER BY _id ASC

{
  "skip": 0,
  "sort": [
    {
      "_id": "asc"
    }
  ],
  "selector": {
    "month": {
      "$eq": "June"
    }
  }
}

SELECT *
FROM sales
WHERE month='June'
ORDER BY _id DESC

{
  "skip": 0,
  "sort": [
    {
      "_id": "desc"
    }
  ],
  "selector": {
    "month": {
      "$eq": "June"
    }
  }
}

INSERT Statements

The INSERT statement is mapped to the HTTP POST request, as shown in the following query and request payload.

SQL Query

INSERT INTO users (_id, age, status, [address.city], [address.postalcode]) 
VALUES ('bcd001', 45, 'A', 'Chapel Hill', 27517)

Cloudant Query

{
  "address": {
    "city": "Chapel Hill",
    "postalcode": 27517
  },
  "_id": "bcd001",
  "age": 45,
  "status": "A"
}

UPDATE Statements

The UPDATE statement is mapped to the HTTP PUT method, as shown in the following query and request payload.

SQL Query

UPDATE users 
SET status = 'C', [address.postalcode] = 90210
WHERE _id = 'bcd001'

Cloudant Query

{
  "_id": "bcd001",
  "_rev": "1-446f6c67e3a483feae8eaf112f18892c",
  "status": "C",
  "age": 45,
  "address": {
    "city": "Chapel Hill",
    "postalcode": 90210
  }
}

DELETE Statements

The DELETE statement is mapped to the DELETE method as shown below.

SQL Query

DELETE FROM users WHERE _id = 'bcd001'

Cloudant Query

For this query a DELETE request is sent to the users/{_id}?rev={_rev} endpoint.

You can query the system tables described in this section to access schema information, information on data source functionality, and batch operation statistics.

Schema Tables

The following tables return database metadata for Cloudant:

• sys_catalogs: Lists the available databases.
• sys_schemas: Lists the available schemas.
• sys_tables: Lists the available tables and views.
• sys_tablecolumns: Describes the columns of the available tables and views.
• sys_procedures: Describes the available stored procedures.
• sys_procedureparameters: Describes stored procedure parameters.
• sys_keycolumns: Describes the primary and foreign keys.
• sys_indexes: Describes the available indexes.

Data Source Tables

The following tables return information about how to connect to and query the data source:

• sys_connection_props: Returns information on the available connection properties.
• sys_sqlinfo: Describes the SELECT queries that the Cloud can offload to the data source.

Query Information Tables

The following table returns query statistics for data modification queries:

• sys_identity: Returns information about batch operations or single updates.

Lists the available databases.

The following query retrieves all databases determined by the connection string:

SELECT * FROM sys_catalogs

Columns

Lists the available schemas.

The following query retrieves all available schemas:

          SELECT * FROM sys_schemas
          

Columns

Lists the available tables.

The following query retrieves the available tables and views:

          SELECT * FROM sys_tables
          

Columns

Describes the columns of the available tables and views.

The following query returns the columns and data types for the Movies table:

SELECT ColumnName, DataTypeName FROM sys_tablecolumns WHERE TableName='Movies' 

Columns

Lists the available stored procedures.

The following query retrieves the available stored procedures:

          SELECT * FROM sys_procedures
          

Columns

Describes stored procedure parameters.

The following query returns information about all of the input parameters for the SelectEntries stored procedure:

SELECT * FROM sys_procedureparameters WHERE ProcedureName='SelectEntries' AND Direction=1 OR Direction=2

Columns

Describes the primary and foreign keys.

The following query retrieves the primary key for the Movies table:

         SELECT * FROM sys_keycolumns WHERE IsKey='True' AND TableName='Movies' 
          

Columns

Describes the foreign keys.

The following query retrieves all foreign keys which refer to other tables:

         SELECT * FROM sys_foreignkeys WHERE ForeignKeyType = 'FOREIGNKEY_TYPE_IMPORT'
          

Columns

Describes the primary keys.

The following query retrieves the primary keys from all tables and views:

         SELECT * FROM sys_primarykeys
          

Columns

Describes the available indexes. By filtering on indexes, you can write more selective queries with faster query response times.

The following query retrieves all indexes that are not primary keys:

          SELECT * FROM sys_indexes WHERE IsPrimary='false'
          

Columns

Returns information on the available connection properties and those set in the connection string.

When querying this table, the config connection string should be used:

jdbc:cdata:cloudant:config:

This connection string enables you to query this table without a valid connection.

The following query retrieves all connection properties that have been set in the connection string or set through a default value:

SELECT * FROM sys_connection_props WHERE Value <> ''

Columns

Describes the SELECT query processing that the Cloud can offload to the data source.

See SQL Compliance for SQL syntax details.

Discovering the Data Source's SELECT Capabilities

Below is an example data set of SQL capabilities. Some aspects of SELECT functionality are returned in a comma-separated list if supported; otherwise, the column contains NO.

The following query retrieves the operators that can be used in the WHERE clause:

SELECT * FROM sys_sqlinfo WHERE Name = 'SUPPORTED_OPERATORS'

Columns

Returns information about attempted modifications.

The following query retrieves the Ids of the modified rows in a batch operation:

         SELECT * FROM sys_identity
          

Columns

Stored procedures are function-like interfaces that extend the functionality of the Cloud beyond simple SELECT/INSERT/UPDATE/DELETE operations with Cloudant.

Stored procedures accept a list of parameters, perform their intended function, and then return any relevant response data from Cloudant, along with an indication of whether the procedure succeeded or failed.

CData Cloud - Cloudant Stored Procedures

Customizing the SSL Configuration

By default, the Cloud attempts to negotiate SSL/TLS by checking the server's certificate against the system's trusted certificate store.

To specify another certificate, see the SSLServerCert property for the available formats to do so.

Connecting Through a Firewall or Proxy

HTTP Proxies

To connect through the Windows system proxy, you do not need to set any additional connection properties. To connect to other proxies, set ProxyAutoDetect to false.

In addition, to authenticate to an HTTP proxy, set ProxyAuthScheme, ProxyUser, and ProxyPassword, in addition to ProxyServer and ProxyPort.

Other Proxies

Set the following properties:

• To use a proxy-based firewall, set FirewallType, FirewallServer, and FirewallPort.
• To tunnel the connection, set FirewallType to TUNNEL.
• To authenticate, specify FirewallUser and FirewallPassword.
• To authenticate to a SOCKS proxy, additionally set FirewallType to SOCKS5.

The connection string properties are the various options that can be used to establish a connection. This section provides a complete list of the options you can configure in the connection string for this provider. Click the links for further details.

Authentication

SSL

Logging

Schema

Miscellaneous

This section provides a complete list of the Authentication properties you can configure in the connection string for this provider.

The type of authentication to use when connecting to Cloudant.

Possible Values

Data Type

string

Default Value

"Auto"

Remarks

• Auto: Lets the driver decide automatically based on the other connection properties you have set.
• Basic: Set this to use BASIC user / password authentication.
• OAuth: Set this to perform OAuth authentication. For this method you will need the APIKey and URL of Cloudant instance.

The URL used to connect to the Cloudant.

Data Type

string

Default Value

""

Remarks

The HTTP or HTTPS URL used to connect to the Cloudant in the format https://{instance id}-bluemix.cloudantnosqldb.appdomain.cloud or in case of self hosting server http[s]://{server}:{port}

The Cloudant user account used to authenticate.

Data Type

string

Default Value

""

Remarks

Together with Password, this field is used to authenticate against the Cloudant server.

The password used to authenticate the user.

Data Type

string

Default Value

""

Remarks

The User and Password are together used to authenticate with the server.

The API Key used to identify the user to IBM Cloud.

Data Type

string

Default Value

""

Remarks

Access to resources in the Cloudant REST API is governed by an API key in order to retrieve token. An API Key created by navigating to Manage --> Access (IAM) --> Users and clicking 'Create'.

This section provides a complete list of the SSL properties you can configure in the connection string for this provider.

The certificate to be accepted from the server when connecting using TLS/SSL.

Data Type

string

Default Value

""

Remarks

If using a TLS/SSL connection, this property can be used to specify the TLS/SSL certificate to be accepted from the server. Any other certificate that is not trusted by the machine is rejected.

This property can take the following forms:

If not specified, any certificate trusted by the machine is accepted.

Use '*' to signify to accept all certificates. Note that this is not recommended due to security concerns.

This section provides a complete list of the Logging properties you can configure in the connection string for this provider.

The verbosity level that determines the amount of detail included in the log file.

Data Type

string

Default Value

"1"

Remarks

The verbosity level determines the amount of detail that the Cloud reports to the Logfile. Verbosity levels from 1 to 5 are supported. These are detailed in the Logging page.

This section provides a complete list of the Schema properties you can configure in the connection string for this provider.

This property restricts the schemas reported to a subset of the available schemas. For example, BrowsableSchemas=SchemaA,SchemaB,SchemaC.

Data Type

string

Default Value

""

Remarks

Listing the schemas from databases can be expensive. Providing a list of schemas in the connection string improves the performance.

Whether to list views from Cloudant or not.

Data Type

bool

Default Value

false

Remarks

Set this property to true for Cloudant views to show up when listing tables.

This section provides a complete list of the Miscellaneous properties you can configure in the connection string for this provider.

Set FlattenArrays to the number of array elements to flatten into columns. Otherwise, arrays are returned as JSON strings.

Data Type

string

Default Value

""

Remarks

By default, nested arrays are returned as strings of JSON. The FlattenArrays property can be used to flatten the elements of nested arrays into columns of their own: Set FlattenArrays to the number of elements you want to return from nested arrays. The zero-based index is concatenated to the column name with a dot or the character specified by SeparatorCharacter. Other elements are ignored.

For example, you can return an arbitrary number of elements from an array of strings:

["FLOW-MATIC","LISP","COBOL"]

Setting FlattenArrays to -1 will flatten all the elements of nested arrays.

Using FlattenArrays is only recommended for arrays that are expected to be short. See NoSQL Database for other ways to access nested arrays.

Set FlattenObjects to true to flatten object properties into columns of their own. Otherwise, embedded objects as raw JSON strings.

Data Type

bool

Default Value

true

Remarks

Set FlattenObjects to true to flatten object properties into columns of their own. Otherwise, embedded objects as raw JSON strings. The property name is concatenated onto the object name with a dot or the character specified by SeparatorCharacter.

For example, you can flatten the nested objects below at connection time:

[
  { "grade": "A", "score": 2 },
  { "grade": "A", "score": 6 },
  { "grade": "A", "score": 10 },
  { "grade": "A", "score": 9 },
  { "grade": "B", "score": 14 }
]

Set FlexibleSchema to true to scan for additional metadata on the query result set. Otherwise, the metadata will remain the same.

Data Type

bool

Default Value

true

Remarks

Set FlexibleSchema to true to scan for additional metadata on the query result set. Otherwise, the metadata will remain the same.

Limits the number of rows returned when no aggregation or GROUP BY is used in the query. This takes precedence over LIMIT clauses.

Data Type

int

Default Value

-1

Remarks

Limits the number of rows returned when no aggregation or GROUP BY is used in the query. This takes precedence over LIMIT clauses.

The maximum number of results to return per page from Cloudant.

Data Type

int

Default Value

1000

Remarks

The Pagesize property affects the maximum number of results to return per page from Cloudant. Setting a higher value may result in better performance at the cost of additional memory allocated per page consumed.

This property indicates whether or not to include pseudo columns as columns to the table.

Data Type

string

Default Value

""

Remarks

This setting is particularly helpful in Entity Framework, which does not allow you to set a value for a pseudo column unless it is a table column. The value of this connection setting is of the format "Table1=Column1, Table1=Column2, Table2=Column3". You can use the "*" character to include all tables and all columns; for example, "*=*".

The maximum number of rows to scan to look for the columns available in a table.

Data Type

int

Default Value

50

Remarks

The columns in a table must be determined by scanning table rows. This value determines the maximum number of rows that will be scanned.

Setting a high value may decrease performance. Setting a low value may prevent the data type from being determined properly, especially when there is null data.

The character or characters used to denote hierarchy.

Data Type

string

Default Value

"."

Remarks

In order to flatten out hierarchical structures, the Cloud needs some specifier that states the path to a column through the hierarchy. If this value is "." and a column comes back with the name address.city, this indicates that there is a mapped attribute with a child called city. If your data has columns that already use a single period within the attribute name, set the SeparatorCharacter to a different character or characters.

The value in seconds until the timeout error is thrown, canceling the operation.

Data Type

int

Default Value

60

Remarks

If Timeout = 0, operations do not time out. The operations run until they complete successfully or until they encounter an error condition.

If Timeout expires and the operation is not yet complete, the Cloud throws an exception.

Option for how the provider will scan the data to determine the fields and data types in each document collection.

Data Type

string

Default Value

"RowScan"

Remarks

Since Cloudant is schemaless, the columns in a table must be determined by scanning table rows. Set TypeDetectionScheme to toggle the automatic discovery of data types. Set RowScanDepth to configure type detection.

See Also

Type detection is part of Automatic Schema Discovery. Set FlattenArrays and FlattenObjects to configure how the Cloud projects columns over the hierarchical data.

You can fine-tune the discovered columns and data types by calling CreateSchema to generate a schema file. Schema files have a simple format that makes it easy to change column behavior.