Connection Properties
The connection string properties describe the various options that can be used to establish a connection.
Adapter
CData.BizTalk.GoogleBigQueryAdapter.dll
Remarks
The connection string can be set to a series of "option=value" pairs, separated by semicolons. If a connection string property value has special characters such as semicolons, single quotes, spaces, etc., then you must quote the value using either single or double quotes.
Connection options are case insensitive.
To specify a location to the database where the tables, views, and stored procedures are located, set the Location property.
Authenticate via OAuth Authentication
Use the OAuth authentication standard to connect to Google BigQuery. You can authenticate with a user account or with a service account. A service account is required to grant organization-wide access scopes to the adapter. The adapter facilitates these authentication flows as described below.
Authenticate with a User Account
You can connect without setting any connection properties for your user credentials. After setting the following, you are ready to connect:
- DatasetId: Set this to the Id of the dataset you want to connect to.
- ProjectId: Set this to the Id of the project you want to connect to.
- InitiateOAuth: Set this to GETANDREFRESH. You can use InitiateOAuth to avoid repeating the OAuth exchange and manually setting the OAuthAccessToken.
- Extracts the access token from the callback URL and authenticates requests.
- Obtains a new access token when the old one expires.
- Saves OAuth values in OAuthSettingsLocation to be persisted across connections.
See Using OAuth Authentication for other OAuth authentication flows.
Authenticate with a Service Account
Service accounts have silent authentication, without user authentication in the browser. You can also use a service account to delegate enterprise-wide access scopes to the adapter.
You need to create an OAuth application in this flow. See Creating a Custom OAuth App in the Getting Started section to create and authorize an app. You can then connect to Google BigQuery data that the service account has permission to access.
After setting the following connection properties, you are ready to connect:
- InitiateOAuth: Set this to GETANDREFRESH.
- OAuthJWTCertType: Set this to "PFXFILE".
- OAuthJWTCert: Set this to the path to the .p12 file you generated.
- OAuthJWTCertPassword: Set this to the password of the .p12 file.
- OAuthJWTCertSubject: Set this to "*" to pick the first certificate in the certificate store.
- OAuthJWTSubject: Set this to the email address of the user for whom the application is requesting delegate access. Note that delegate access must be granted by an administrator.
- OAuthJWTIssuer: In the service accounts section, click Manage Service Accounts and set this field to the email address displayed in the service account Id field.
- DatasetId: Set this to the Id of the dataset you want to connect to.
- ProjectId: Set this to the Id of the project you want to connect to.
- Creates and signs the JWT with the claim set required by the adapter.
- Exchanges the JWT for the access token.
- Saves OAuth values in OAuthSettingsLocation to be persisted across connections.
- Submits the JWT for a new access token when the token expires.
Connection String Options
The following is the full list of the options you can configure in the connection string for this provider. Click on the links for further details.
| Allow Aggregate Parameters | Allows raw aggregates to be used in parameters when QueryPassthrough is enabled. |
| Allow Large Result Sets | Whether or not to allow large datasets to be stored in temporary tables for large datasets. |
| Allow Updates Without Key | Whether or not to allow update without primary keys. |
| Audit Limit | The maximum number of rows which will be stored within an audit table. |
| Audit Mode | What provider actions should be recorded to audit tables. |
| Auth Scheme | The type of authentication to use when connecting to Google BigQuery. |
| Auto Cache | Automatically caches the results of SELECT queries into a cache database specified by either CacheLocation or both of CacheConnection and CacheProvider . |
| Big Query Options | A comma separated list of Google BigQuery options. |
| Cache Connection | The connection string for the cache database. This property is always used in conjunction with CacheProvider . Setting both properties will override the value set for CacheLocation for caching data. |
| Cache Location | Specifies the path to the cache when caching to a file. |
| Cache Metadata | This property determines whether or not to cache the table metadata to a file store. |
| Cache Provider | The name of the provider to be used to cache data. |
| Cache Tolerance | The tolerance for stale data in the cache specified in seconds when using AutoCache . |
| Dataset Id | The DatasetId of the dataset you wish to connect to and view tables of. |
| Destination Table | DestinationTable is entered in the format DestinationProjectId:DestinationDataSet.TableName. Setting DestinationTable allows the query to return very large result sets (more than 128mb of result data) to a permanent destination table. |
| Filter Columns | Please set `AllowUpdatesWithoutKey` to true before you could use this property. |
| Firewall Password | A password used to authenticate to a proxy-based firewall. |
| Firewall Port | The TCP port for a proxy-based firewall. |
| Firewall Server | The name or IP address of a proxy-based firewall. |
| Firewall Type | The protocol used by a proxy-based firewall. |
| Firewall User | The user name to use to authenticate with a proxy-based firewall. |
| Generate Schema Files | Indicates the user preference as to when schemas should be generated and saved. |
| Initiate OAuth | Set this property to initiate the process to obtain or refresh the OAuth access token when you connect. |
| Location | A path to the directory that contains the schema files defining tables, views, and stored procedures. |
| Logfile | A filepath which designates the name and location of the log file. |
| Log Modules | Core modules to be included in the log file. |
| Maximum Billing Tier | The MaximumBillingTier is a positive integer that serves as a multiplier of the basic price per TB. For example, if you set MaximumBillingTier to 2, the maximum cost for that query will be 2x basic price per TB. |
| Maximum Bytes Billed | Limits how many bytes BigQuery will allow a job to consume before it is cancelled. |
| Max Log File Count | A string specifying the maximum file count of log files. When the limit is hit, a new log is created in the same folder with the date and time appended to the end and the oldest log file will be deleted. |
| Max Log File Size | A string specifying the maximum size in bytes for a log file (for example, 10 MB). When the limit is hit, a new log is created in the same folder with the date and time appended to the end. |
| Max Rows | Limits the number of rows returned rows when no aggregation or group by is used in the query. This helps avoid performance issues at design time. |
| OAuth Access Token | The access token for connecting using OAuth. |
| OAuth Client Id | The client ID assigned when you register your application with an OAuth authorization server. |
| OAuth Client Secret | The client secret assigned when you register your application with an OAuth authorization server. |
| OAuth Expires In | The lifetime in seconds of the OAuth AccessToken. |
| OAuth JWT Cert | The JWT Certificate store. |
| OAuth JWT Cert Password | The password for the OAuth JWT certificate. |
| OAuth JWT Cert Subject | The subject of the OAuth JWT certificate. |
| OAuth JWT Cert Type | The type of key store containing the JWT Certificate. |
| OAuth JWT Issuer | The issuer of the Java Web Token. |
| OAuth JWT Subject | The user subject for which the application is requesting delegated access. |
| OAuth Refresh Token | The OAuth refresh token for the corresponding OAuth access token. |
| OAuth Settings Location | The location of the settings file where OAuth values are saved when InitiateOAuth is set to GETANDREFRESH or REFRESH. Alternatively, this can be held in memory by specifying a value starting with memory://. |
| OAuth Token Timestamp | The Unix epoch timestamp in milliseconds when the current Access Token was created. |
| OAuth Verifier | The verifier code returned from the OAuth authorization URL. |
| Offline | Use offline mode to get the data from the cache instead of the live source. |
| Other | These hidden properties are used only in specific use cases. |
| Page Size | The number of results to return per page from Google BigQuery. |
| Polling Interval | This determines how long to wait in seconds, between checks to see if a job has completed. |
| Project Id | The ProjectId of the billing project for executing jobs. |
| Proxy Auth Scheme | The authentication type to use to authenticate to the ProxyServer proxy. |
| Proxy Auto Detect | This indicates whether to use the system proxy settings or not. This takes precedence over other proxy settings, so you'll need to set ProxyAutoDetect to FALSE in order use custom proxy settings. |
| Proxy Exceptions | A semicolon separated list of destination hostnames or IPs that are exempt from connecting through the ProxyServer . |
| Proxy Password | A password to be used to authenticate to the ProxyServer proxy. |
| Proxy Port | The TCP port the ProxyServer proxy is running on. |
| Proxy Server | The hostname or IP address of a proxy to route HTTP traffic through. |
| Proxy SSL Type | The SSL type to use when connecting to the ProxyServer proxy. |
| Proxy User | A user name to be used to authenticate to the ProxyServer proxy. |
| Pseudo Columns | This property indicates whether or not to include pseudo columns as columns to the table. |
| Query Passthrough | This option passes the query to the Google BigQuery server as is. |
| Readonly | You can use this property to enforce read-only access to Google BigQuery from the provider. |
| Refresh View Schemas | Allows the provider to determine up-to-date view schemas automatically. |
| RTK | The runtime key used for licensing. |
| Scope | Identifies the Google API access that your application is requesting. Scopes are space-delimited. |
| Show Table Descriptions | Controls whether table descriptions are returned via the platform metadata APIs and sys_tables / sys_views. |
| SSL Server Cert | The certificate to be accepted from the server when connecting using TLS/SSL. |
| Storage Page Size | Specifies the page size to use for Storage API queries. |
| Storage Threshold | The minimum number of rows a query must return to invoke the Storage API. |
| Tables | This property restricts the tables reported to a subset of the available tables. For example, Tables=TableA,TableB,TableC. |
| Temp Table Dataset | The prefix of the dataset that will contain temporary tables when executing queries with large result sets. |
| Temp Table Expiration Time | Time, in seconds until the temporary table expires. Set to 0 to have the table never expire. |
| Timeout | The value in seconds until the timeout error is thrown, canceling the operation. |
| Use Arrow Format | Specifies whether to use the Arrow format with BigQuery's Storage API. |
| Use Legacy SQL | Specifies whether to use BigQuery's legacy SQL dialect for this query. By default, Standard SQL will be used. |
| Use Query Cache | Specifies whether to use Google BigQuery's built-in query cache. |
| Use Storage API | Specifies whether to use BigQuery's Storage API for bulk data reads. |
| Use Streaming Inserts | Specifies whether to use streaming or DML statements when inserting data. By default, streaming inserts are used. |
| Views | Restricts the views reported to a subset of the available tables. For example, Views=ViewA,ViewB,ViewC. |