TDV Adapter for Google BigQuery

Build 22.0.8462

InsertLoadJob

Inserts a Google BigQuery load job, which adds data from Google Cloud Storage into an existing table.

Input

Name Type Description
SourceURIs String A space-separated list of Google Cloud Storage URIs
SourceFormat String The source format that the files are formatted in.

The allowed values are AVRO, NEWLINE_DELIMITED_JSON, DATASTORE_BACKUP, PARQUET, ORC, CSV.

DestinationTable String The destination table for the query, in the format DestProjectId.DestDatasetId.DestTable
DestinationTableProperties String A JSON object containing the table friendlyName, description and list of labels.
DestinationTableSchema String A JSON list contianing the fields used to create the table.
DestinationEncryptionConfiguration String A JSON object giving the KMS encryption settings for the table.
SchemaUpdateOptions String A JSON list giving the options to apply when updating the destination table schema.
TimePartitioning String A JSON object giving the time partitioning type and field.
RangePartitioning String A JSON object giving the range partitioning field and buckets.
Clustering String A JSON object giving the fields to be used for clustering.
Autodetect String Whether options and schema should be automatically determined for JSON and CSV files.
CreateDisposition String Whether to create the destination table if it does not exist.

The allowed values are CREATE_IF_NEEDED, CREATE_NEVER.

The default value is CREATE_IF_NEEDED.

WriteDisposition String How to write data to the destination table, such as truncate existing results, appending existing results, or writing only when the table is empty.

The allowed values are WRITE_TRUNCATE, WRITE_APPEND, WRITE_EMPTY.

The default value is WRITE_APPEND.

Region String The region to start executing the job in. Both the GCS resources and the BigQuery dataset must be in the same region.
DryRun String Whether or not this is a dry run of the job.

The default value is false.

MaximumBadRecords String If provided, the number of records that can be invalid before the entire job is canceled. By default all records must be valid.

The default value is 0.

IgnoreUnknownValues String Whether to ignore unknown fields in the input file or treat them as errors. By default they are treated as errors.

The default value is false.

AvroUseLogicalTypes String Whether to use Avro logical types when converting Avro data into BigQuery types.

The default value is true.

CSVSkipLeadingRows String How many rows to skip at the start of CSV files. Usually used for skipping header rows.
CSVEncoding String The name of the encoding used for CSV files.

The allowed values are ISO-8859-1, UTF-8.

The default value is UTF-8.

CSVNullMarker String If provided, this string is used for NULL values within CSV files. By default CSV files cannot use NULL.
CSVFieldDelimiter String The character used to separate columns within CSV files.

The default value is ,.

CSVQuote String The character used for quoted fields in CSV files. May be set to empty to disable quoting.

The default value is ".

CSVAllowQuotedNewlines String Whether CSV files can contain newlines within quoted fields.

The default value is false.

CSVAllowJaggedRows String Whether lines in CSV files may contain missing fields. False by default

The default value is false.

DSBackupProjectionFields String A JSON list of fields to load from a Cloud datastore backup.
ParquetOptions String A JSON object giving the Parquet-specific import options.
DecimalTargetTypes String A JSON list giving the preference order applied to numeric types.
HivePartitioningOptions String A JSON object giving the source-side partitioning options.

Result Set Columns

Name Type Description
JobId String The JobId of the newly insert Job.
Region String The region where the job is executing.
Configuration_load_destinationTable_tableId String The destination table tableId of the newly inserted Job.
Configuration_load_destinationTable_projectId String The destination table projectId of the newly inserted Job.
Configuration_load_destinationTable_datasetId String The destination table datasetId of the newly inserted Job.
Status_State String Running state of the job.
Status_errorResult_reason String A short error code that summarizes the error.
Status_errorResult_message String A human-readable description of the error.

Copyright (c) 2023 CData Software, Inc. - All rights reserved.
Build 22.0.8462