INSERT INTO SELECT Statements
To perform multiple inserts in a single request to Google BigQuery, use the INSERT ... SELECT syntax to insert a temporary table of data into Google BigQuery. This works by first populating a temporary table with the data you are going to submit to Google BigQuery. Once you have all of the data you want to insert, the temporary table is then passed into the table you are inserting to.
Populate the Temporary Table
The temporary table you are populating is dynamic and is created at run time the first time you insert to it. Temporary tables are denoted by a # appearing in their name. When using a temporary table to insert, the temporary table must be named in the format [TableName]#TEMP, where TableName is the
name of the table you will be inserting to. For example:
INSERT INTO publicdata.samples.github_nested#TEMP (repository.name, MyCustomField__c) VALUES ('New publicdata.samples.github_nested', '9000'); INSERT INTO publicdata.samples.github_nested#TEMP (repository.name, MyCustomField__c) VALUES ('New publicdata.samples.github_nested 2', '9001'); INSERT INTO publicdata.samples.github_nested#TEMP (repository.name, MyCustomField__c) VALUES ('New publicdata.samples.github_nested 3', '9002');
This creates a temporary table called publicdata.samples.github_nested#TEMP with two columns and three rows of data. Since type cannot be determined on the temporary table itself, all values are stored in memory as strings. They are later converted to the proper type when they are submitted to the publicdata.samples.github_nested table.
Insert to the Actual Table
Once your temporary table is populated, it is now time to insert to the actual table in Google BigQuery. You can do this by performing an INSERT to the actual table and selecting the input data from the temporary table. For example:
INSERT INTO publicdata.samples.github_nested (repository.name, MyCustomField__c) SELECT repository.name, MyCustomField__c FROM publicdata.samples.github_nested#TEMPIn this example, the full contents of the publicdata.samples.github_nested#TEMP table are passed into the publicdata.samples.github_nested table. This results in fewer requests being submitted to Google BigQuery since multiple inserts may be submitted with each request, which is much better for performance if you have many records to insert.
The results of the query are stored in the LastResultInfo#TEMP temporary table. This table is cleared and repopulated the next time data is modified by passing in a temporary table. Please be aware that the LastResultInfo#TEMP table has no predefined schema. You need to check its metadata at run time before reading data.
Temporary Table Life Span
Temporary tables only last as long as the connection remains open. When the connection to Google BigQuery is closed, all temporary tables are cleared, including the LastResultInfo#TEMP table.