Google BigQuery Targets in Mappings

Property	Description
Connection	Name of the active Google BigQuery connection that is associated with a dataset.
Target Type	Type of the Google BigQuery target objects available. You can write data to a single Google BigQuery target object. You cannot write data to multiple objects or parameterize the object.
Object	Name of the Google BigQuery target object based on the target type selected.
Operation	You can select one the following operations: - Insert - Update - Upsert (Update or Insert) - Delete - Data Driven Note: If you use complex connection mode, you cannot configure update, upsert, and delete operations.
Update Columns	Specifies the temporary primary key columns to update, upsert or delete target data. If the Google BigQuery target does not include a primary key column, and the mapping performs an update, upsert, or delete task operation, click Add to add a temporary key. You can select multiple columns. By default, no columns are specified.

Property	Description
Target Dataset ID	Optional. Overrides the Google BigQuery dataset name that you specified in the connection.
Target Table Name	Optional. Overrides the Google BigQuery target table name that you specified in the Target transformation.
Create Disposition	Specifies whether the Secure Agent must create the target table if it does not exist. You can select one of the following values: - Create if needed. If the table does not exist, the Secure Agent creates the table. - Create never. If the table does not exist, the Secure Agent does not create the table and displays an error message.
Write Disposition	Specifies how the Secure Agent must write data in bulk mode if the target table already exists. You can select one of the following values: - Write append. If the target table exists, the Secure Agent appends the data to the existing data in the table. - Write truncate. If the target table exists, the Secure Agent overwrites the existing data in the table. - Write empty. If the target table exists and contains data, the Secure Agent displays an error and does not write the data to the target. The Secure Agent writes the data to the target only if the target table does not contain any data. Note: Write disposition is applicable for bulk mode. Note: Write disposition is applicable only when you perform an insert operation on a Google BigQuery target.
Write Mode	Specifies the mode that the Secure Agent uses to write data to the BigQuery target. You can select one of the following modes: - Bulk. In bulk mode, the Secure Agent first writes the data to a staging file in Google Cloud Storage. When the staging file contains all the data, the Secure Agent transfers the data from the staging file to the BigQuery target. The Secure Agent then deletes the staging file unless you configure the mapping to persist the staging file. - Streaming. In streaming mode, the Secure Agent directly writes data to the BigQuery target. The Secure Agent writes the data into the target row by row. Default is Bulk mode.
Streaming Template Table Suffix	Specifies the suffix that Google BigQuery adds to the individual target tables that it creates based on the template target table. This property applies to streaming mode.
Rows per Streaming Request	Specifies the number of rows that the Secure Agent streams to the BigQuery target for each request. Default is 500 rows. The maximum row size that the Secure Agent can stream to the BigQuery target for each request is 10 MB. This property applies to streaming mode.
Staging File Name	Specifies the name of the staging file that the Secure Agent creates in Google Cloud Storage before it writes the data to the BigQuery target. This property applies to bulk mode.
Data Format of the staging file	Specifies the data format of the staging file. You can select one of the following data formats: - JSON (Newline Delimited). Supports flat and record data with nested and repeated fields. - CSV. Supports flat data. Note: If you use hybrid and complex connection mode, you cannot use CSV format as the data format of the staging file.
Persist Staging File After Loading	Indicates whether the Secure Agent must persist the staging file in the Google Cloud Storage after it writes the data to the BigQuery target. You can persist the staging file if you want to archive the data for future reference. By default, the Secure Agent deletes the staging file in Google Cloud Storage. This property applies to bulk mode.
Enable Staging File Compression	Indicates whether the Secure Agent must compress the size of the staging file before it writes data to Google Cloud Storage and decompress the staging file before it writes the data to the BigQuery target. You can enable staging file compression to reduce cost and transfer time.
Job Poll Interval in Seconds	The number of seconds after which the Secure Agent polls the status of the write job operation. Default is 10.
Number of Threads for Uploading Staging file	The number of threads that the Secure Agent must create to upload the staging file in bulk mode. If you configure more than one thread, you must increase the Java heap size in the JVMOption3 field for DTM under the System Configuration Details section of the Secure Agent. You must then restart the Secure Agent for the changes to take effect.
Local Storage File Directory	Specifies the location where the Secure Agent stores the files temporarily before writing the data to the staging file in Google Cloud Storage. The directory is on the local machine where you installed the Secure Agent. This property applies to bulk mode.
Allow Quoted Newlines	Indicates whether the Secure Agent must allow the quoted data sections with newline character in a .csv file.
Field Delimiter	Indicates whether the Secure Agent must allow field separators for the fields in a .csv file.
Allow Jagged Rows	Indicates whether Secure Agent must accept the rows without trailing columns in a .csv file.
Success File Directory	Not applicable for Google BigQuery Connector.
Error File Directory	Not applicable for Google BigQuery Connector.
Forward Rejected Rows	Not applicable for Google BigQuery Connector.