Google BigQuery Sources in Mappings

Property	Description
Connection	Name of the active Google BigQuery source connection.
Source Type	Type of the Google BigQuery source objects available. You can read data from a single Google BigQuery source object. You cannot read data from multiple objects or parameterize the object.
Object	Name of the Google BigQuery source object based on the source type selected.
Filter	Configure a simple filter or an advanced filter to remove rows at the source. You can improve efficiency by filtering early in the data flow. A simple filter includes a field name, operator, and value. Use an advanced filter to define a more complex filter condition, which can include multiple conditions using the AND or OR logical operators.

Property	Description
Source Dataset ID	Optional. Overrides the Google BigQuery dataset name that you specified in the connection.
Number of Rows to Read	Specifies the number of rows that the Secure Agent must read from the Google BigQuery source.
Allow Large Results	Specifies whether the Secure Agent must allow large results for large source files. If you select this option, you must specify a destination table to store the query results.
Query Results Table Name	Required if you select the Allow Large Results option. Specifies the destination table name where the Secure Agent stores the query results. If the table is not present in the dataset, the Secure Agent creates the destination table with the name that you specify.
Job Poll Interval in Seconds	The number of seconds after which the Secure Agent polls the status of the read job operation. Default is 10.
Read Mode	Specifies the read mode that the Secure Agent uses to read data from the Google BigQuery source. You can select one the following read modes: - Direct. In direct mode, the Secure Agent reads data directly from the Google BigQuery source table. Note: When you use hybrid and complex connection mode, you cannot use direct mode to read data from the Google BigQuery source. - Staging. In staging mode, the Secure Agent downloads data from the Google BigQuery source into Google Cloud Storage. After the download is complete, the Secure Agent downloads the data from Google Cloud Storage into the local stage file that you specify in Google Cloud Storage and then reads data from the local stage file. Default is Direct mode.
Number of Threads for Downloading Staging Files	Specifies the number of threads that the Secure Agent creates to enable parallel download. This property applies to staging mode.
Local Stage File Directory	Specifies the location in Google Cloud Storage where the Secure Agent stores Google BigQuery source data temporarily before Google BigQuery reads the data. This property applies to staging mode.
Staging File Name	Specifies the name of the staging file that the Secure Agent creates in Google Cloud Storage to store the Google BigQuery source data. This property applies to staging mode.
Enable Staging File Compression	Indicates whether the Secure Agent must compress the size of the staging file in Google Cloud Storage before it reads data from the staging file. You can enable staging file compression to reduce cost and transfer time. This property applies to staging mode.
Persist Destination Table	Indicates whether the Secure Agent must persist the query results table after it reads data from the query results table. By default, the Secure Agent deletes the query results table

Property	Description
Terse	The Secure Agent logs initialization information, error messages, and notification of rejected data.
Normal	The Secure Agent logs initialization and status information, errors encountered, and skipped rows due to transformation row errors. Summarizes session results, but not at the level of individual rows.
Verbose Initialization	In addition to normal tracing, the Secure Agent logs additional initialization details, names of index and data files used, and detailed transformation statistics.
Verbose Data	In addition to verbose initialization tracing, the Secure Agent logs each row that passes into the mapping. Also notes where the Secure Agent truncates string data to fit the precision of a column and provides detailed transformation statistics. When you configure the tracing level to verbose data, the Secure Agent writes row data for all rows in a block when it processes a transformation.