Property | Description |
|---|---|
Connection Name | Name of the connection. Each connection name must be unique within the organization. Connection names can contain alphanumeric characters, spaces, and the following special characters: _ . + -, Maximum length is 255 characters. |
Description | Description of the connection. Maximum length is 4000 characters. |
Use Secret Vault | Stores sensitive credentials for this connection in the secrets manager that is configured for your organization. This property appears only if secrets manager is set up for your organization. This property is not supported by Data Ingestion and Replication and the Data Access Management services. When you enable the secret vault in the connection, you can select the credentials that the Secure Agent retrieves from the secrets manager. If you don't enable this option, the credentials are stored in the repository or on a local Secure Agent, depending on how your organization is configured. Note: If you’re using this connection to apply data access policies through pushdown or proxy services, you cannot use the Secret Vault configuration option. For information about how to configure and use a secrets manager, see "Secrets manager configuration" in the Administrator help. |
Runtime Environment | The name of the runtime environment where you want to run tasks. Select a Secure Agent, Hosted Agent, or serverless runtime environment. For application ingestion and replication tasks and database ingestion and replication tasks, you can use a Secure Agent or serverless runtime environment. The tasks must have a source type supported for serverless use. You cannot run an application ingestion and replication task, database ingestion and replication task, file ingestion or replication task, or streaming ingestion and replication task on a Hosted Agent or elastic runtime environment. For more information about runtime environments, see Runtime Environments in the Administrator help. |
Authentication type | Description |
|---|---|
Service Account | Authenticate with the credentials in a Google service account key file. |
WIF (Authorization Code) | Authenticate with WIF and an identity provider that uses the OAuth 2.0 authorization code grant. You sign in to the identity provider to generate an access token. You can configure Okta as the identity provider. |
WIF (Client Credentials) | Authenticate with WIF and an identity provider that uses the OAuth 2.0 client credentials grant. The connection doesn't require a user to sign in. You can also impersonate a service account and access datasets in a different project. You can configure Microsoft Entra ID as the identity provider. |
Property | Description |
|---|---|
Service Account Email | The client_email value from the Google service account key JSON file. |
Service Account Key | The private_key value from the Google service account key JSON file. To use the private key from the JSON file, enter the entire block of text found within the double quotation marks in the private_key field, starting with -----BEGIN PRIVATE KEY-----\n and ending at -----END PRIVATE KEY-----\n. |
Project ID | The project_id value from the Google service account key JSON file. If you have created multiple projects with the same service account, enter the ID of the project that contains the dataset that you want to connect to. |
Property | Description |
|---|---|
Enable BigQuery Storage API | Uses Google BigQuery Storage to stage the files when you read or write data. Default is unselected. |
Storage Path | Path in Google Cloud Storage where the agent creates a local stage file to store the data temporarily. The agent uses this storage when it reads data in staging mode or writes data in bulk mode. Use one of the following formats:
When you enable cross-region replication in Google BigQuery, enter a Google Cloud Storage path to a dual-region bucket. This property is not applicable if you use Google BigQuery Storage to stage the files. |
Connection Mode | The mode that you want to use to read data from or write data to Google BigQuery. Select one of the following connection modes:
Default is Simple. This property is applicable if you use Google Cloud Storage to stage the files. |
Use Legacy SQL for Custom Query¹ | Select this option to use legacy SQL to define a custom query. If you clear this option, use standard SQL to define a custom query. This property is applicable if you use Google Cloud Storage to stage the files. This property doesn't apply if you configure the Google BigQuery V2 connection in hybrid or complex mode. |
Dataset Name for Custom Query¹ | The name of the Google BigQuery dataset that you want to specify in a custom query. |
Schema Definition File Path¹ | Directory on the Secure Agent machine where the Secure Agent creates a JSON file with the sample schema of the Google BigQuery table. The JSON file name is the same as the Google BigQuery table name. Alternatively, you can specify a storage path in Google Cloud Storage where the Secure Agent creates a JSON file with the sample schema of the Google BigQuery table. You can download the JSON file from the specified storage path in Google Cloud Storage to a local machine. The schema definition file is required if you configure complex connection mode in the following scenarios:
When you use a serverless runtime environment, specify a storage path in Google Cloud Storage. This property is applicable if you use Google Cloud Storage to stage the files. |
Region ID | The region name where the Google BigQuery dataset that you want to access resides. Note: Ensure that you specify a bucket name or the bucket name and folder name in the Storage Path property that resides in the specified region. For the list of Google BigQuery regions, see Dataset locations. |
Staging Dataset¹ | The Google BigQuery dataset name where you want to create the staging table to stage the data. You can define a Google BigQuery dataset that is different from the source or target dataset. This property is applicable if you use Google Cloud Storage to stage the files. |
Provide Optional Properties¹ | Comma-separated key-value pairs of custom properties in the Google BigQuery V2 connection to configure certain source and target functionalities. For more information about the list of custom properties that you can specify, see Optional Properties configuration Knowledge article. |
¹ Doesn't apply to mappings in advanced mode. | |
Property | Description |
|---|---|
Project Number | The unique numeric identifier of the project in the Google service account that contains the dataset that you want to connect to. |
Authorization URL | The endpoint URL provided by the identity provider where user authentication requests are redirected to initiate the authorization process. |
Access Token URL | The endpoint URL provided by the identity provider where the client exchanges authorization codes or credentials to obtain access tokens. |
Client ID | A unique identifier assigned to your application by the identity provider. This ID is used to recognize the requester during authentication and authorization processes. |
Client Secret | A confidential key issued alongside the client ID, acting as a password between your application and the identity provider to prove the authenticity of the client during token exchanges. |
Pool ID | The identifier for the Workload Identity Pool configured in Google Cloud Platform which aggregates trusted external identities. |
Provider ID | The ID of the specific external identity provider configured within the Workload Identity Pool. |
Access Token | The access token granted by Google Cloud Platform to securely access Google BigQuery resources. Click Generate Token to generate a new access token. |
Property | Description |
|---|---|
Enable BigQuery Storage API | Uses Google BigQuery Storage to stage the files when you read or write data. Default is unselected. |
Storage Path | Path in Google Cloud Storage where the agent creates a local stage file to store the data temporarily. The agent uses this storage when it reads data in staging mode or writes data in bulk mode. Use one of the following formats:
When you enable cross-region replication in Google BigQuery, enter a Google Cloud Storage path to a dual-region bucket. This property is not applicable if you use Google BigQuery Storage to stage the files. |
Dataset Name for Custom Query | The name of the Google BigQuery dataset that you want to specify in a custom query. |
Region ID | The region name where the Google BigQuery dataset that you want to access resides. Note: Ensure that you specify a bucket name or the bucket name and folder name in the Storage Path property that resides in the specified region. For the list of Google BigQuery regions, see Dataset locations. |
Provide Optional Properties | Comma-separated key-value pairs of custom properties in the Google BigQuery V2 connection to configure certain source and target functionality. For more information about the list of custom properties that you can specify, see Optional Properties configuration Knowledge article. |
Property | Description |
|---|---|
Project Number | The unique numeric identifier of the Google Cloud project that contains the Workload Identity Pool. |
Access Token URL | The endpoint URL provided by the identity provider where the client exchanges credentials to obtain access tokens. Enter an HTTPS URL. For Microsoft Entra ID, use the following syntax: https://login.microsoftonline.com/<tenant ID>/oauth2/v2.0/token |
Client ID | A unique identifier assigned to your application by the identity provider. This ID is used to recognize the requester during authentication and authorization processes. |
Client Secret | A confidential key issued alongside the client ID, acting as a password between your application and the identity provider to prove the authenticity of the client during token exchanges. |
Pool ID | The identifier for the Workload Identity Pool configured in Google Cloud Platform which aggregates trusted external identities. |
Provider ID | The ID of the specific external identity provider configured within the Workload Identity Pool. |
Scope | The OAuth scope that the connector requests when it exchanges credentials for an access token. Use one of the following formats:
This field has no default value. Specify a scope. |
Access Token | The access token granted by Google Cloud Platform to securely access Google BigQuery resources. Click Generate Token to generate a new access token. |
Enable Service Account Impersonation | Determines whether to impersonate a Google service account to access BigQuery resources. When you select this option, the connector exchanges the federated token for a short-lived access token of the target service account. The Target Service Account Email property appears. Default is unselected. |
Target Service Account Email | Email address of the Google service account to impersonate. The connector uses this account's permissions to access BigQuery resources. Required when you select the Enable Service Account Impersonation property. Use the following syntax: <service account name>@<project ID>.iam.gserviceaccount.com |
Use Data Project | Access datasets in a project other than the Workload Identity Federation pool project. When you select this option, the Data Project ID property appears. If you clear this option, the Resource Manager endpoint is used to resolve the project from the pool. Default is unselected. |
Data Project ID | ID of the Google Cloud project that contains your BigQuery datasets. Required when you select the Use Data Project property. |
Enable BigQuery Storage API | Uses Google BigQuery Storage to stage the files when you read or write data. Default is unselected. |
Storage Path | Path in Google Cloud Storage where the agent creates a local stage file to store the data temporarily. The agent uses this storage when it reads data in staging mode or writes data in bulk mode. Use one of the following formats: gs://<bucket_name> or gs://<bucket_name>/<folder_name>. When you enable cross-region replication in Google BigQuery, enter a Google Cloud Storage path to a dual-region bucket. This property is not applicable if you use Google BigQuery Storage to stage the files. |
Dataset Name for Custom Query | The name of the Google BigQuery dataset that you want to specify in a custom query. |
Region ID | The region name where the Google BigQuery dataset that you want to access resides. Ensure that you specify a bucket name or the bucket name and folder name in the Storage Path property that resides in the specified region. For the list of Google BigQuery regions, see Dataset locations. |
Provide Optional Properties | Comma-separated key-value pairs of custom properties in the Google BigQuery V2 connection to configure certain source and target functionality. For more information about the list of custom properties that you can specify, see Optional Properties configuration Knowledge article. |
Property | Description |
|---|---|
Enable Retry¹ | Select this option if you want the Secure Agent to attempt a retry to receive the response from the Google BigQuery endpoint. For more information about the errors for which the connector can attempt a retry, see Error messages in the Google BigQuery documentation. You can configure the retry strategy to read data from Google BigQuery in direct or staging mode and write data to Google BigQuery in bulk mode. The retry strategy is not applicable in the CDC and streaming modes when you write data to a Google BigQuery target. The connection retry option also applies to a connection configured to use the proxy server to connect to the endpoint. Default is unselected. |
Maximum Retry Attempts | Appears only if you select the Enable Retry property. The maximum number of retry attempts that the Secure Agent performs to receive the response from the Google BigQuery endpoint. If the Secure Agent fails to connect to Google BigQuery within the maximum retry attempts, the connection fails. Default is 6 attempts. |
Initial Retry Delay | Appears only if you select the Enable Retry property. The initial wait time in seconds before the Secure Agent attempts to retry the connection. Default is 1 second. |
Retry Delay Multiplier | Appears only if you select the Enable Retry property. The multiplier that the Secure Agent uses to exponentially increase the wait time between successive retry attempts up to the maximum retry delay time. Default multiplier is 2.0. You can also use fractional values. |
Maximum Retry Delay | Appears only if you select the Enable Retry property. The maximum wait time in seconds that the Secure Agent waits between successive retry attempts. Default is 32 seconds. |
Total Timeout | Appears only if you select the Enable Retry property. The total time duration in seconds that the Secure Agent attempts to retry the connection after which the connection fails. Default is 50 seconds. |
¹ Doesn't apply to mappings in advanced mode. | |