You can use the Google BigQuery catalog source to extract metadata from a Google BigQuery source system.
Google BigQuery is a serverless data warehouse that performs scalable analysis over petabytes of data. Use a Google BigQuery catalog source to collect metadata from the assets in a Google BigQuery source system.
Extracted metadata
You can use the Google BigQuery catalog source to extract metadata from a Google BigQuery source system.
Metadata Command Center extracts the following metadata from a Google BigQuery source system:
•Database
•Schema
•Table
•External Table
•Column
•View
•View Column
•Materialized View
Note:
Objects of the Materialized View type appear as View in
Data Governance and Catalog
.
•Nested Field
•Stored procedure
•SQL User-defined functions (UDF)
You can extract metadata from columns with the following complex data types, along with their nested fields, from Google BigQuery source systems:
•Array
•Range
•Struct
Data Governance and Catalog displays columns with their complex data types and nested fields, including data types of those nested fields. The nested fields within Array and Range column data types appear under element.
Note:
Each time
Metadata Command Center
runs a scan on a
Google BigQuery
catalog source,
Data Governance and Catalog
displays the extracted objects along with the objects from the previous scans. This means that if you modify the filters in
Metadata Command Center
and run a scan,
Data Governance and Catalog
does not replace the objects from the previous scan. Instead,
Data Governance and Catalog
adds the newly extracted objects to the existing scanned objects. To see the results of only the latest scan, choose Delete as the Metadata Change Option before you run the scan.
Data profiling for Google BigQuery objects
Configure data profiling to run profiles on the metadata extracted from a Google BigQuery source system. You can run data profiles on the following objects:
•Table
•External Table
•Partitioned Table
•Views
•Materialized Views
The data profiling task runs profiles on the following data types:
•BOOLEAN
•DATE
•DATETIME
•FLOAT
•INTEGER
•NUMERIC
•STRING
•TIME
•TIMESTAMP
Compatible connectors
Before you configure a Google BigQuery catalog source, you must connect to the Google BigQuery source system.
Use the Google BigQuery V2 connector to connect to the Google BigQuery source system.
For information about configuring a connection, see Connections.