Data lineage is a visual representation of the flow of data across the systems in your organization. Lineage depicts how the data flows from the system of its origin to the system of its destination.
Data lineage views are available for technical assets in the catalog source. You can view lineage at the catalog source, or data set level.
The lineage at the catalog source level shows how data flows from one catalog source to another. The lineage at the data set level shows how other technical assets such as files or tables contribute to the selected asset.
If linking catalog sources is available for your catalog source, you can use Metadata Command Center to generate data lineage based on rules or by generating automated lineage with CLAIRE. You can choose source and target catalog sources and objects to link and generate lineage.
To determine whether linking catalog sources is available for your catalog source, navigate to the Configuration tab of the Link Catalog Sources page. The catalog source must appear in the list of source and target catalog sources.
The catalog source level shows how data flows from one catalog source to another with the lineage data from the data set level.
To view data lineage at the catalog source level, open a technical asset, click the Lineage tab, and then verify that the level is set to Catalog Source Level.
The following image shows how data flows between the GBQ_Vertex_10 and the Google_Vertex_Basic_10 catalog sources after connection assignment:
After connection assignment, the referenced object icons change to specific object icons.
View lineage at the data set level
The data set level displays individual sets of data in the data flow.
To view lineage at the data set level, open a technical asset, click the Lineage tab, and then verify that the level is set to Data Set Level.
The following image shows the data set level lineage where the 'Version 1' AI core model target reference data set gets data from the 'customer_records.csv' source reference data set before connection assignment:
The following image shows the data set level lineage where the 'Version 1' AI core model target data set gets data from the 'customer_records.csv' file after connection assignment:
After connection assignment, the referenced object icons change to specific object icons.
