You can view the lineage of an individual field in a mapping. A field's lineage shows how the field is created, renamed, mapped, or changed within each transformation in the data pipeline.
You might want to view a field's lineage to help you troubleshoot a mapping with incorrect target data. For example, if fields are missing in the target, you can trace each field's lineage from its source to find where the field is excluded.
When you view the lineage for a field, Data Integration highlights the field's path on the mapping canvas. Depending on the transformation, it also highlights the field on the Incoming Fields and Output Fields tabs of the selected transformation. To see how the field moves through the highlighted data flow, select another highlighted transformation.
The following image shows the lineage of the incoming field OrderAmount in the Target transformation:
Data Integration displays the full field lineage. A field's lineage begins at its source and ends at its target. If you add a field midstream, the transformation where you add it is the source. For example, lineage for an expression field begins at the Expression transformation where the field was added. A field's lineage ends at the transformation that does not output the field. For example, if an incoming field is not mapped to a normalized field in a Normalizer transformation, the field lineage stops at the Normalizer transformation.
A field's lineage can depend on the transformation you view the lineage from. For example, when you view the lineage for an incoming field used in the lookup condition of a Lookup transformation, the downstream lineage includes all fields returned by the lookup. If you view the lineage of the same field from a downstream transformation, the upstream lineage does not include the fields returned by the lookup.
You can view the lineage of fields that pass through or are changed by any transformation in the pipeline except the following transformations:
•Hierarchy Builder
•Hierarchy Parser
•Hierarchy Processor
•Java
•Machine Learning
•Mapplet transformations that reference an SAP or PowerCenter mapplet
•Python
•SQL transformations that process an SQL query
•Unconnected transformations
If the mapping contains hierarchical data, you can view the field lineage for primitive fields in the data flow. You can also view the lineage for primitive fields that are output from a hierarchical expression. The upstream lineage includes the hierarchical field but does not include its nested elements.
Lineage for renamed fields
When you view the lineage for a field that was renamed, the upstream lineage includes instances of the field with the original name and the new name.
For example, you have a mapping that joins customer and order tables that have some common field names. In the Joiner transformation, you resolve the field name conflict by prefixing fields in the customer table with cust_ and then configure the transformation to join data based on the customer ID.
In the Target transformation, you want to view the lineage for the cust_Customer_Region field. Data Integration includes the Customer_Region source field in the lineage as shown in the following image:
Lineage for fields containing expressions
You can view the lineage for expression fields, aggregate fields, and for advanced filters that contain incoming fields.
When the expression contains incoming fields, the upstream lineage includes the fields used in the expression.
For example, you configure the expression field Full_Name to concatenate the Firstname and Lastname source fields before loading the data to the target. When you view the field lineage of the expression field Full_Name, the upstream lineage includes the Firstname and Lastname fields.
If the expression doesn't contain incoming fields, the lineage includes only downstream transformations. Lineage also doesn't include variable fields used in the expression.
You can view the upstream field lineage for an advanced filter condition if the expression contains one or more incoming fields. Data Integration doesn't include the downstream lineage because Filter transformations do not include output fields. If you want to view the full lineage for a field included in a filter condition, view the lineage from an upstream or downstream transformation.
Lineage for mapped fields
When you view the lineage for a field that is mapped to another field, the lineage includes both the incoming and output fields.
For example, you have a Normalizer transformation that normalizes incoming quarterly sales data. You create the normalized field Y with an occurs value of 4.
In the Normalizer transformation, you configure the following field mapping:
Incoming field
Normalized field
Q1
Y_1
Q2
Y_2
Q3
Y_3
Q4
Y_4
The normalized field Y is then passed to the downstream transformation.
On the Incoming Fields tab of the downstream transformation, you get the lineage for field Y. Because fields Q1, Q2, Q3, and Q4 are mapped to occurrences of Y, they are included in the upstream lineage of Y.
Mapplet field lineage
You can view the lineage of fields that pass through a Mapplet transformation when the transformation references a mapplet that you created in Data Integration.
If the field has at least one lineage path through the mapplet, Data Integration highlights the lineage on the Incoming Fields and Output Fields tabs of the Mapplet transformation. You cannot drill down to the mapplet to view field lineage. If you want to see how the field is transformed within the mapplet, open the mapplet and view the field lineage in the Mapplet Designer.
If the mapplet was changed, synchronize the mapplet before viewing field lineage in the Mapping Designer. Synchronizing the mapplet ensures that you get the most up-to-date lineage.
If the Mapplet transformation references an SAP or PowerCenter mapplet, field lineage stops at the Mapplet transformation.
Lineage for lookup fields
When you view the lineage of a field used in the lookup condition of a Lookup transformation, the downstream lineage includes all fields returned by the lookup.
For example, you have a source table with customer data and you want to augment the data with data from an orders table before loading the data to a new target table. You configure the Lookup transformation to return fields when the source field Src_CustomerID equals the lookup field Customer_ID. On the Incoming Fields tab of the Lookup transformation, you view the lineage for the Src_CustomerID field. The downstream lineage includes all fields returned by the lookup.
The following image shows the lineage for the Src_CustomerID field on the Returned Fields tab:
Lineage for transformations that read Data Quality assets
A transformation in a mapping may read an asset that originates in Data Quality.
Field lineage for transformations that read Data Quality assets behaves in the same manner as other transformations in Data Integration, with the following exceptions:
Rule Specification transformations
When you view the field lineage for a mapped or unmapped output field from a Rule Specification transformation on the target transformation, the field lineage stops at the Rule Specification transformation.
Cleanse transformations
When you view the field lineage for a merged output field from a Cleanse transformation on the target transformation, the field lineage stops at the Cleanse transformation.
Deduplicate transformations
When you view the field lineage for a metadata field from a Deduplicate transformation on the target transformation, the field lineage stops at the Deduplicate transformation.
The following image shows the lineage of the output field in the target transformation:
Viewing field lineage
Depending on the transformation, you can view field lineage from the Incoming Fields or the transformation-specific properties tabs.
Before you view a field's lineage, resolve any parameters in the mapping that include the field you are investigating. Data Integration does not include parameters in field lineage.
1In the row that contains the field that you want to investigate, click Link Path.
Data Integration highlights the field's path on the mapping canvas and highlights the field on the properties tab.
2Click an upstream or downstream transformation to see how it transforms the field.
Data Integration automatically opens the relevant properties tab and highlights the field lineage. Depending on the transformation, Data Integration highlights instances of the field on the Output Fields and Incoming Fields tabs.
3Select another transformation in the field's path and view instances of the field in the Properties panel.
4When you are finished, click Clear in the upper-left corner of the mapping canvas.
