View Automated Lineage from Enterprise Data Catalog
In Axon, you can choose to automatically create links between attributes based on the recommendations from Enterprise Data Catalog. You need not manually review the relationship recommendations and curate attributes for each data set.
A physical field F1 from table T1 is directly linked to physical field F2 from table T2. The physical field F1 is linked to attribute A1, and the physical field F2 is linked to attribute A2. Enterprise Data Catalog provides lineage recommendation between attributes A1 and A2. If you choose to accept lineage recommendations and create links, you can see that the link between the attributes A1 and A2 are automatically created in Axon after automated onboarding of attributes. A daily job runs to discover additional relationships that can be onboarded from Enterprise Data Catalog. You can see the linked attribute A1 in the Relationships tab of the data set that contains attribute A2. The Type column displays the value "Sourced From" to indicate that the related attribute relationship is created based on Enterprise Data Catalog recommendations. You can view the attribute links in both inbound and outbound relationship grids. To find the newly created attribute links, you can filter based on the Created Date column that is hidden by default.
To accept the lineage recommendations, you need to enable the Accept Lineage Recommendations and Create Links option from the Admin Panel. If the fields are linked to attributes and the fields are linked with each other in Enterprise Data Catalog, the links between attributes are automatically created. After the link is created between attributes, Axon notifies the data set stakeholders only with the Stewardship Role type. The notification appears on the Catalog tab of the notification center that you can access from the toolbar. The notification contains information about all the attribute links that are created. If you delete an automatically created relationship, the same link is not created again in the next onboarding process.
The onboarded links between the attributes is unique. When the onboarding script runs, duplicate attribute links are not onboarded. The uniqueness of the Attribute X Attribute links is not validate based on the Sourcing Type. If you delete a link between attributes, the link is not onboarded again when the onboarding script runs. You can manually activate the deleted link when you click Show Recommendations from the recommended attributes.
If you do not select the Accept Lineage Recommendations and Create Links option from the Admin Panel, Axon sends a notification through the interface. The notification contains information about the discovery of additional linked attributes and the link that directs you to the Relationship tab of a Data Set object. Click Show Recommendations to view the recommended attributes. You can accept or reject recommendations for each data set.
Example
The following image shows a sample linking between Enterprise Data Catalog physical fields and Axon attributes:
Consider that there are Employee, Finance, Admin, and Org tables that contain salary information in Enterprise Data Catalog. The physical fields of these tables are linked based on the salary information. Emp_Sal and Admin_Sal physical fields are linked to Fin_Sal physical field, which is linked to Org_Sal physical field. The linking between Enterprise Data Catalog physical fields and Axon attributes is as follows:
- •Emp_Sal physical field to Emp_attr1 attribute
- •Admin_Sal physical field to Admin_attr2 attribute
- •Fin_Sal physical field to Fin_attr3 attribute
- •Org_Sal physical field to Org_attr4 attribute
If you choose to accept the lineage recommendations from Enterprise Data Catalog, the links between attributes are automatically created in Axon based on the linked fields in Enterprise Data Catalog. For attribute Fin_attr3, you can see the linked attributes Emp_attr1 and Admin_attr2 in the Relationships > Inbound Relationships view of the data set. You can see the linked attribute Org_attr4 in the Relationships > Outbound Relationships view.
Lineage Onboarding Scenarios
Automated lineage between attributes is created in Axon based on the following conditions:
- •Physical fields must be directly connected to each other in Enterprise Data Catalog.
- •Attributes must be associated with the fields.
The following image shows the sample automated lineage onboarding process:
Axon supports automated lineage creation for single hop or direct relationships between the fields. In the example, Column 1 is directly linked to Column 2, Column 1 is linked to Attribute1, and Column 2 is linked to Attribute2. When the onboarding job runs, automated lineage is created between Attribute1 and Attribute2. Column 2 is linked to Column 5 through multiple hops and the lineage between Attribute1 and Attribute3 is not created automatically.
The attribute links are automatically created in the following scenarios:
- •Consider that physical fields F1 and F2 are linked to each other in Enterprise Data Catalog, and the physical field F1 is linked to attribute A1 and F2 is linked to A2. If you choose to accept the lineage recommendations from the Admin Panel, the attributes A1 and A2 are automatically linked in Axon.
- •Consider that physical fields F1 and F2 are linked to each other in Enterprise Data Catalog, and the physical field F1 is linked to glossary G1 and F2 is linked to G2. If you choose to automatically onboard attributes and accept the lineage recommendations from the Admin Panel, the attributes with names G1 and G2 are automatically onboarded, and the relationship between attributes G1 and G2 is created in Axon.
The attribute links are not automatically created in the following scenarios:
- •Consider that physical fields F1 and F2 are linked to each other in Enterprise Data Catalog, and the physical field F1 is linked to attribute A1 and F2 is not linked to any attribute. If you choose to accept the lineage recommendations from the Admin Panel, you cannot see any lineage recommendation for attribute A1 in Axon.
- •Consider that physical fields F1 and F2 are linked to each other in Enterprise Data Catalog, and the physical field F1 is linked to glossary G1 and F2 is linked to G2. If you do not choose to automatically onboard attributes and choose to accept the lineage recommendations from the Admin Panel, you cannot see any lineage recommendation in Axon.
