Data Governance User Guide > Linking to Enterprise Metadata > Automate Object Onboarding from Enterprise Data Catalog

Automate Object Onboarding from Enterprise Data Catalog

After you integrate Axon with Enterprise Data Catalog, you can choose to automate the process of onboarding the discovered objects from Enterprise Data Catalog. In the onboarding process, Informatica identifies the critical data elements and automatically creates objects, such as data sets and attributes, and brings them to Axon. This automation eliminates the need for you to manually create content and link Axon objects and Enterprise Data Catalog technical assets. The automation increases efficiency, productivity, and adds value to the business.

You define a glossary and system in Axon. Enterprise Data Catalog is master catalog of technical assets and has a data discovery tool that discovers attributes based on associations between a glossary and field. You can choose to discover the attributes that are related to an Axon glossary term with the help of Enterprise Data Catalog instead of manually searching and linking them. Informatica automates the process of creating data sets and attributes and onboards them into Axon. Axon sends notifications about the discovered objects for a glossary to the glossary stakeholders. You can curate the attributes by accepting or rejecting them. Ensure that the onboarded data sets have assigned stakeholders to curate the attributes.

Note: If objects are assigned to segments in Axon, the automated onboarding of objects are based on the segments that the objects belong to.

Prerequisites

Verify the following prerequisites:

•You have installed Enterprise Data Catalog that runs on Informatica version 10.2.2 or higher.
•You have configured the Enterprise Data Catalog parameters from the Admin panel in Axon. For more information, refer to Configure Access to Enterprise Data Catalog.
•Ensure that the Data Set Type parameter contains the "Data Set" value in the dropdown configuration in the Admin Panel.
•You have created the required systems and glossaries in Axon.
•You have linked an Axon system to at least one Enterprise Data Catalog resource. For more information, refer to Linking Resources with Systems Manually
•Each glossary has a stakeholder to receive onboarding notifications.

Automated Onboarding Process

The automated onboarding process includes identifying the key elements based on the links between a field and glossary and onboarding the automatically created data sets and attributes into Axon.

The following image shows the automated onboarding process flow from Enterprise Data Catalog to Axon:

The image shows the automated onboarding process flow from Enterprise Data Catalog to Axon.

The following process describes the series of events for automatic onboarding of objects into Axon:

1. Create a resource in Enterprise Data Catalog. Optionally, you can enable data domain and similarity discovery options. For more information, refer the topics Creating a Resource and Enable Data Discovery in the Informatica 10.2.2 Catalog Administrator Guide.
2. Create an Axon resource type in Enterprise Data Catalog. Use the Axon scanner to scan the Axon glossaries. For more information, refer the topics Creating a Resource and Axon Resource Type Properties in the Informatica 10.2.2 Catalog Administrator Guide.
3. Create glossaries and systems in Axon. Assign at least one stakeholder to a glossary to receive notifications about onboarding objects into Axon. For more information, see Creating an Object.
4. To automatically onboard discovered objects from Enterprise Data Catalog, make sure that you enable the option to automate onboarding in the Admin panel. For more information, see Configure Access to Enterprise Data Catalog.
5. A background job is scheduled to run daily at a predefined interval. When the job runs, Informatica identifies critical data elements based on the glossary and field linking and automatically creates data sets and attributes. After the job runs, the discovered data sets and attributes are onboarded to Axon. For more information, see Onboarding Scheduler.

The onboarding criteria is based on the following associations between a field and glossary:

- When you link a field to a glossary in Enterprise Data Catalog. For more information, refer the Add Business Title topic in the Informatica 10.2.2 Enterprise Data Catalog User Guide.
- When you link a field to a data domain and the data domain is linked to a glossary in Enterprise Data Catalog. For more information, refer the Column and Field Assets and Data Domain Assets topics in the Informatica 10.2.2 Enterprise Data Catalog User Guide.
- When a field displays recommended glossaries in Enterprise Data Catalog. For more information, refer the Add Business Title topic in the Informatica 10.2.2 Enterprise Data Catalog User Guide.
- When a glossary is linked to a data domain discovery rule in Axon. For more information, see Data Domain Discovery Rules.

For more information, see Linking Glossaries.

6. Axon sends notifications to the glossary stakeholders about the discovered objects. For more information, see Notify Stakeholders.
7. View the discovered data sets and attributes. Curate the attributes by accepting or rejecting them. Ensure that the onboarded data sets have assigned stakeholders with Edit permissions to curate the attributes. For more information, see View Discovered Objects and Curate Discovered Attributes.
8. You can view newly created relationships between attributes in Axon if you choose to accept the lineage recommendations from Enterprise Data Catalog and create links automatically. You can view the attribute links in both inbound and outbound relationship grids. You can filter based on the created date to find the newly created attribute links.

Configure Automatic Onboarding

To automatically onboard discovered objects from Enterprise Data Catalog, you need to configure Enterprise Data Catalog parameters in Axon.

Configure the following Enterprise Data Catalog parameters from the Admin Panel:

Enable Automatic Onboarding: If you choose this option, the objects are automatically onboarded from Enterprise Data Catalog to Axon.
Confidence Score Threshold: The conformance percentage threshold to discover the objects from Enterprise Data Catalog.
Accept Onboarded Objects: If you choose this option, the attributes from Enterprise Data Catalog appear with the Accepted review status in Axon.
Axon Super Admin Email: The email address of the SuperAdmin user responsible for onboarding discovered objects.
Accept Lineage Recommendations and Create Links: Enable if you want to automatically create attribute links based on the lineage recommendations from Enterprise Data Catalog.
Configure Onboarded Attribute Names: Choose to have either the glossary name or Enterprise Data Catalog field name for the onboarded attributes.

You can configure the Enterprise Data Catalog onboarding parameters only after the Axon installation process. For more information on how to configure onboarding parameters, refer to Configure Access to Enterprise Data Catalog.

Onboarding Scheduler

Axon has a predefined scheduled job that runs every days at 2 a.m. After the job runs, the discovered objects are automatically onboarded from Enterprise Data Catalog to Axon.

When the onboarding script runs for the first time, Informatica onboards all the discovered data sets and attributes into Axon. When it runs the next time, the newly discovered data sets and attributes are onboarded.

When the onboarding job runs, the relationship between attributes is also onboarded if you choose to automatically create links from the Admin Panel.

Common Onboarding Conditions

The following conditions apply to an onboarding process:

•The name of the onboarded attribute is same as that of either the glossary name or physical field name based on the configuration in the Admin Panel.
•The name of the onboarded data set is that of the parent name of the field.
•The glossary that is required to automatically create a data set is randomly selected. For example, consider that table EMP_DATA has columns EmpName and EmpID.The column EmpName is linked to glossary "Name" and column "EmpID" is linked to glossary ID. A data set with the name EMP_DATA is automatically created using either glossary "Name" or "ID".
•If a data set exists in Axon, only the attributes are onboarded within the data set.
•If data sets and attributes exist in Axon, the data sets and attributes are not onboarded, but the link is created in Axon.
•If you reject a discovered attribute in Axon, the attribute is not onboarded again.
•If a field in a custom resource does not have a parent in Enterprise Data Catalog, then the attribute corresponding to the field is not onboarded to Axon. This happens because you need a data set to onboard attributes, and without a parent, the data set context is missing.
•If you configure a mandatory custom field for data sets and attributes in Axon, the onboarded data sets and attributes display the mandatory field with the default value that you provide while creating the mandatory custom field. You can change the default value for the mandatory custom field from the Data Set and Attribute objects.
•If you configure a non-mandatory custom field for data sets and attributes in Axon, the onboarded data sets and attributes display the non-mandatory field without any value. You can specify a value from the Data Set and Attribute objects.
•If objects are part of different segments, the objects are not automatically onboarded. For example, consider that glossary G1 is part of segment Seg1 and system S1 is part of segment Seg2. If G1 is connected to physical field C1 of resource R1 and R1 is connected to S1, the attributes are not automatically onboarded because S1 and G1 belong to two different segments.

Objects Onboarding Scenarios

Consider the onboarding scenarios for resources with relational sources, such as tables and columns. The process of automated onboarding of objects vary based on the following types of links between a system and resource:

•A single system is linked to a single resource.
•A single system is linked to multiple resources.
•Multiple systems are linked to a single resource.

In all the scenarios, consider that you choose the glossary names for the onboarded attributes.

Scenario 1. Single System to Single Resource

After you link an Axon system to a single Enterprise Data Catalog resource, the onboarding results can vary based on whether columns of the resource are linked to different glossaries or the same glossary.

Consider that system S1 is linked to resource R1. Resource R1 contains tables T1 and T2. Table T1 contains columns C1, C2, C3, and C5. Table T2 has a column C4.

Multiple Columns Linked to Different Glossaries

In this case, columns C1, C2, C3, and C5 of table T1 are linked to different glossaries G1, G2, G3, and G5 respectively. Column C4 of table T2 is linked to glossary G4.

The following image shows a sample linking of multiple columns from different tables of a single resource to different glossaries:

The image shows a sample linking of multiple columns from different tables of a single resource to the different glossaries.

In this case, the objects are onboarded in the following way:

•A data set is created and onboarded with the same name as of table T1.
•Attributes are created and onboarded with the same names as of glossaries G1, G2, G3, and G5. Attributes G1, G2, G3, and G5 are created within data set T1.
•Another data set is created and onboarded with the same name as of table T2.
•Another attribute is created and onboarded with the same name as of glossary G4. Attribute G4 is created within the data set T2.

The following image shows a sample list of the onboarded objects:

The image shows a sample list of onboarded objects.

Multiple Columns Linked to Same Glossary

In this case, resource R1 is linked to system S1. Consider that all the columns of tables T1 and T2 of the resource R1 are linked to the same glossary G1.

The following image shows a sample linking of multiple columns from different tables of a single resource to the same glossary:

The image shows a sample linking of multiple columns from different tables of a single resource to the same glossary.

In this case, the objects are onboarded in the following way:

•A data set is created and onboarded with the same name as of table T1.
•An attribute is created and onboarded with the same name as of glossary G1. Attribute G1 is created within data set T1.
•Another data set is created and onboarded with the same name as of table T2.
•Another attribute is created and onboarded with the same name as of glossary G1. Attribute G1 is created within the data set T2.

The following image shows a sample list of the onboarded objects:

Scenario 2. Single System to Multiple Resources

After you link an Axon system to multiple Enterprise Data Catalog resources, the onboarding results can vary based on whether columns of the same table from multiple resources are linked to different glossaries or the same glossary, or columns of different tables from multiple resources are linked to the same glossary.

Consider that system S1 is linked to resources R1 and R2.

Multiple Columns from Same Tables Linked to Different Glossaries

In this case, both the resources R1 and R2 contain the same table T1. Table T1 from resource R1 contains columns C1, C2, and C5. Table T1 from resource R2 contains columns C3 and C4. Columns C1, C2, and C5 are linked to different glossaries G1, G2, and G5 respectively. Columns C3 and C4 are linked to different glossaries G3 and G4 respectively.

The following image shows a sample linking of multiple columns of the same table from multiple resources to different glossaries:

The image shows a sample linking of multiple columns of the same table from multiple resources to different glossaries.

In this case, the objects are onboarded in the following way:

•A data set is created and onboarded with the same name as of table T1 for resources R1 and R2.
•Within data set T1, the attributes are created and onboarded with the same names as of glossaries G1, G2, G3, G4, and G5.

The following image shows a sample list of the onboarded objects:

Multiple Columns from Same Tables Linked to Same Glossaries

In this case, both the resources R1 and R2 contain the same table T1. Table T1 from resources R1 and R2 contains columns C1 and C2. Column C1 from both the resources R1 and R2 is linked to the same glossary G1. Column C2 from both the resources R1 and R2 is linked to the same glossary G2.

The following image shows a sample linking of multiple columns in the same table from different resources to the same glossary:

The image shows a sample linking of multiple columns in the same table from different resources to the same glossary.

In this case, the objects are onboarded in the following way:

•A data set is created and onboarded with the same name as of table T1 for resources R1 and R2.
•Within data set T1, the attributes are created and onboarded with the same names as of glossaries G1 and G2.

The following image shows a sample list of the onboarded objects:

Multiple Columns from Different Tables linked to Same Glossary

In this case, both the resources R1 and R2 contain tables T1 and T2. Table T1 from resources R1 and R2 contains columns C1, C3, and C5. Table T2 from resources R1 and R2 contains columns C2 and C4. All the columns are linked to the same glossary G1.

The following image shows a sample linking of multiple columns in different tables from different resources to the same glossary:

The image shows a sample linking of multiple columns in different tables from different resources to the same glossary.

In this case, the objects are onboarded in the following way:

•A data set is created and onboarded with the same name as of table T1 for resources R1 and R2.
•An attribute is created and onboarded with the same name as of glossary G1. Attribute G1 is created within data set T1.
•Another data set is created and onboarded with the same name as of table T2 for resources R1 and R2.
•An attributes is created and onboarded with the same name as of glossary G1. Attribute G1 is created within the data set T2.

The following image shows a sample list of the onboarded objects:

Scenario 3. Multiple Systems to Single Resource

After you link multiple Axon systems to a single Enterprise Data Catalog resource, the onboarding results can vary based on whether columns of the resource are linked to different glossaries or same glossary.

Consider that systems S1 and S2 are linked to resource R1. Resource R1 contains tables T1 and T2. Table T1 contains columns C1, C2, and C3. Table T2 contains columns C4 and C5.

Multiple Columns Linked to Different Glossaries

In this case, columns C1, C2, and C3 of table T1 are linked to different glossaries G1, G2, and G3 respectively. Columns C4 and C5 of table T2 are linked to glossaries G4 and G5 respectively.

The following image shows a sample linking of multiple columns from different tables of a resource to different glossaries:

The image shows a sample linking of multiple columns from different tables of a resource to different glossaries.

If multiple systems are linked to a single resource and multiple columns from different tables of the resource are linked to different glossaries, the data sets and attributes are created and onboarded to each system. In this case, the objects are onboarded in the following way:

•A data set is created and onboarded with the same name as of table T1 in systems S1 and S2.
•Attributes are created and onboarded with the same names as of glossaries G1, G2, and G3. Attributes G1, G2, and G3 are created within data set T1 in systems S1 and S2.
•Another data set is created and onboarded with the same name as of table T2 in systems S1 and S2.
•Attributes are created and onboarded with the same names as of glossaries G4 and G5. Attributes G4 and G5 are created within data set T2 in systems S1 and S2.

The following image shows a sample list of the onboarded objects:

Multiple Columns Linked to Same Glossary

In this case, all columns from tables T1 and T2 are linked to the same glossary G1.

The following image shows a sample linking of multiple columns from different tables to the same glossary:

The image shows a sample linking of multiple columns from different tables to the same glossary.

If multiple systems are linked to a single resource and multiple columns from different tables of the resource are linked to the same glossary, data sets along with attributes are created and onboarded to each system. In this case, the objects are onboarded in the following way:

•A data set is created and onboarded with the same name as of table T1 in systems S1 and S2.
•Another data set is created and onboarded with the same name as of table T2 in systems S1 and S2.
•An attribute is created and onboarded with the same name as of glossary G1 in both data sets T1 and T2 for systems S1 and S2.

The following image shows a sample list of the onboarded objects:

Notify Stakeholders

Axon sends notifications about the onboarded objects to all the stakeholders of a glossary. You can click the bell icon (

) and view the onboarding and relationship recommendation notifications in the Catalog tab.

The following image shows a sample list of discovered objects that you can view in the notifications:

The notifications show the Catalog tab with the discovered data sets and attributes.

If the notifications include links to the glossaries for which the data sets and attributes were discovered, you can click the glossary name to navigate to the glossary object and view the discovered data sets and attributes. For example, 11 new attributes and 8 data sets were discovered for Glossary FNAME. When you click the FNAME glossary, you can view the discovered data sets and attributes under the Data tab.

Axon displays a notification for each glossary. After the next onboarding scheduler run, the discovered objects are updated for each glossary.

View Discovered Objects

In Axon, you can view data sets and attributes that are discovered and onboarded from Enterprise Data Catalog.

You can view the discovered data sets from Data > Data Sets view of a Glossary object. You can view the discovered attributes from the following areas in the Axon interface:

•The Data > Data Attributes view of a Glossary object
•The Attributes facet view from the Unison search
•The Attributes tab of a Data Set object
•The Data > Data Attributes view of a System object

The following image shows the Data > Data Attributes view of a Glossary object:

The EMPLOYEE_NUM golossary shows the discovered attributes in the Data | Discovery column.

You can identify the attributes that are populated from Enterprise Data Catalog based on the following properties:

•The onboarded name of the attribute is same as that of the glossary name or physical field name.
•The attribute definition displays the same definition of the glossary.
•The Origin field has the "Enterprise Catalog" value.
•The data type is the same as that of the Enterprise Data Catalog field.
•The data length is same as that of the data type length in Enterprise Data Catalog.
•The data set name is the same name as of the parent of the field in Enterprise Data Catalog.
•The Physical Fields column shows the names in orange color. You can click the hyperlink on the linked physical field to view the details of the field, such as Last Updated, Field Type, Resource, Table, Schema, Data Domains, and Data Domain Groups.
•The Review Status column of the discovered attributes show "Accepted" or "Discovered" status. If you choose to accept onboarded objects from the Admin Panel and the Confidence Score is 100%, the attributes from Enterprise Data Catalog appear with the "Accepted" review status in Axon. If you do not choose to accept onboarded objects, the review status appears as "Discovered." The "Accepted" review status appears only when a field is directly linked to a glossary. You cannot see the "Accepted" review status when you use recommended glossaries and data domain links.
•The Confidence Score column shows the percentage of conformance of the discovered attributes.

Curate Discovered Attributes

To curate the discovered attributes, ensure that the onboarded data sets have assigned stakeholders with Edit permissions. You can curate the discovered attributes from the following areas in the Axon interface:

Objects View: If you want to curate the attributes within a data set, you must have Edit permissions on the data set. You can view and curate the discovered attributes from the following objects:; You can choose whether you want to accept or reject the discovered attributes. Select the attributes with the "Discovered" review status, click the Actions menu and choose Accept or Reject. You cannot undo the changes after you accept or reject the discovered attributes.; If you do not have the required permissions to curate the attributes, the Actions menu does not appear for the Attributes tab of a Data Set object. In System and Glossary objects, you can view the Actions menu even if you do not have the required permissions to curate the attributes. But, if you try to curate the discovered attributes, an error occurs.; In a Glossary object, you cannot curate the attributes that are discovered for the child glossaries. To curate the child glossary attributes, navigate to the child glossary objects.
Bulk Update: Use the Bulk Update option from the Unison search to curate the discovered attributes.; The following image shows the Attributes facet with a sample list of discovered objects that you can curate from the Unison search:

In the Attributes facet, select the attributes with the "Discovered" review status, and click Bulk Update. In the Bulk Update Items section, you can see the selected attributes. In the Definition section, you can view the fields that you can update for the attributes. To curate the attributes, select Accept or Reject in the Review Status field.

If you choose to reject a discovered attribute, Axon permanently deletes the discovered attribute without changing the review status to "Rejected". You can view the audit history to identify who curated the attributes and when the attributes were curated.