Amazon SageMaker AI Sources > Create catalog sources in Metadata Command Center > Step 2. Configure capabilities

Step 2. Configure capabilities

When you configure the Amazon SageMaker AI catalog source, you define the settings for the metadata extraction capability and other optional capabilities.

The metadata extraction capability extracts source metadata from external source systems. You can also configure other capabilities that the catalog source includes.

You can save the catalog source configuration at any point after you enter the connection information. After you save the catalog source, you can choose to run the catalog source job. To run the job once, click Run. To run metadata extraction and other capabilities on a recurring schedule, configure schedules on the Schedule tab.

Configure metadata extraction

When you configure the Amazon SageMaker AI catalog source, you choose a runtime environment and enter configuration parameters for metadata extraction.

1In the Connection and Runtime area, choose a serverless runtime environment or the Secure Agent group where you want to run catalog source jobs.

Note:

Serverless runtime environment options are available if the catalog source works with a serverless runtime environment.

2Choose to retain, delete, or deprecate objects that are deleted from the source system in the catalog with the Metadata Change Option.

- Retain. Retains objects that are deleted from the source system in the catalog. If you update or add a filter, the catalog retains objects extracted from the previous job and extracts additional objects that match the current filter. Objects deleted from the source system are not deleted from the catalog. Enrichments added on deleted objects and relationships are retained.
- Delete. Deletes metadata from the catalog based on objects deleted from the source system and changes you make to the filter. Enrichments added on deleted objects and relationships are also permanently lost. Objects renamed in the source system are removed and recreated in the catalog.
- Deprecate. When objects are deleted from the source system, the catalog changes the lifecycle of the imported objects to Obsolete. The lifecycle of objects also changes to Obsolete when you add exclusion filters or reduce the scope so that objects extracted in earlier jobs are no longer included. For example, the first job includes the sales schema. The second job also includes the sales schema, but excludes the sales.transactions table. In this case, the second job marks the sales.transactions object as Obsolete, even though the first job includes it. This does not impact enrichments applied to deprecated objects and relationships. Objects renamed in the source system are removed and recreated in the catalog. When you run the catalog source job again for other capabilities such as data classification, relationship discovery, or glossary association, the job doesn't consider obsolete objects. Obsolete objects remain in the catalog until they are purged when you run a Purge Obsolete Objects job on the Explore page.

Note:

You can also change the configured metadata change option when you run a catalog source.

3In the Filters area, define one or more filter conditions to apply for metadata extraction.

aSelect Yes to view filter options.
bFrom the Include/Exclude list, choose to include or exclude metadata based on the filter parameters.
cPerform one of the following steps:

▪ Enter the filter values.
▪ In the filter value field, click the Search button and select an object from a list of objects available in the source system.

The Object type field updates based on the selected object.

If you select an object type and then click the Search button, the list of objects includes all object types, but you can only select objects that match the selected object type.

You can edit the filter value after you select an object from the list.

Note:

You can only search for object types that work with the search functionality. If you don't see the Search button for the selected object, enter the object path as the filter value.

Note:

If the object metadata is available in Data Governance and Catalog, a check mark appears next to the object.

Note:

To select an object, you need to have permissions on the connection to the source system.

The filters contain options to include or exclude metadata based on specific object types.

Filters can contain a question mark to represent a single character and an asterisk wildcard to represent multiple characters. Filters are case-sensitive.

When you apply the filter, it also includes metadata from all endpoints that the AI model is a part of.

Examples:

▪ Canvas1? includes or excludes all AI models with names that start with ‘Canvas1’ followed by a single character.
▪ Model_AI* includes or excludes all AI models with names that start with ‘Model_AI’.

dOptionally, to define an additional filter with an AND condition, click the Add icon.

4Optional. In the Configuration Parameters area, enter additional settings.

The following table describes the property that you enter for additional settings:

Note:

The Additional Settings section appears when you click Show Advanced.

Property	Description
Expert Parameters	Enter additional configuration options to be passed at runtime. Required if you need to troubleshoot the catalog source job. Caution: Use expert parameters when it is recommended by Informatica Global Customer Support.

5Configure additional capabilities for the catalog source by clicking on the tabs.

Configure Automation

Enable lineage-based propagation to receive enrichments propagated from catalog sources connected through lineage.

Enable Lineage-based Propagation on the Autonomous Enrichment tab of the Configure page before you configure this catalog source to receive enrichments. For more information, see Administration.

With the lineage-based propagation settings on this tab, you configure how this catalog source receives enrichments propagated from connected catalog sources. Enrichments propagate only to assets that don't already have enrichments.

1Click the Automation tab.

2In the Autonomous Enrichment section, select Enable Lineage-based Propagation.

The lineage-based propagation settings appear.

3Select the enrichment types to receive from connected catalog sources.

You can select Classification, Description, Business Name, and Glossary Term.

4Choose how to apply received enrichments.

- Select Auto-accepted to apply received enrichments automatically.
- Select Recommended to list them as suggestions that you can curate in Data Governance and Catalog.

5Choose whether to receive enrichments only from certified assets.

Assets are certified in Data Governance and Catalog. This option is selected by default. If you clear it, this catalog source also receives enrichments from assets that are not certified, which might be less reliable.

6Choose where this catalog source receives enrichments from.

- Select Catalog Source to receive enrichments from a specific catalog source.
- Select Nearest Neighbor to receive enrichments from the closest connected catalog source based on lineage.

7If you select Catalog Source, select a catalog source from the list.

Ensure that the catalog source you select is within 15 lineage hops. Enrichments don't propagate from a catalog source beyond that distance.

8If you select Nearest Neighbor, configure the lineage settings:

aSet Lineage Direction to search upstream, downstream, or in both directions.

Upstream catalog sources feed data into this catalog source. Downstream catalog sources receive data from this catalog source. Select both directions to search upstream and downstream at the same time.

bSet Number of Hops to define how far along lineage to search.

The search stops at the first connected catalog source that has enrichments, up to a maximum of 15 hops.

9Click Save.

When you save, the catalog source begins receiving enrichments created from that point onward. Saving doesn't retrieve enrichments created prior to this date and time. To receive such enrichments, run the catalog source job manually and include the Autonomous Enrichment capability.