To help your organization manage AI technologies and the data used by them responsibly and effectively, Data Governance and Catalog introduces the following enhancements:
New asset types
The October release introduces the following types of assets to Data Governance and Catalog:
- AI Use Case. An AI Use Case asset represents an initiative that uses AI to solve a specific problem or produce measurable outcomes.
- AI Agent. An AI Agent asset represents an AI-powered autonomous entity that uses a model and a set of tools to make decisions and take actions toward a goal.
- Questionnaire Template. A Questionnaire Template asset represents a set of questions that gathers information for a governance assessment, such as an impact, risk, or intake review. You can associate the questionnaire template with an AI Use Case that you want to assess.
- Risk Template. A Risk Template asset helps you document details of a potential risk in an enterprise initiative. You can associate a risk template with an AI Use Case, and associate one or more mitigation plans with the risk to address it.
- Mitigation Plan Template. A Mitigation Plan Template asset helps you document the plan of action for addressing a risk, and track its progress.
You can create the assets individually in Data Governance and Catalog interface or use bulk import templates to create multiple assets at once.
On the AI Inventory and Risk and Evaluation tabs of the Browse page, you can discover the new assets. Additionally, the AI Inventory tab contains the AI Model assets and AI System assets.
In Metadata Command Center, your organization administrator can customize these assets and enforce workflows for creating and updating them, as well as for associating them with an AI Use Case.
Orchestration flow
This release introduces a new lineage type that is unique to AI Agent assets, orchestration flow. On the Lineage tab of an AI Agent asset, you can view a visual representation of the tools, data, and other AI assets that are associated with the AI agent.
For more information about business assets, see Business Assets.
Curation Center
Data Governance and Catalog introduces the Curation Center, a single page where you can review and act on all recommendations for the assets that you're responsible for, without the need to hunt through individual assets one at a time. Curation Center shows recommendations for glossary, business names, business descriptions, and classifications from all sources, including CLAIRE. You can curate recommendations and use Gen AI to generate business names and descriptions for assets.
You can use the following features of the Curation Center:
•View the summary of your recommendations from all enrichment sources at a glance. Summary charts show the number of assets that have pending, accepted, declined, or no recommendations.
•Act on recommendations from the new lineage propagation enrichment source. See Enrichment propagation based on lineage.
•Filter assets using various filter criteria to find recommendations you want to review.
•Review and act on one or more recommendations at once with full context.
•Use the Advanced Edit page to act on recommendations in bulk and edit them inline.
You can access the Curation Center page directly, or navigate to it from one of the following locations:
•The CLAIRE Activity widget
•In-app and email notifications you receive for recommendations in a catalog source
•The Assets with Recommendations widget on the catalog source asset page
For more information about Curation Center, see Curation Center.
Enhancements to data lineage
This release includes the following enhancements to data lineage:
View control flows
Show or hide control flow lines on an asset's lineage page. An asset's control flow reveals objects that influence how your data moves in the catalog. These lines show join conditions and expressions that apply to the transformation of data and appear as flow lines on the lineage interface.
The following image shows the control flow options for a lineage:
Open a lineage page that shows different transformation details for an asset in a lineage. On the lineage interface, click Open Transformation Details to view how your data moves from source to target. Detailed lineage for an asset show its data mapping, ETL pipeline, control flow, data type, data input and output, and applied SQL expressions.
The following image shows a data element's focused path and the asset's transformation details:
View an AI-generated explanation of the detailed lineage based on the catalog source metadata. Click Generate Summary when you view the detailed lineage of an asset to generate the explanation.
Manage additional configurations for a lineage view
Adjust the depth and direction of a lineage, and define the assets you want to see from an asset hierarchy. You can set the scope of the lineage view to a number of hops, and you can choose the lineage direction from the seed asset. You can also show or hide schemas in a lineage based on the data set parent name.
The following image shows Scope in Manage Configuration and the scope overlay on the toolbar:
Assign lineage views
As an administrator in Data Governance and Catalog, you can assign lineage views to user roles in your organization. You can also set a lineage view as the default view that a user sees when they land on the Lineage tab of an asset page.
Performance enhancement
For organizations that run Data Governance and Catalog on Neo4j, you can load additional hops when you expand the depth of the lineage for upstream and downstream assets.
Lineage across spaces for SAP Datasphere
You can now extract and view lineage across multiple spaces within the SAP Datasphere source systems.
The SAP Datasphere catalog source extracts lineage across spaces for the following object types:
This release includes the following enhancements to data profiling:
Adding passive mapplets as rules
You can add passive mapplets as rules to a profiling task to calculate advanced column statistics on the output ports of the mapplets.
The mapplets can contain the following passive transformations:
- Data Masking
- Expression
Profiling business data sets with source mapplets
You can link a business data set with a source mapplet and then create a data profile for the business data set. This allows you to run a profiling task and view advanced statistics for columns from multiple data sets.
Customize the Create Asset page for business assets
In Metadata Command Center, you can customize what a user sees on the Create page for business assets. You can customize an asset's create page by selecting the attributes that users fill when creating the asset. It also determines what a user views on an asset's Overview tab.
You can perform the following tasks to customize the create page for an asset:
•Design multiple versions of the Create page and assign them to different users, user groups, and user roles.
•Organize attribute panel and even remove optional attributes from the page.
•After you publish a custom create page, you can hide the visibility of default asset creation and preview pages.
The following image shows the New Asset page for an AI Model asset in Data Governance and Catalog:
This release includes the following enhancements to bulk import:
Asset Group management through bulk import
You can assign and update asset groups for multiple business and technical assets in a single operation using import files. In import and export template files, use the Asset Group column to associate asset groups with assets.
Manage recommendations for Glossaries and Data Classifications
You can remove glossary and data classification associations from multiple assets using import files. You can remove an association regardless of its current state. Enter the glossary or classification name or reference ID in the Glossaries: Remove and Data Classifications: Remove column.
For more information about curating assets through bulk import, see Curate assets in bulk.
Manage Policy to Data Classification relationships
The export and import files for Policy assets include the Classifications column. When you import assets,you can use the column to create associations between a Policy and Data Element or Entity classifications. When you export assets, existing associations between a Policy and Data Classifications appear in the sheets for Policy.
Export the technical data elements of a business data set
When you export a business data set that includes child assets, the exported file includes technical data elements. In the export file, the assets appear in a separate sheet that includes the data elements contained within each business data set. The sheet also contains an Operation column so that you can add or remove those relationships when you re-import the file.
Enhancements to stakeholder propagation for assets
This release includes the following enhancements to stakeholder propagation for assets:
Control propagation on assets
You can control the propagation of an asset's stakeholders to child assets in the hierarchy. With the Propagate Stakeholders and Append Stakeholders options, you can set how you want the stakeholders of a parent asset to propagate to its child assets. You can also configure these options to remove inherited stakeholders from child assets.
For technical assets, propagation options appear when an administrator enables Assign Stakeholders for a catalog source in Metadata Command Center. In Data Governance and Catalog, the options appear at the top most asset in a technical hierarchy. For business assets, you can set stakeholder propagation from asset pages in Data Governance and Catalog and during asset creation.
In Data Governance and Catalog, the Stakeholder tab of an asset pages shows different propagation details. The header of the Stakeholder tab shows if the propagation and append status is enabled or disabled. In the Stakeholder grid, the Stakeholder Source column shows Inherited if a stakeholder is inherited from a parent asset and Assigned indicates if the stakeholder is assigned to the asset. You can also view if a parent asset is propagating and appending on the asset in the Edit Stakeholders dialog box.
The following image shows the Modify Stakeholders dialog box with details of a parent asset's propagation on the asset and the Stakeholder Source column for the asset:
Propagate stakeholders through bulk import
When you export assets and re-import them, you can set the Propagate and Append options for an asset's stakeholders. In the Propagate Stakeholders and Append Stakeholders columns, select Disabled or Enabled to configure stakeholder propagation for an asset.
This release includes the following enhancements to governing unstructured data:
Glossary association
You can associate glossary terms with unstructured files to relate them to your business vocabulary. Associate a metric or a business term manually, or use the bulk import and export templates to manage associations.
Data element classification
Data Governance and Catalog uses the name and description of a data element classification to classify unstructured files with greater accuracy. You can use custom data element classifications to classify unstructured files.
Sensitivity
Data Governance and Catalog automatically applies sensitivity level to unstructured files, based on the rules defined in data classifications in Metadata Command Center.
Preview valid rows of a data quality rule occurrence
You can preview the valid rows for a data quality rule occurrence from the Data Preview options in the Score tab of a rule occurrence or the Ruel Occurrences tab of a rule template. The preview grid displays up to 1,000 valid rows.
Valid row preview and download are available for rule occurrences that you create in Data Governance and Catalog and for rule occurrences that you migrated from Data Profiling.
Data quality score upload using a JSON payload
You can use a public REST API to update the data quality score of a data quality rule occurrence by passing the score as a JSON payload in the request body. The API accepts one or more data quality scores in a single request and returns a success response with the job URI and job ID or a failure response with the appropriate response code.
Define column-level data quality checks through mapplets
You can use mapplets to define column-level data quality checks in Data Governance and Catalog. Mapplets let you encapsulate complex, reusable data quality logic and apply it consistently across rule occurrences, rule templates, and deep profiling workflows. Use mapplets when you want to reuse logic across multiple rules.
You can associate a mapplet with the following assets:
•A data quality rule template
•A manually created data quality rule occurrence
•A rule occurrence generated by CLAIRE
Additional asset search options
On the search page and within the search widgets, you can search for, filter, and group data elements by the following attributes:
•Associated data classifications
•Associated glossaries
•Classification categories
•Critical data element
•Sensitivity level
Additionally, on the data element page, Data Governance and Catalog displays a Critical Data Element (CDE) icon to indicate that the data element is a CDE on an associated glossary.
Data access management enhancements
You can use the following new and enhanced features for data access management:
•You can view pushdown enforcement information on the new Enforcement tab. You can view which assets a data access policy protects and which data access rules a data access policy applies to an asset. The status of the data access policy also appears. You can also view which technical assets have data access policies applied.
•You can push down data de-identification policies to the Databricks and Snowflake source systems.
•You can push down data access policies to external tables in the Databricks and Snowflake source systems.
Enhancements to widgets
This release includes the following enhancements:
Search-based widgets
The following enhancements apply to search-based widgets:
- Alphabetical ordering. The list of assets in the Group By options and the Select Display Columns list appear in alphabetical order.
- Additional Group By options and columns. You can group by and display additional columns based on the following asset types:
▪ AI System. AI System Type, Development Stage, Hierarchy, Asset Groups
- Improved stakeholder display. In the list view, stakeholders are shown by role, with a separate column for each role that lists the users assigned to it. Stakeholders display with their avatars.
- Classifications for unstructured assets.
When a grid contains unstructured assets, each supported classification type appears as its own column in the following way:
▪ Document classification
▪ Data entity classification
▪ Data element classification
Each classification type is a separate column that you can show or hide in grid customization. You can view the classification type in the grid.
- Grid customization in the widget drilldown. Similar to the Data Quality widget, you can customize the columns displayed in the widget drilldown grid. You can also set a sorting preference by Name or Last Updated.
- Change of assets in a widget. If the asset types in a widget change because the query results change, the system keeps your saved grid customization in the following ways:
▪ Available Columns: A column that applies to only some of the asset types remains in the list of available columns.
▪ Grid Display: When a selected column does not apply to an asset type, the column stays in the grid, and cells for that asset type remain blank.
▪ Saved Preferences: Every column you selected are saved in the saved preferences when the widget content changes.
- Additional filters. The widget drilldown includes additional filters, some of which apply only to specific asset types. In addition to the Group By attribute, you can filter by the following attributes:
▪ AI System Type
▪ Asset Groups
▪ Catalog Source Name
▪ Catalog Source Type
▪ Certified
▪ Critical Data Element
▪ Data range
▪ Environment
▪ Last Updated
▪ Policy Type
▪ Ratings
▪ Stakeholders
▪ System Type
- Filter components.
The drilldown grid uses the following filter components.
▪ Single Select with Find. It is used when you can select one value, for example, Critical Data Element, Certified, Last Updated, custom attributes.
▪ Multi-select with Find. It is used when you can select more than one value. Every other grid filter uses this component.
▪ Critical Data Element and Certified filters. Only True and False value options are displayed. The No value option is not displayed.
- Grid header.
The drilldown grid header is Assets for list, count, bar, and donut charts. The header stays Assets when you apply a filter.
- Percentage with counts. When you select a donut chart or a bar chart in a search-based widget, you can enable the option to show the percentage along with the counts.
- Result count. Search-based widgets show the result count against the total count.
Dashboard widgets
You can resize dashboard widgets to smaller dimensions. For the Donut, Count, and Bar Chart widgets, the chart adjusts as you resize the widget, displaying more or less detail based on the available space.
Use REST APIs to create and manage data profiles
You can use the following REST APIs to create and manage data profiles:
•Retrieve data profile summary. Use the API to retrieve the details of a data profile.
•Update data profile. Use the API to edit a data profile.
•Delete data profile. Use the API to delete a data profile.
•Retrieve data profiles configured for a data set. Use the API to retrieve a list of data profiles associated with the specified data set.
•Create a data profile for a data set. Use the API to create a data profile for the specified data set.
•Retrieve profile run details. Use the API to retrieve the details of a data profiling job instance.
•Export profiling results. Use the API to export the profiling results of a data profiling job instance to a CSV or Microsoft Excel file and download the file.
•Retrieve data profile runs. Use the API to retrieve a list of job instances from the last ten occasions on which the data profiling job ran.
•Run a data profile. Use the API to run a data profiling task.
Receive in-app and email notifications that summarize the number of assets with pending and automatically accepted recommendations in a catalog source. To review the recommendations in the Curation Center, click Review All Recommendations in the in-app notification or use the link in the email notification.
Notifications for business asset changes
Stakeholders of a business asset receive in-app and email notifications when users change the system and custom attribute values of the asset. Data Governance and Catalog groups the changes made within an hour into each notification.
Note:
Data Governance and Catalog doesn’t send notifications for changes to AI Agent business assets.
New catalog sources
This release includes the following new catalog sources:
•Amazon Bedrock
•Amazon SageMaker AI
•API Design
•dbt Cloud
•MuleSoft
•OpenLineage
•MCP Server
•Microsoft Fabric
•SAP Ariba
•SAP Cloud for Customer
•Trino
For more information about catalog sources, see the corresponding catalog source help.
Enhanced catalog sources
This release includes the following enhancements to catalog sources:
Amazon S3
You can use the IAM Roles Anywhere authentication type to connect to Amazon S3 source systems.
- You can extract metadata from Informatica Application Integration source systems.
- You can perform connection assignment to view lineage between an SAP ERP source system and an Informatica Intelligent Cloud Services source system.
- You can perform connection assignment to view lineage between an SAP Business Warehouse source system and an Informatica Intelligent Cloud Services source system.
- You can now configure how the catalog source reads Cloud Data Integration mapping task XML files when you extract metadata. Choose one of the following input fetch modes:
▪ API. Default. Fetches tasks through the Informatica Intelligent Cloud Services API.
▪ Files. Reads only the files in the input files folder.
▪ Both. Combines both modes, matching objects in the files and through the API.
- You can now extract metadata from the following activities in Automation Studio:
▪ Data Copy or Import
▪ Data Extract with no wildcard
▪ File Transfer
▪ Filter
▪ SQL Query with data extension and data view sources
- You can add metadata extraction filters based on the automation object type.
- You can now extract metadata and add metadata extraction filters for the following Salesforce Data 360 and Salesforce CRM objects:
▪ Activation
▪ Segmentation
- You can perform connection assignment to view lineage between a Salesforce Marketing Cloud source system and the Salesforce Data 360 and Salesforce CRM applications.
- You can perform connection assignment to view lineage between a Cloudera Hadoop source system and a Tableau source system.
- You can extract metadata from published data sources and Salesforce sources such as CRM, Data360, and Semantic Layer through Tableau. You can also view the real project hierarchy along with more detailed assets information and lineage in Data Governance and Catalog.
You can now define rules that assign a sensitivity level to Document assets. A Document asset represents an unstructured file, such as a PDF or Microsoft Word document, in the catalog. You can apply sensitivity levels to these assets based on the classification policies that you define.
You can configure catalog sources to propagate enrichments to related assets based on lineage.
Enable autonomous enrichment to propagate curation values such as classifications, descriptions, business names, and glossary associations across catalog source assets. Enable autonomous enrichment globally, and then configure each catalog source to control how it receives enrichments.
Propagation settings determine which enrichment types propagate, from how many lineage levels they propagate, and whether values are automatically accepted or recommended.
For more information about autonomous enrichment, see Autonomous Enrichment and the corresponding catalog source help.
Global settings for generated descriptions
You can now configure global settings that shape the descriptions Data Governance and Catalog generates for your assets. Set the industry and text length that apply to all generated descriptions. You can override these settings in Data Governance and Catalog.
For more information about configuring generated descriptions, see Autonomous Enrichment.
Incremental metadata extraction
You can now run incremental metadata extraction jobs on the following catalog sources:
•Microsoft Fabric OneLake
•Microsoft OneDrive
A full metadata extraction extracts all objects from the source to the catalog. An incremental metadata extraction considers only the changed and new objects since the last successful catalog source job run. Incremental metadata extraction doesn’t remove deleted objects from the catalog and doesn’t extract metadata of code-based objects.
For more information about catalog sources, see the corresponding catalog source help.
LIKE operators in data classification rules and lookup tables
When you build data classification rules in Data Governance and Catalog, you can use the LIKE ALL, LIKE SOME, and LIKE ANY operators to match a value against multiple patterns in a single condition. You can also reference a lookup table column as the pattern source. Use LIKE (lkp.<table>.<column>), LIKE SOME (lkp.<table>.<column>), or LIKE ANY (lkp.<table>.<column>) to match against the values in a lookup column.
You can now extract metadata from additional file formats across multiple catalog sources.
Newly supported file types:
•EML
•EMLX
•MSG
•RTF
•PPT
•PPTX
•JSONL
Supported catalog sources:
•Amazon S3
•File System
•Google Cloud Storage
•Hadoop Distributed File System
•Microsoft Azure Blob Storage
•Microsoft Azure Data Lake Storage Gen2
•Microsoft Fabric OneLake
•Microsoft OneDrive
•Microsoft SharePoint Online
•Oracle Cloud Object Storage
•SFTP File System
For more information about catalog sources, see the corresponding catalog source help.
Classify Microsoft Word documents
You can now use Document Classification in Metadata Command Center to classify Microsoft Word documents with the DOC and DOCX extensions. You can classify Microsoft Word documents for the following catalog sources:
•Amazon S3
•Hadoop Distributed File System
•Microsoft Azure Blob Storage
•Microsoft Azure Data Lake Storage Gen2
•Microsoft SharePoint Online
•Oracle Cloud Object Storage
•SFTP File System
For more information about catalog sources, see the corresponding catalog source help.
Deleting lookup tables
You can delete a lookup table that is not used in any data classification rule.
This release includes the following workflow enhancements:
Service task to retrieve asset details in a workflow
You can add a service task to route a workflow dynamically, based on asset groups and the custom attribute values of the asset the workflow runs on. You can use these values in expressions that define the gateway conditions.
Using the asset details available in the gateway conditions, you can configure an inclusive gateway that directs the workflow based on the asset metadata. For example, you can route the workflow along a different path for an asset that belongs to a specific asset group.
Inclusive gateway
You can add an inclusive gateway to a workflow to route the flow along one or more outgoing paths based on conditions. The inclusive gateway evaluates the condition on each outgoing path independently. The workflow follows every path that has a condition that evaluates to true. If no condition is true, the workflow follows a default path. An inclusive gateway activates one or more paths based on conditions.
Periodic review of assets (Preview)
You can set up a periodic review of assets while configuring workflow events in Metadata Command Center to certify business and data access assets on a schedule. Periodic review helps you keep assets accurate, up to date, and audit ready. You can configure a workflow event that reviews the assets that meet a condition, such as glossaries in a specific domain. When the schedule runs, the event automatically creates a review ticket for each qualifying business asset and data access asset.
Data Governance and Catalog creates review tickets automatically for the qualifying assets based on the schedule. The stakeholders defined in the workflow can review the asset, set its lifecycle to complete the review, and close the ticket. Each asset shows the last review date and the next review date. You can also search and browse assets by the last review date and the next review date.
Approval workflows for asset lifecycle changes
You can configure a workflow that runs when the lifecycle of an asset changes. Under the asset management event category, a new event type, Approval Request for Lifecycle Change, triggers a workflow when an asset's lifecycle state changes. The workflow runs when an asset moves from the Published state to the Obsolete state, or from the Obsolete state to the Published state. When you configure a workflow for this event type, you can select only the Obsolete or Published state.
Access control based on data classification sensitivity
When you create or modify an access policy in Metadata Command Center, you can add a sensitivity level as a predicate to a condition. The new predicate allows you to control access to assets based on the sensitivity level of their data classifications.
