Enterprise Data Catalog
This section describes new Enterprise Data Catalog features in version 10.4.1.
Business Term Association
With effective in version 10.4.1, you can associate business terms to assets from Azure Data Lake Store Gen2 resource in the catalog.
For more information, see the Enterprise Data Catalog Concepts chapter in the Informatica 10.4.1 Catalog Administrator Guide.
Configure Conflict Resolution for Data Rule and Column Name Rule
Effective in version 10.4.1, you can choose to accept data rule result, column name rule result, or both the data rule and column name rule results if the rules conflict during auto acceptance of data domains.
For more information, see the Enterprise Data Catalog Concepts chapter in the Informatica 10.4.1 Catalog Administrator Guide.
Context Lineage Information
Effective in version 10.4.1, you can use the Lineage and Impact tab to view the context lineage information of the assets. Create a custom resource in Catalog Administrator to extract the context information for assets that include a process definition or mapping execution.
For more information, see the "Ingesting Custom Metadata into the Catalog" chapter in the Informatica 10.4.1 Catalog Administrator Guide and see the "View Lineage and Impact" chapter in the Informatica 10.4.1 Enterprise Data Catalog User Guide.
Data Asset Analytics
Effective in version 10.4.1, you can use Data Asset Analytics to view analytical insights about the catalog in the form of reports. Analytical insights include information about users configured to access the catalog, assets and resources, asset usage, and enrichment and collaboration details associated with assets.
For more information, see the Informatica 10.4.1 Enterprise Data Catalog User Guide
Data Curation through Export and Import
Effective in version 10.4.1, you can accept or reject the multiple inferred assets for data domains. You can export the assets to the comma-separated (CSV) file and import the updated CSV file into the Enterprise Data Catalog.
You can remove the data domains from the assets by deleting the data domains from the accepted or inferred data domain columns in the exported CSV file.
For more information, see the View Assets chapter in the Informatica 10.4.1 Enterprise Data Catalog User Guide.
Data Discovery
Effective in version 10.4.1, you can run profiles to discover data domains on the following resources and file types:
- •Azure Data Lake Store Gen2 resource.
You can run profiles on the structured, unstructured, and extended unstructured file types.
- •Cassandra resource.
You can perform data discovery on the Native engine.
- •Parquet file for the Amazon S3 and Azure Data Lake Store resources.
For more information, see the Enterprise Data Catalog Concepts chapter in the Informatica 10.4.1 Catalog Administrator Guide.
Enterprise Data Catalog Walkthroughs
Effective in version 10.4.1, you can access walkthroughs to help you understand and quickly start using Enterprise Data Catalog. Walkthroughs are guided tutorials within Enterprise Data Catalog that explain how you can use a specific functionality.
Note: Before you access the walkthroughs, ensure that you have access to the following Walkme domains:
- •https://cdn.walkme.com
- •https://playerserver.walkme.com
- •https://ec.walkme.com
- •https://rapi.walkme.com
- •https://papi.walkme.com
The following walkthroughs are available in Enterprise Data Catalog:
- •Getting Started and Introducing Catalog Homepage
- •Introduction to Search Results
- •Introduction to Table Overview
- •Introduction to Lineage
- •Introduction to Column Overview
- •Enhance Asset Credibility
To launch a walkthrough, click the ? help icon in the toolbar, and then select and click the Walkthrough link.
Extracting Transformation Endpoints
Effective in version 10.4.1, you can extract the transformation endpoints for a PowerCenter resource that has web service transformation mappings. In Enterprise Data Catalog, the transformation endpoints appear as connections, which links the data sources or targets.
Field-Level Lineage for Flat Files
Effective in version 10.4.1, you can view the field-level lineage for a source or target flat file in the Catalog.
The following table includes the file systems supported by the resources that you can configure to view the field-level lineage:
Resources | Supported File Systems |
|---|
PowerCenter | Linux Windows |
SQL Server Integration Service | Windows Azure Data Lake Store Gen2 |
Informatica Cloud Service | Azure Data Lake Store Gen2 Azure Data Lake Store Gen1 Amazon S3 Azure Blob Storage Google Cloud Storage Linux Windows |
IBM InfoSphere DataStage | Linux Windows |
Informatica Platform | Linux Windows |
To view the field-level lineage for flat files without headers, you must select the Enable Reference Resources, and Retain Unresolved Reference Assets properties during resource configuration. You do not require these properties to view the field-level lineage for flat files with headers.
File Lineage for Cloud Storage
Effective in version 10.4.1, you can use the Informatica Cloud Service resource to view the file lineage for Cloud Storage.
You can view the file lineage for the Informatica Cloud Service mappings that use data sources such as Amazon S3, Microsoft Azure Blob Storage, and Azure Data Lake Store Gen2.
Hive Resource
Effective in version 10.4.1, the Hive resource includes the following enhancements:
- Extract connection metadata
- You can use the Hive resource to extract connection details for views of different schemas.
- Automatically assign connections
- When you create the resource, you can choose to automatically assign the database schemas to the Hive resource. You can view the list of automatically assigned schemas and their connections for the resource. You can assign or unassign schemas in the auto-assigned connections.
For more information, see the Informatica 10.4.1 Enterprise Data Catalog Scanner Configuration Guide.
Informatica MDM Resource
Effective in version 10.4.1, the Informatica MDM resource extracts metadata from the MDM data source through APIs.
The Informatica MDM resource includes the following enhancements:
- Metadata extraction
- You can configure the Informatica MDM resource to extract metadata such as base objects, landing, and staging tables along with the field and attribute information.
- Lineage enhancements
- You can view the lineage between landing tables up to business entities. You can also view detailed lineage from the Informatica Platform applications.
For more information, see the Informatica 10.4.1 Enterprise Data Catalog Scanner Guide.
Microsoft SQL Server Resource
Effective in version 10.4.1, when you create the Microsoft SQL Server resource, you can choose to automatically assign the database schemas to the resource. You can view the list of automatically assigned schemas and their connections for the resource. You can assign or unassign schemas in the auto-assigned connections.
For more information, see the Informatica 10.4.1 Enterprise Data Catalog Scanner Configuration Guide.
MicroStrategy Resource
Effective in version 10.4.1, the MicroStrategy resource includes the following enhancements:
- View lineage at the report level
- You can view the lineage at the report level for a MicroStrategy resource. The report level lineage does not include containers such as page header, page footer, page body, and grouping elements such as columns, rows, and metrics.
- Support for incremental loading
You can enable incremental loading for a MicroStrategy resource. An incremental load causes the data source to load recent changes to the metadata instead of loading complete metadata. Incremental loading reduces the amount of time it takes to load the resource.
For more information, see the Informatica 10.4.1 Enterprise Data Catalog Scanner Configuration Guide.
Partitioned File Discovery
Effective in version 10.4.1, you can use the Amazon S3 and Azure Data Lake Store Gen2 resources to identify and publish horizontally partitioned files under the same directory, and files organized in Hive based hierarchical directory structures as a single partitioned file.
For more information, see the Informatica 10.4.1 Enterprise Data Catalog Scanner Configuration Guide.
Reference Resources and Reference Assets
Effective in version 10.4.1, you can configure the Hive and Microsoft SQL Server resources to extract metadata from data sources or other resources in the catalog that are referenced by the resource.
For more information, see the Informatica 10.4.1 Catalog Administrator Guide and the Informatica 10.4.1 Enterprise Data Catalog User Guide.
Resource Level Permissions
Effective in version 10.4.1, you can assign configuration permissions and apply restrictions on resources for users or user groups.
Note: You can also apply restrictions on resources during resource creation.
For more information, see the Informatica 10.4.1 Catalog Administrator Guide.
Extract the Database Name
Effective in version 10.4.1, you can use the Oracle resource to extract the names of Oracle databases that are configured as metadata sources with the Import Database Name option. The option is enabled by default.
You cannot use the -DextractDatabaseName=true JVM option to extract the database name of the Oracle metadata source.
Note: If you upgrade from an earlier version, the Import Database Name option replaces the -DextractDatabaseName=true JVM option in the Oracle resource configuration.
REST APIs
Effective in version 10.4.1, you can use the following Informatica Enterprise Data Catalog REST APIs:
- •Catalog Events REST APIs. In addition to the existing REST APIs, you can list the class types of events for the objects that the user subscribed.
- •Lineage Filter REST APIs. In addition to the existing REST APIs, you can create a default lineage filter.
- •Monitoring Info REST APIs. In addition to the existing REST APIs, you can perform the following tasks:
- - List the logs for a job.
- - Submit and list backup jobs, resource scan log jobs, and service log jobs.
- - Download a specific log for a job and ZIP files for a backup job, resource scan log job, and service log job.
For more information about the REST APIs, see the Informatica 10.4.1 Enterprise Data Catalog REST API Reference.
SAP BW and SAP BW/4HANA Resources
Effective in version 10.4.1, the SAP BW and SAP BW/4HANA resources extract metadata from the InfoObject as InfoProvider asset. You can view the lineage and the relationship information for the InfoObject as InfoProvider asset in the Catalog.
For more information, see the Informatica 10.4.1 Enterprise Data Catalog Scanner Configuration Guide.
SSIS Resource
Effective in version 10.4.1, the SSIS resource includes the following enhancements:
- View detailed lineage for transformations
- You can view the detailed lineage for transformations such as aggregate, audit, and character map in the Catalog.
- Metadata extraction from SSIS database
- You can configure the SSIS resource to extract metadata from the SSIS database.
- Filed level and control lineage support
- You can view the field-level lineage for flat files, and the control summary of SSIS assets in the Catalog.
For more information, see the Informatica 10.4.1 Enterprise Data Catalog Scanner Guide.