Identity Match Analysis and Persistent Index Data

When you run a mapping to analyze identity information, the Match transformation generates an index that stores the alternative versions of the identities in the data set. By default, the Match transformation writes the index data to temporary files. You can configure the transformation to save the index data to database tables.

The index tables that you generate represent a data store that you can reuse in subsequent mappings. You can compare the index tables to a data source, and you can optionally update the index tables with index data from the data source. Because the transformation does not regenerate the index tables, the subsequent mappings run faster. Additionally, the index tables can represent a trusted data store of identity data.

Rules and Guidelines for Persistent Index Data

Consider the following rules and guidelines when you configure a Match transformation to analyze a master data set of identity information:

•To generate a reusable index for a master data set, configure the transformation to write the index data to database tables. The database tables constitute a persistent store of the index data.

•To compare the identities in another data set to the index data store, configure the data set as the mapping data source. Configure the Match transformation to read the data source and the index data store. Select the index tables from the default schema of the database connection that you specify.

•The Match transformation adds the sequence identifier value from the input record to the index data rows that correspond to the record. The SequenceID input port contains the sequence identifiers. The transformation uses the sequence identifiers to track the index data through the different steps in the match analysis. Do not disconnect the sequence ID port.

•When you connect a Match transformation to an index store, the transformation reuses the population, key level, key type, and key field property values from the transformation that created the store. The transformation also reuses the port configuration from the transformation that created the store.

If the transformation properties do not match, the identity analysis cannot compare the mapping source data and the index data correctly.

•The Match transformation uses the data on the input port that you select as the key field to generate the identity index. The transformation can also write data from other ports to the index. If you disconnect the non-key field data ports from the transformation, you erase any data in the corresponding index columns when you run the mapping. To preserve the input port data in the index tables, do not disconnect the input data ports.

•You can disable the match analysis in the Match transformation when you generate the index table data for a data set. For example, you might disable the match analysis when you create an index store for a data set. When you disable match analysis, the mapping runs faster.

When you disable match analysis, the Match transformation can generate and display persistence status codes and persistence status descriptions. The transformation does not generate or display match scores or other data associated with the results of the match analysis. For example, if you configure the transformation to assign records to clusters and you disable match analysis, the transformation does not generate or display cluster ID values.

•You determine whether the Match transformation updates the index store with data from the mapping source. The Match transformation uses sequence identifiers to determine whether the rows in the index store and in the mapping data represent the same records.