Match Performance in Identity Analysis
To improve the mapping performance when you perform identity analysis on two data sets, configure the Match transformation to read identity index data from database tables. Run a mapping to create the index tables for the master data set. Run the mapping again to compare the index data to another data source.
Use the options on the Match Type view to identify the database tables that store the index data. Use the same options to select the index tables when you configure the transformation to compare the index data to data from another source.
To write the index data to database tables, perform the following tasks:
- 1. Create a mapping that reads a data source of identity information.
- 2. Configure a Match transformation in the mapping to write the index data to a database.
- 3. Run the mapping to generate the index data. The index data represents a data store that you can reuse.
To read the index data from database tables, perform the following tasks:
- 1. Create a mapping that reads another identity data source.
- 2. Configure a Match transformation in the mapping to read the index data from the database that you specified earlier.
When the mapping data source and the index data share a common structure, you can reuse the mapping that generated the index data.
- 3. Run the mapping to compare the data source to the index data.
The mapping generates index data for the data source. The mapping does not need to generate index data for the larger data set. Therefore, the mapping runs faster than a dual-source mapping that generates index data for both data sets.
Creating a Data Store for Identity Index Data
Configure a mapping that reads a data source that contains identity information. Use a Match transformation to write the index data to a database.
1. Create a mapping, and add the data source to the mapping canvas.
2. Add a Match transformation to the mapping canvas.
3. On the data source, select the ports that contain the identity information.
- - Connect the identity information ports to the Match transformation.
- - Connect the port that contains the sequence identifier values to the Match transformation.
4. On the Match transformation, select the Match Type view.
5. Set the match type to Identity Match with Persistent Record ID.
6. Configure the following options to create the index data store.
- - Set the persistence method to Update the database with new IDs.
- - Verify the matching process value. The default value is Enable. If you disable the matching process, the mapping creates the identity index tables but does not perform match analysis on the data.
- - On the DB Connection menu, select a database for the index tables.
- - On the Persistent Store menu, select Create New. Enter a name for the index in the Create Storage Tables dialog box.
7. Perform any other configuration step that the Match transformation requires. For example, configure a transformation strategy.
8. Add a target data object to the mapping.
9. Connect the Match transformation output ports to the target data object.
10. Run the mapping.
The mapping writes the index data for the data source to the database tables that you specify.
Using the Index Data Store in Single-Source Analysis
Configure a mapping that reads a data source of identity information. Use a Match transformation to compare the data source to the index data store for the master data set.
Before you configure the mapping, verify that the data source contains a column of sequence identifier values.
To save time, you can copy or reuse the mapping that created the index data store.
1. Open the mapping that generated the index data store.
Alternatively, open a copy of the mapping.
2. Verify the data source in the mapping.
If required, replace the data source with a source that contains the current data.
Note: If you delete the data source, you also delete the port connections on the Match transformation.
3. Identify the data source ports that contain the identity information.
- - Connect the identity information ports to the Match transformation.
- - Connect the port that contains the sequence identifier values to the Match transformation.
The input ports and the input port order must correspond to the input ports on the transformation that created the index tables.
4. On the Match transformation, select the Match Type view.
5. Set the match type to Identity Match with Persistent Record ID.
6. Verify the population, key level, key type, and key field values.
7. Configure the options to identify the index data store:
- - Set the persistence method. For example, select Do not update the database to preserve the current data in the index tables.
- - Set the matching process to Enable.
- - On the DB Connection menu, select the database that contains the index tables.
- - On the Persistent Store menu, browse to the tables that contain the index data.
8. Configure the following properties on the Match Output view:
- - Match. Identifies the records to analyze when the transformation reads index data from database tables.
- - Output. Filters the records that the transformation writes as output.
9. Perform any other configuration step that the Match transformation requires.
For example, configure a transformation strategy.
10. Verify the data target in the mapping.
If required, replace the data target with another target object.
11. Connect the Match transformation output ports to the target data object.
12. Run the mapping.
The mapping compares the data source records with the index data store. The transformation writes the index data for the data source to the data store.
Identity Match Analysis and Persistent Index Data