Monitor MDM SaaS > Monitoring match model jobs > Monitoring match and merge jobs
  

Monitoring match and merge jobs

You can define a match job and a merge job separately or a single match and merge job. You can also choose to reset the matched records. After you run a job, you can monitor and view the job details. Each job performs a series of steps, and the steps vary based on the process type that you select.
After a match and merge job completes, you can view the job metrics, such as the total records processed for matching. You can evaluate the metrics to determine the performance of the match and merge job and the number of record pairs generated.
The following table lists the process types and their steps that are part of match and merge jobs:
Process Type
Process Steps
Match and Merge
Matches records and then merges the matching records based on the match model configuration.
The match and merge process type contains the following steps:
  • - Lock the Job
  • - Match
  • - Merge
  • - Index for Search
  • - Release the Job
Match Only
Matches records based on the match model configuration.
The match only process type contains the following steps:
  • - Lock the Job
  • - Match
  • - Release the Job
Merge Only
Merges the matching records based on the match model configuration.
The merge only process type contains the following steps:
  • - Lock the Job
  • - Merge
  • - Index for Search
  • - Release the Job
Reset the Matched Records
Resets the matched records that are not yet merged.
The reset the matched records process type contains the following steps:
  • - Lock the Job
  • - Reset
  • - Release the Job

Lock the job

The lock the job step locks the job instance so that you can run only a single job instance at a time for a business entity.
If the job fails at this step, check whether a similar job is already running.

Match records

The match step compares records based on the match model configuration and then identifies the matching records. You can evaluate the metrics and decide whether to enhance the match model configuration.
The following table describes the overall metrics:
Metric Name
Metric ID
Description
Total Records
MATCH_TOTAL_RECORDS
The total number of records that the step processed.
Skipped Records
MATCH_SKIPPED_RECORDS
The number of records that the step skipped because of data errors, such as data type mismatch. Fix the data errors so that the job can process the records.
Failed Records
MATCH_FAILED_RECORDS
The number of records that failed to match because of internal errors.
N/A
TOTAL_MATCH_JOBS
The total number of internal jobs that run to complete the match step.
Note: To view this metric, copy the detailed metrics of this step in the JSON format.
The following table describes the metrics based on declarative rules:
Metric Name
Metric ID
Description
Records Pairs for Automatic Merge
MATCH_DECLARATIVE_AUTO_MERGE_PAIRS
The number of record pairs that will automatically merge.
Nonmatching Record Pairs
MATCH_DECLARATIVE_NEVER_MERGE_PAIRS
The number of record pairs that didn't match.
Record Pairs for Manual Review
MATCH_DECLARATIVE_STEWARD_REVIEW_PAIRS
The number of record pairs that require manual review based on declarative rules.
The following table describes the metrics based on the machine learning model:
Metric Name
Metric ID
Description
Total Record Pairs
MATCH_AI_MODEL_EVALUATION_PAIRS
The total number of record pairs available.
Records Pairs for Automatic Merge
MATCH_AI_AUTO_MERGE_PAIRS
The number of record pairs that will automatically merge.
Nonmatching Record Pairs
MATCH_AI_NEVER_MERGE_PAIRS
The number of record pairs that didn't match.

Merge records

The merge step merges the records that are set for automatic merge based on declarative rules and the machine learning model.
The merge step forms groups of record pairs to efficiently merge the matching record pairs. Each group contains multiple record pairs.
The following table describes the metrics of the merge step:
Metric Name
Metric ID
Description
Total Record Pairs
number_of_match_pairs_read
The number of record pairs that the step processed.
Failed Groups
number_of_groups_that_failed_to_merge
The number of record pair groups that didn't merge due to internal errors, such as database connection errors.
Successful Groups
number_of_groups_merged
The number of record pair groups that the step successfully merged. The metric doesn't include the groups that require manual review and approval.
Pending Groups
number_of_pending_groups
The number of record pair groups that require manual review and approval.
Consolidated Record Pairs
number_of_match_pairs_state_updated
The number of record pairs that the step set to the CONSOLIDATED state to indicate that these records are successfully merged.
N/A
number_of_groups
The total number of record pair groups that the step created.
Note: To view this metric, copy the detailed metrics of this step in the JSON format.

Index records for search

The index for search step indexes all master records for search after the merge step.
The following table describes the metrics of the index for search step:
Metric ID
Description
SEARCH_INDEX_TOTAL_RECORDS
The total number of records that the step processed.
SEARCH_INDEX_FAILED_RECORDS
The number of records that failed to index.
OVERVIEW_SUCCESS_RECORDS
The number of records that are indexed for search requests.
SEARCH_INDEX_UPSERT_SUCCESS
The number of records for which the search indexes are added or updated.
SEARCH_INDEX_UPSERT_FAILURE
The number of records that failed to index for search.
SEARCH_INDEX_DELETE_SUCCESS
The number of records whose indexes were deleted.
SEARCH_INDEX_DELETE_FAILURE
The number of records whose indexes failed to be deleted.

Group potential matches

The group potential matches step optimizes the number of record pair groups by identifying related record pairs and grouping them.
For example, record A matches with record B, and record B matches with record C. The match process might result in three record pair groups, one for A and B, one for B and C, and one for A and C. The group potential match pairs step identifies these record pair groups and merges them into one record pair group with records A, B, and C.
If you use an approval workflow, when you generate merge tasks, you get one approval task instead of three tasks.
If the number of record pairs or groups exceeds the processing limit, a warning message appears, and the match and merge job runs successfully without completing the group potential match pairs step. You must reconfigure the match model so that the number of record pairs or groups doesn't exceed the limit. After you configure the match model, you can run the reset the matched records job and then run another match and merge job.
The following table describes the metrics of the group potential matches step:
Metric ID
Description
TOTAL_PAIRS
Number of record pairs that are available for manual review.
NUMBER_OF_GROUPS
Number of groups that the step processed.
NUMBER_OF_MATCH_PAIRS_READ
Number of record pairs that are available for manual review.
number_of_pairs_exceeding_limit
Number of record pairs that exceeds the processing limit of 1,000,000 record pairs.
number_of_groups_exceeding_limit
Number of groups that exceeds the processing limit. of 100,000 groups.

Reset matched records

The reset step resets the already matched records that are yet to be merged. When you reset the matched records, the record state changes from MATCHED to MATCH_DIRTY. The MATCH_DIRTY record state indicates that the record did not undergo the match process.
The following table describes the metrics of the reset step:
Metric ID
Description
MATCH_TOTAL_RECORDS
The total number of records that step processed.
MATCH_FAILED_RECORDS
The number of records that failed to reset.
TOTAL_FAILED_STATE_TRANSIT_COUNT
The number of records that failed to reset as the record state didn't change.
TOTAL_MATCH_PAIRS_DELETED_COUNT
The number of match pairs that the step deleted after resetting the state of records.
TOTAL_RECORDS_RESET_STATE_TRANSIT_COUNT
The number of records for which the step reset the record state.

Release the job

The release the job step releases the locked job instance.