The generate tokens step generates match keys for all the records. Currently, the step uses SSA-NAME3 algorithm to generate match keys. The TF-IDF algorithm is not available in the current release.
If an ingress job fails at this step or the step is in the Running status for a long time, go to the My Jobs page of Data Integration to troubleshoot. If the taskflow is in the Suspended status, view the subtasks and select the failed mapping task. Download the sessions log and check for errors.
The following table lists the metrics that you can view for the generate tokens step:
Metric
Description
Total Records
The total number of records for which the step generated match keys.
Error Records
The number of records for which the step didn't generate match keys because of internal errors.
Success Records
The number of records for which match keys are generated.
Starting Records for Asset
The total number of records that the step processed for a specific asset.