You can configure a Lookup transformation to use a persistent cache. When you use a persistent cache, Data Integration saves and reuses the cache files from mapping run to mapping run.
By default, Data Integration uses a non-persistent cache when you enable caching in a Lookup transformation. When you use a non-persistent cache, Data Integration deletes the cache files at the end of the mapping run. The next time you run the mapping, Data Integration builds the memory cache from the database.
If the lookup table does not change between mapping runs, you can use a persistent cache. A persistent cache can improve mapping performance because it eliminates the time required to read the lookup table. The first time that Data Integration runs a mapping using a persistent lookup cache, it saves the cache files to disk. The next time that Data Integration runs the mapping, it builds the memory cache from the cache files.
Configure the Lookup transformation to use a persistent lookup cache in the transformation advanced properties. To use a persistent cache, enable the Lookup Cache Persistent property.
You can configure the following options when you use a persistent cache:
Specify a name for the cache files.
When you use a persistent lookup cache, you can specify a name for the cache files.
To specify a name, enter the file name prefix in the Cache File Name Prefix field on the Advanced tab of the Lookup transformation. Do not enter a suffix such as .idx or .dat.
Rebuild the lookup cache.
If the lookup table changes occasionally, you can configure the Lookup transformation to rebuild the lookup cache. When you do this, Data Integration rebuilds the lookup cache from the lookup source when it first calls the Lookup transformation instance.
To configure the transformation to rebuild the cache, enable the Re-cache from Lookup Source property on the Advanced tab of the Lookup transformation.
Rebuilding the lookup cache
You can rebuild the lookup cache if you think the lookup source changed since the last time Data Integration built the persistent cache.
When you rebuild a cache, Data Integration creates new cache files, overwriting existing persistent cache files. Data Integration writes a message to the session log when it rebuilds the cache.
If Data Integration cannot reuse the cache, it rebuilds the cache or fails the mapping task. The behavior can differ based on whether the cache is named or unnamed.
The following table summarizes how Data Integration handles named and unnamed persistent caches when the mapping changes between runs:
Mapping changes between runs
Named cache
Unnamed cache
Data Integration cannot locate cache files. For example, the file no longer exists.
Rebuilds cache
Rebuilds cache
Enable or disable the Enable High Precision option in the mapping task advanced session properties.
Fails mapping task
Rebuilds cache
Edit the transformation in the Mapping Designer or Mapplet Designer, excluding editing the transformation description.
Fails mapping task
Rebuilds cache
Edit the mapping, excluding the Lookup transformation.
Reuses cache
Rebuilds cache
Change the number of partitions in the pipeline that contains the Lookup transformation.
Fails mapping task
Rebuilds cache
Change database connection or the file location used to access the lookup table.