When a mapping includes a Lookup transformation, the mapping task queries the lookup object based on the fields and properties that you configure in the Lookup transformation. The mapping task runs a default lookup query when the first row of data enters the Lookup transformation. If the Lookup transformation performs a relational lookup, you can override the default query.
You can override the default query for both cached and uncached lookups. To view the default query, run the mapping task. The default query appears in the log file.
Lookup SQL overrides for cached lookups
In a cached lookup, you can override the default SQL query to change the ORDER BY clause, add a WHERE clause, or transform the data. The mapping task uses the override statement to build the lookup cache.
The default query contains a SELECT statement that includes all lookup fields in the mapping. The SELECT statement also contains an ORDER BY clause that orders all columns in the same order in which they appear in the Lookup transformation.
If you want to change the ORDER BY clause, add a WHERE clause, or transform the lookup data before it is cached, override the default query. For example, you might use database functions to adjust the data types or formats in the lookup table to match the data types and formats of fields that are used in the mapping. Or, you might override the default query to query multiple tables.
Override the default query on the Advanced tab of the Lookup transformation. Enter the entire SELECT statement in the Lookup SQL Override field. Use an alias for each column in the query. If you want to change the ORDER BY clause, append "--" to the end of the query to suppress the ORDER BY clause that the mapping task generates.
Example
A Lookup transformation returns the following fields from Microsoft SQL Server table ALC_ORDER_DETAILS:
The transformation uses the following lookup condition:
ORDERID=in_ORDERID
When you run the mapping task, the following query appears in the log file:
LKPDP_1> DBG_21097 [2018-11-07 14:11:33.509] Lookup Transformation [lkp_ALC_ORDER_DETAILS]: Default sql to create lookup cache: SELECT PRODUCTID,UNITPRICE,QUANTITY,DISCOUNT,ORDERID FROM "icsauto"."ALC_ORDER_DETAILS" ORDER BY ORDERID,PRODUCTID,UNITPRICE,QUANTITY,DISCOUNT
To override the ORDER BY clause and sort by PRODUCTID, enter the following query in the Lookup SQL Override field on the Advanced tab:
SELECT PRODUCTID AS PRODUCTID, UNITPRICE AS UNITPRICE, QUANTITY AS QUANTITY, DISCOUNT AS DISCOUNT, ORDERID AS ORDERID FROM "icsauto"."ALC_ORDER_DETAILS" ORDER BY PRODUCTID --
When you run the mapping task again, the following query appears in the log file:
LKPDP_1> DBG_21312 [2018-11-07 14:14:36.734] Lookup Transformation [lkp_ALC_ORDER_DETAILS]: Lookup override sql to create cache: SELECT PRODUCTID AS PRODUCTID, UNITPRICE AS UNITPRICE, QUANTITY AS QUANTITY, DISCOUNT AS DISCOUNT, ORDERID AS ORDERID FROM "icsauto"."ALC_ORDER_DETAILS" ORDER BY PRODUCTID -- ORDER BY ORDERID,PRODUCTID,UNITPRICE,QUANTITY,DISCOUNT
Lookup SQL override for uncached lookups
You can override the default SQL query for an uncached lookup including the WHERE and ORDER BY clauses. The mapping task doesn't build a cache from the override statement for an uncached lookup.
When you run the mapping task, the task generates a default SELECT statement that includes the lookup fields, output fields, and WHERE clause based on the lookup condition. If the Lookup transformation is an unconnected lookup, the SELECT statement includes the lookup fields and the return field, but the mapping task doesn't generate the WHERE clause from the lookup condition.
Override the default query on the Advanced tab of the Lookup transformation. Enter the entire SELECT statement in the Lookup SQL Override field. Use an alias for each column in the query. If you want to change the ORDER BY clause, append " --" to the end of the query to suppress the ORDER BY clause that the mapping task generates.
To reference incoming fields in the WHERE clause, configure parameter binding by adding a "?" before and after the field name. The query data changes based on the value of the data in the field. The following example includes a WHERE statement that references the Name field:
SELECT EMPLOYEE.NAME as NAME, max(EMPLOYEE.ID) as ID from EMPLOYEE WHERE EMPLOYEE.NAME=?NAME1?
If you add a function to the SQL statement, the return data type must match the data type of the alias column. For example, the data type of ID matches the return type of the MAX function:
SELECT EMPLOYEE.NAME as NAME, MAX(EMPLOYEE.ID) as ID FROM EMPLOYEE
Note: You can't use subqueries in the SQL override for uncached lookups.
Guidelines for overriding the lookup query
Certain rules and guidelines apply when you override a lookup query.
Use the following guidelines:
•You can override the lookup SQL query for relational lookups.
•Enter the entire SELECT statement using the syntax that is required by the database.
•Enclose all database reserved words in quotes.
•Include all lookup and return fields in the SELECT statement.
If you add or subtract fields in the SELECT statement, the mapping task fails.
•Use an alias for each column in the query.
If you do not use column aliases, the mapping task fails with the following error:
Failed to initialize transformation [<Lookup Transformation Name>]
•The query must contain a FROM clause.
•To override the ORDER BY clause, append "--" at the end of the query.
The mapping task generates an ORDER BY clause, even when you enter one in the override. Therefore, you must enter two dashes (--) at the end of the query to suppress the generated ORDER BY clause.
•If the ORDER BY clause contains multiple columns, enter the columns in the same order as the fields in the lookup condition.
•If the mapping task uses SQL ELT optimization or is based on a mapping in SQL ELT mode, you can't override the ORDER BY clause or suppress the generated ORDER BY clause with comment notation.
•If multiple Lookup transformations share a lookup cache, use the same lookup SQL override for each Lookup transformation.
•When you configure a Lookup transformation that returns all rows, the mapping task builds the lookup cache with sorted keys. When the transformation retrieves all rows in a lookup, the mapping task builds the data cache with the keys in sorted order. The mapping task cannot retrieve all the rows from the cache if the rows are not sorted. If the data is not sorted on the keys, you might get unexpected results.
•You can't include parameters in the lookup SQL override.
•If you configure a lookup SQL override and a lookup source filter in the same transformation, the mapping task ignores the filter.
