When you read data from a Microsoft SQL Server source, transform the data, and write the data to a target, you can configure SQL ELT optimization to push the transformation logic to the source or target database system. If the source and target databases are the same, you can configure full SQL ELT optimization for improved performance.
When the Secure Agent applies SQL ELT optimization, it pushes transformation logic to a database. The Secure Agent translates the transformation logic into SQL queries and sends the SQL queries to the database. The database runs the SQL queries to process the transformations.
SQL ELT optimization improves mapping performance when the database can process the transformation logic faster than the Secure Agent. The Secure Agent also reads less data from the database.
The amount of transformation logic that the Secure Agent pushes to the database depends on the database, the transformation logic, and the mapping task. The Secure Agent processes all transformation logic that it cannot push to a database.
When you configure SQL ELT optimization for the mapping, the Secure Agent analyzes the optimized mapping from the source to the target or until it reaches a downstream transformation that it cannot push to the source database. The Secure Agent generates and executes a SELECT statement for each source that has transformation logic pushed down. Then, it reads the results of this SQL query and processes the remaining transformations in the mapping.
You cannot configure SQL ELT optimization for a mapping in advanced mode.
The Secure Agent can push the following transformation logic to a Microsoft SQL Server source or target:
Transformations
Supported SQL ELT optimization Type
Aggregator
Source, Full
Expression
Source, Full
Filter
Source, Full
Joiner
Source, Full
Sorter
Source, Full
Union
Source, Full
Router
Full
Full SQL ELT optimization
When the Secure Agent applies full SQL ELT optimization, it pushes all the transformation logic in the mapping to the target database.You can configure full SQL ELT optimization in the SQL ELT Optimization section.
Full SQL ELT optimization is ideal when the source and target are in the same connections. For example, if a mapping contains a Microsoft SQL Server source and a Microsoft SQL Server target, configure full SQL ELT optimization to push all the transformation logic for processing from a Microsoft SQL Server source database to a Microsoft SQL Server target database.
Source SQL ELT optimization
When the Secure Agent applies source SQL ELT optimization, it analyzes the mapping from source to target or until it reaches a downstream transformation it cannot push to the source database.
The Secure Agent generates and executes a SELECT statement based on the transformation logic for each transformation it can push to the database. Then, it reads the results of this SQL query and processes the remaining transformations.
You can configure a mapping to use source SQL ELT optimization if the source and target reside in different databases. For example, if a mapping contains a Microsoft SQL Server source and an Oracle target, you can configure source SQL ELT optimization to push some transformation logic for processing to the Microsoft SQL Server source.
SQL ELT optimization functions
When you use SQL ELT optimization, the Secure Agent converts the expression in the transformation by determining equivalent functions in the database. If there is no equivalent function in the database, the Secure Agent processes the transformation logic.
The following table summarizes the availability of SQL ELT optimization functions in Microsoft SQL Server:
Functions
SQL ELT optimization Type
ABS()
Source, Full
ADD_TO_DATE()
Full
ASCII() 1
Full
AVG()
Source, Full
CEIL() 1
Full
CHR()
Full
CONCAT() 1
Full
COS()
Source, Full
COSH()
Full
COUNT()
Source, Full
DATE_COMPARE()
Source, Full
DATE_DIFF()2
Full
DECODE()
Source, Full
EXP()
Source, Full
FIRST()
Full
FLOOR() 1
Full
GET_DATE_PART()
Full
IIF()
Source, Full
IN()
Source, Full
INSTR()
Full
IS_DATE()2
Full
IS_NUMBER()2
Full
ISNULL()
Source, Full
LAST_DAY()2
Full
LENGTH() 1
Full
LN()
Full
LOG()
Full
LOWER()
Source, Full
LPAD()2
Full
LTRIM() 1
Full
MAX()
Source, Full
MIN()
Source, Full
MD5() 2
Source, Full
MOD() 1
Full
POWER()
Source, Full
REPLACECHR()2
Full
REPLACESTR()2
Full
ROUND(NUMBER)
Full
RTRIM() 1
Full
SIGN()
Full
SIN()
Source, Full
SINH()
Full
SOUNDEX()
Full
SQRT()
Source, Full
STDDEV()
Full
SUBSTR()
Full
SUM()
Source, Full
SYSTIMESTAMP() 1
Full
TAN()
Source, Full
TANH()
Full
TO_BIGINT
Full
TO_CHAR(DATE) 1
Full
TO_CHAR(NUMBER) 1
Full
TO_DATE()
Full
TO_DECIMAL()
Full
TO_FLOAT()
Full
TO_INTEGER()
Full
TO_NUMBER()
Full
TRUNC(NUMBER)
Full
UPPER()
Source, Full
VARIANCE()
Full
1Applies also in Expression transformations for mappings enabled with source SQL ELT optimization.
2Applies only in an Expression transformation.
SQL ELT optimization variables
When you use SQL ELT optimization, the Secure Agent converts the expression in the transformation by determining equivalent variables in the database. If there is no equivalent variable in the database, the Secure Agent processes the transformation logic.
The following table summarizes the availability of SQL ELT optimization variables in Microsoft SQL Server:
Variables
SQL ELT optimization Type
SESSSTARTTIME
Full
SYSDATE
Full
Configuring SQL ELT optimization
To optimize a mapping, add the mapping to a task, and then configure SQL ELT optimization in the mapping task.
1In the Runtime Options tab of the Mapping task, navigate to the SQL ELT Optimization section.
2From the SQL ELT Optimization list, select the required type of SQL ELT optimization.
Cross-schema SQL ELT optimization
You can use cross-schema SQL ELT optimization for a mapping task to read from or write data to Microsoft SQL Server objects associated with different schemas within the same database.
To use cross-schema SQL ELT optimization, create two Microsoft SQL Server connections and specify the schema in each connection. Ensure that the schema in the source connection is different from the schema in the target connection, but both the schemas must belong to the same database. When you configure SQL ELT optimization for the mapping task, enable cross-schema SQL ELT optimization in the advanced session properties. By default, the check box is selected.
Configuring cross-schema SQL ELT optimization
Consider the following steps to configure cross-schema SQL ELT optimization for a Microsoft SQL Server mapping task:
1Create the following two Microsoft SQL Server connections, each defined with a different schema:
aCreate a sqlsv_1 Microsoft SQL Server connection and specify CQA_SCHEMA1 schema in the connection properties.
bCreate a sqlsv_2 Microsoft SQL Server connection and specify CQA_SCHEMA2 schema in the connection properties.
2Create a Microsoft SQL Server mapping, m_sqlsv_pdo_acrossSchema. Perform the following tasks:
aAdd a Source transformation and include a Microsoft SQL Server source object and connection sqlsv1 to read data using CQA_SCHEMA1.
bAdd a Target transformation and include a Microsoft SQL Server target object and connection sqlsv2 to write data using CQA_SCHEMA2.
3Create a Microsoft SQL Server mapping task, and perform the following tasks:
aSelect the configured Microsoft SQL Server mapping, m_sqlsv_pdo_acrossSchema.
bOn the Runtime Options tab, in the SQL ELT Optimization section, set the pSQL ELT optimization value to Full.
cIn the Advanced Session Properties section, select the Enable cross-schema SQL ELT optimization check box.
dSave the task and click Finish.
When you run the mapping task, the Secure Agent reads data from the Microsoft SQL Server source object associated with the CQA_SCHEMA1 schema and writes data to the Microsoft SQL Server target object associated with CQA_SCHEMA2 schema.
Rules and guidelines for SQL ELT optimization
Consider the following rules and guidelines when you configure SQL ELT optimization for a Microsoft SQL Server mapping:
•When you push transformation logic to the database, ensure that the database has enough resources to process the queries faster. Otherwise, there could be a performance degradation.
• If the following transformation or mapping conditions is true, the Secure Agent processes a logic instead of pushing it to the database:
- The transformation contains a variable port.
- The transformation is not a Sorter transformation, Union transformation, or target in a mapping.
- The transformation downstream from a Sorter or Union transformation, or contains a distinct sort.
- A configured mapping task to override the default values of input or output ports.
- The database does not have an equivalent operator, variable, or function to use in an expression in the transformation.
- The mapping contains too many branches. The Secure Agent can't generate an SQL query for a mapping that contains more than 64 two-way branches, 43 three-way branches, or 32 four-way branches. If the number of branches exceeds these limitations, the Integration Service processes the downstream transformations.
- A mapping task to log row errors.
Consider the following rules and guidelines when you configure full SQL ELT optimization for a Microsoft SQL Server mapping:
•You can't push the LTRIM(), RTRIM(), or MOD() function that contains more than one argument.
•To push the MOD() function to the Microsoft SQL Server database, the argument that you pass must be of the Integer date type.
•When you push the INSTR() function, you can only define string, search_value, and start arguments.
Consider the following rules and guidelines when you use an Expression transformation in a Microsoft SQL Server mapping enabled with full SQL ELT optimization:
•When you push the IS_DATE() function that contains the Text or Ntext data type, the IS_DATE() function returns 0.
•When you push the IS_DATE() and IS_NUMBER() functions and the values in the argument contain NULL, the functions return 0.
•You can't get a case-sensitive return value for the REPLACECHR() or REPLACESTR() functions.
•When you push the MD5() function that contains the nchar data type, the function returns a different value for the nchar data type as compared to a mapping that runs without SQL ELT optimization.
•To get the same return value for the nchar and char data types when you push the MD5() function, enter the ConvertToVarcharForMD5InPDO=Yes property in the mapping task.
•When you push the LPAD() function that contains the second_string argument, the function truncates the second string from left to right.
For example, for the LPAD('Infa',9,'RELATIONAL CONNECTIVITY') expression, the function returns the following value: IVITYInfa
•When you push the DATE_DIFF() function that contains the date1 and date2 arguments, the function returns the following different values as compared to a mapping that runs without SQL ELT optimization:
- The function returns a negative number when the value of date1 is later than the value of date2.
- The function returns a positive number when the value of date1 is earlier than the value of date2.