Complete the following steps to create a mapping task:
1Create the mapping task.
2Configure the source.
3Configure the target.
4Define input parameters.
5Optionally, define persistent values.
6Optionally, configure runtime options.
As you configure the task, you can click Save to save your work at any time.
Creating a mapping task
Define general properties of the task on the General page.
1To create a mapping task, click New > Tasks, and then complete one of the following steps:
- To create a mapping task based on a mapping, select Mapping Task and click Create.
- To create a mapping task using a template, expand the appropriate template category, select the template you want to use, and then click Create.
2To edit a mapping task, on the Explore page, navigate to the mapping task. In the row that contains the task, click Actions and select Edit.
3Configure the following fields:
Field
Description
Name
Name of the task.
Task names can contain alphanumeric characters, spaces, and the following special characters: _ . + -
Maximum length is 100 characters. Task names are not case sensitive.
Location
Project or folder in which the task resides.
If the Explore page is currently active and a project or folder is selected, the default location for the asset is the selected project or folder. Otherwise, the default location is the location of the most recently saved asset.
Description
Description of the task.
Maximum length is 4000 characters.
Runtime Environment
Runtime environment that contains the Secure Agent to run the task.
Mapping
Mapping associated with the task.
To select the mapping, click Select and navigate to the mapping that you want to use.
To open the mapping, click View.
4Click Next.
Configuring sources
Configure source and lookup source parameters on the Sources page. The Sources page displays differently based on the basis for the task. If the mapping does not include source or lookup parameters, the Sources page does not appear.
You can add a single source object or multiple source objects based on the connection type and mapping configuration.
If the mapping specifies a connection parameter, and after you create the mapping task you edit the mapping to change the source, you might need to edit the task. Check the specific connection in the mapping task to determine if you need to reset it, and then validate the task again.
1For each source or lookup source parameter, configure the following details as required:
Parameter detail
Description
Connection
Select a connection.
To create a connection, click New. To edit a connection, click View, and in the View Connection dialog box, click Edit.
Tip: Hover over the source connection tool tip to view more information about the connection.
Source type
Source type. The options available depend on the connection type.
Object
Select an object or enter a query.
If a list of objects doesn't appear, click Select.
The Select Source Object dialog box displays up to 200 objects. If the object you want to use doesn't appear, enter a search string to reduce the number of objects.
Tip: Hover over the source object tool tip to view more information about the object.
Display Technical Field Names Instead of Labels
Displays technical names instead of business names.
Not available for all connection types.
Add Currently Processed File Name
Adds the source file name to each row. Data Integration adds the CurrentlyProcesedFileName field to the source at run time.
Available for parameterized source objects with flat file connections.
2Configure formatting options if required.
The properties available vary based on the connection type. For information about a particular connector's properties, see the help for the appropriate connector.
3Optionally, expand the Data Preview area to preview the data.
To display fields in alphabetical order, click Display fields in alphabetical order. By default, fields appear in the order returned by the source system.
To download the preview results, click Download.
4For parameterized source objects, configure data filters if necessary.
5For parameterized source objects, configure sort options if necessary.
6Configure field metadata if required.
You can configure field metadata for sources with certain connection types. To see if a connector supports field metadata configuration, see the help for the appropriate connector. You can't update field metadata in mapping tasks that are based on mappings in SQL ELT mode.
To edit field metadata, expand the Field Data Types area and select the row that contains the field that you want to edit. Configure the following attributes:
Data type attribute
Description
Retain existing fields at runtime
When enabled, the task uses the field metadata that is configured in the task.
If field metadata changes after a task is saved, Data Integration uses the updated field metadata. Typically, this is the desired behavior. However, if the task uses a flat file connection and you want to retain the metadata used at design time, enable this option.
Type
The data type of the field.
Precision
Total number of digits in a number. For example, the number 123.45 has a precision of 5.
The precision must be greater than or equal to 1.
Scale
Number of digits to the right of the decimal point of a number. For example, the number 123.45 has a scale of 2.
Scale must be greater than or equal to 0.
The scale of a number must be less than its precision.
The maximum scale for a numeric data type is 65535.
Not editable for all data types.
7Optionally, configure advanced options for the selected connection.
The options available depend on the connection.
When you edit advanced options, you can restore default values for individual properties or for all advanced properties. Default values are defined in the mapping that the task is based on.
For information about a particular connector's properties, see the help for the appropriate connector.
8Click Next.
Configuring targets
The Targets page displays differently depending on the basis for the task.
The Targets page displays when the mapping includes parameters for target connections or target objects. The properties that you need to specify are based on the type of parameter.
1For each target parameter, configure the following details:
Target parameter detail
Description
Connection
Select a connection.
To create a connection, click New. To edit a connection, click View, and in the View Connection dialog box, click Edit.
Tip: Hover over the target connection tool tip to view more information about the connection.
Object
Select a target object.
If a list of objects does not appear, click Select.
The Select Target Object dialog box displays up to 200 objects. If the object you want to use does not display, enter a search string to reduce the number of objects that display.
Tip: Hover over the target object tool tip to view more information about the object.
Operation
Target operation, either insert, update, upsert, delete, or data driven.
Not editable for all connection types.
Truncate target
Truncates the target object before inserting new rows.
Applies to insert and data driven operations.
Not available for all connection types.
Enable target bulk load
Uses the database bulk API to perform an insert operation.
Use the bulk API to write large amounts of data to the database with a minimal number of API calls. Loading in bulk mode can improve performance, but it limits the ability to recover because no database logging occurs.
Applies to insert operations.
Not available for all connection types.
Display technical names instead of labels
Displays technical names instead of business names.
Not available for all connection types.
2Configure formatting options if required.
The properties available vary based on the connection type. For information about a particular connector's properties, see the help for the appropriate connector.
3Optionally, expand the Data Preview area to preview the data.
To display fields in alphabetical order, click Display mapplet fields in alphabetical order. By default, fields appear in the order returned by the target system.
To download the preview results, click Download.
4Configure field metadata if required.
You can configure field metadata for targets with certain connection types. To see if a connector supports field metadata configuration, see the help for the appropriate connector. You can't update field metadata in mapping tasks that are based on mappings in SQL ELT mode.
To edit field metadata, expand the Field Data Types area and select the row that contains the field that you want to edit. Configure the following attributes:
Data type attribute
Description
Retain existing fields at runtime
When enabled, the task uses the field metadata that is configured in the task.
If field metadata changes after a task is saved, Data Integration uses the updated field metadata. Typically, this is the desired behavior. However, if the task uses a flat file connection and you want to retain the metadata used at design time, enable this option.
Type
The data type of the field.
Precision
Total number of digits in a number. For example, the number 123.45 has a precision of 5.
The precision must be greater than or equal to 1.
Scale
Number of digits to the right of the decimal point of a number. For example, the number 123.45 has a scale of 2.
Scale must be greater than or equal to 0.
The scale of a number must be less than its precision.
The maximum scale for a numeric data type is 65535.
Not editable for all data types.
5Optionally, configure advanced options for the selected connection.
The options available depend on the connection.
When you edit advanced options, you can restore default values for individual properties or for all advanced properties. Default values are defined in the mapping that the task is based on.
For information about a particular connector's properties, see the help for the appropriate connector.
6Click Next.
Configuring input parameters
The Input Parameters page displays differently depending on the basis for the task. If the mapping does not contain input parameters, the page doesn't display.
The Input Parameters page displays input parameters such as filter and lookup conditions, field mappings, and expressions. It also displays stored procedure connections. It doesn't display source, target, or lookup connections and objects.
1Configure the parameters that display.
Depending on the mapping data flow, you might need to configure some parameters before you configure other parameters. For more information, see Mappings.
Warning: If the mapping contains a filter parameter, the task might display as valid even if you haven't configured the filter value. You must configure all parameters before the task is valid and you can run it.
This displays when a stored procedure requires a connection. Select a connection or optionally create a new one.
3If necessary, configure lookup conditions.
This displays when the lookup condition is parameterized.
4If necessary, configure mapplet details.
This displays when a mapplet requires a connection and the mapplet is configured to display on this page.
For some connection types, you can select Display technical field names instead of labels to display technical names instead of business names.
To display fields in alphabetical order, click Display mapplet fields in alphabetical order.
5If necessary, configure field mappings.
This displays when the field mapping is parameterized. Use the Object list to display fields from different objects.
Map fields in one of the following ways:
- To automatically map fields with the same name, click Automap > Exact Field Name. Or, to map fields with similar names, click Automap > Smart Map.
You can undo all automapped field mappings by clicking Smart Map > Undo Automap. To unmap a single field, select the field to unmap and click Actions > Unmap.
- To manually map fields, select and drag the source fields to the applicable target fields.
6Click Next.
Configuring persistent values
The Persistent Values page displays if the mapping contains persisted values such as in-out parameters or sequences.
1For each sequence value, edit or reset the current value.
Note: If the Sequence transformation is configured to reset the value for each run, editing the current value in the mapping task has no effect.
2Configure in-out parameters as necessary.
In-out parameters don't apply to mapping tasks that are based on mappings in SQL ELT mode.
3Click Next.
Configuring runtime options
On the Runtime Options page, configure optional runtime options for the task. Expand each area to see the options you can configure.
1Specify whether to run the task on a schedule or without a schedule.
Choose one of the following options:
- If you want to run the task on a schedule, click Run on a schedule. Select the schedule you want to use or click New to create a schedule.
- If you want to run the task without a schedule, click Do not run on a schedule.
2Configure email notification options for the task.
3Optionally, in the Advanced Options area, configure the advanced options that are displayed for your connection:
Field
Description
Pre-Processing Commands
Commands to run before the task.
Post-Processing Commands
Commands to run after the task completes.
Maximum Number of Log Files
Number of session log files to retain. By default, Data Integration stores each type of log file for 10 runs before it overwrites the log files for new runs.
Note: If a dollar sign ($) is present in a custom session log file name, for example, MyLog_$CurrentTime, the file name is dynamic. If you customize the session log file name using a dynamic name, this property doesn't apply. To purge old log files, delete the files manually.
Schema Change Handling
Determines how Data Integration picks up changes to the object schema. Select one of the following options:
- Asynchronous. Data Integration refreshes the schema when you update the mapping or mapping task, and after an upgrade.
- Dynamic. Data Integration refreshes the schema every time the task runs.
Default is Asynchronous.
Dynamic Schema Handling
Determines how Data Integration applies schema changes from upstream transformations to the target object. Available when the schema change handling is dynamic and the field mapping is automatic.
For each target, select how Data Integration updates the target schema. The options available are based on the target connection.
For more information, see Schema change handling or the help for the appropriate connector.
4Optionally, to create a parameter file based on the parameters and default values specified in the mapping on which the task is based, click Download Parameter File Template within the parameter file location.
For more information about parameter file templates, see Mappings.
5Optionally, if the mapping task contains parameters, you can use parameter values from a parameter file. Choose one of the following options:
- To use a parameter file on a local machine, select Local. Enter the following information:
Field
Description
Parameter File Directory
Path for the directory that contains the parameter file, excluding the parameter file name. The Secure Agent must be able to access the directory.
You can use an absolute file path or a path relative to one of the following $PM system variables:
- $PMRootDir
- $PMSourceFileDir
- $PMLookupFileDir
- $PMCacheDir
- $PMSessionLogDir
- $PMExtProcDir
- $PMTempDir
By default, Data Integration uses the following parameter file directory:
Name of the file that contains the definitions and values of user-defined parameters used in the task.
You can provide the file name or the relative path and file name in this field.
- To use a cloud-hosted file, select Cloud Hosted. Enter the following information about the file:
Field
Description
Connection
Connection where the parameter file is stored. You can use the following connection types:
- Amazon S3
- Google Storage V2
- Azure Data Lake Store Gen2
Object
Name of the file that contains the definitions and values of user-defined parameters used in the task.
6Choose whether to run the task in standard or verbose execution mode, if this option is visible.
If you select verbose mode, the mapping generates additional data in the logs that you can use for troubleshooting. Select verbose execution mode only for troubleshooting purposes. Verbose execution mode impacts performance because of the amount of data it generates.
7Optionally, configure SQL ELT optimization.
The following table describes the SQL ELT optimization properties:
Property
Description
SQL ELT Optimization Type
Type of SQL ELT optimization. Use one of the following options:
- None. The task processes all transformation logic for the task.
- To Source. The task pushes as much of the transformation logic to the source database as possible.
- To Target. The task pushes as much of the transformation logic to the target database as possible.
- Full. The task pushes all of the transformation logic to the source and target databases. The source and target schema must be the same.
- $$PushdownConfig. The task uses the SQL ELT optimization type specified in the user-defined parameter file for the task.
When you use $$PushdownConfig, ensure that the user-defined parameter is configured in the parameter file.
When you use SQL ELT optimization, do not use the Error Log Type advanced session property.
Default is Full.
Note: SQL ELT optimization functionality varies depending on the support available for the connector. For more information, see the help for the appropriate connector. SQL ELT optimization doesn't apply to mapping tasks that are based on mappings in SQL ELT mode because mappings in SQL ELT mode are automatically configured to push transformation logic to the cloud data warehouse.
Optimization Context Type
Provides context about the mapping configuration for SQL ELT optimization. If you select an option other than None, Data Integration constructs a single query for SQL ELT optimization by combining multiple targets in the mapping based on the target configurations. If you select None, the query is not optimized.
If Data Integration cannot apply the selected context, Data Integration uses the default SQL ELT optimization behavior.
Select one of the following options:
- None
- SCD Type 2 merge
- Multi-insert
Default is None.
For more information, see the help for the appropriate connector.
SQL ELT Optimization Fallback Option
If full SQL ELT optimization is not available for the connection, choose how Data Integration handles SQL ELT optimization.
Choose one of the following options:
- Partial SQL ELT. Default. Data Integration pushes as much transformation logic as possible to the source and target database. The task processes any transformation logic that it can't push to a database.
- Non SQL ELT. The task runs with no SQL ELT optimization.
- Fail Task. Data Integration fails the task.
Default is disabled.
Create Temporary View
Allows the task to create temporary view objects in the database when it pushes the task to the database.
Use when the task includes an SQL override in the Source Qualifier transformation or Lookup transformation.
Default is enabled.
Disabled when the SQL ELT optimization type is None.
Create Temporary Sequence
This option is not used.
8Choose to enable cross-schema SQL ELT optimization.
Cross-schema SQL ELT optimization doesn't apply to mapping tasks that are based on mappings in SQL ELT mode.
9If you want to run multiple instances of the task at the same time, enable simultaneous runs of the mapping task.
Some mapping features might produce unexpected results in simultaneous task runs.