Data Synchronization Task Configuration
You can configure a Data Synchronization task using the Data Synchronization Task wizard.
To configure a Data Synchronization task using the Data Synchronization Task wizard, complete the following steps:
- 1. Define the Data Synchronization task.
- 2. Configure the source.
- 3. Configure the target.
- 4. Optionally, configure data filters.
- 5. Configure field mappings.
- 6. Optionally, configure a schedule and advanced options.
When you configure a Data Synchronization task, you can save your work at any time. You can choose one of the following options:
- •Save and continue
- •Save and run
- •Save and close
Step 1. Define the Data Synchronization Task
Define the Data Synchronization task.
1. To create a Data Synchronization task, click Task Wizards > Data Synchronization, and then click New.
To edit a Data Synchronization task, click Task Wizards > Data Synchronization, click the name of the Data Synchronization task, and then click Edit.
2. In the Data Synchronization Task Details area, configure the following fields:
Field | Description |
---|
Task Name | Name of the Data Synchronization task. The names of Data Synchronization tasks must be unique within the organization. Task names can contain alphanumeric characters, spaces, and the following special characters: _ . + - Maximum length is 100 characters. Task names are not case sensitive. |
Description | Description of the Data Synchronization task. Maximum length is 4000 characters. |
Task Operation | Select one of the following task operation types: - - Insert
- - Update
- - Upsert
- - Delete
The list of available targets in a subsequent step depend on the operation you select. |
3. Click Next.
Step 2. Configure the Source
Select the source for the Data Synchronization task. The steps to configure a source depend on the source type that you select.
The following table describes the procedures you can use:
Source Type | Procedure |
---|
Single object or saved query | |
Multiple database tables | |
Configuring a Single Object or Saved Query as the Source
You can configure a single object as the source of a Data Synchronization task. You can also configure a saved query as the source of a Data Synchronization task.
1. On the Source page, select a connection.
To create a connection, click New. To edit a connection, click View and then click Edit.
2. To use a single source, select Single.
To use a saved query, select Custom.
You can select a saved query when you use a database connection.
3. If the connection includes less than 200 objects, select a source object or click Select.
If the connection includes more than 200 objects, click Select.
The Select Source Object dialog box displays up to 200 objects. If the object you want to use does not display, enter a search string to reduce the number of objects that display.
Select an object and click Select.
4. To display technical names instead of business names for some connection types, select Display technical field names instead of labels.
Not available for all connection types.
5. To display source fields in alphabetical order, click Display source fields in alphabetical order.
By default, fields appear in the order returned by the source system.
6. For a Flat File or FTP/SFTP single source, click Formatting Options. Select a delimiter and text qualifier. Optionally, select an escape character.
If you choose Other for the delimiter, the delimiter cannot be an alphanumeric character or a double quotation mark.
If you choose a delimiter for an FTP/SFTP flat file, Informatica Cloud applies the delimiter to the local file, not the remote file, when previewing and reading data. If the remote and local files are not synchronized, you might see unexpected results.
7. If preview data does not display automatically, click Show Data Preview to preview data.
The Data Preview area shows the first ten rows of the first five columns in the object. It also displays the total number of columns in the object.
The Data Preview area does not display certain Unicode characters correctly. If the data contains binary data, the Data Preview area shows the following text:
BINARY DATA
8. To preview all source columns in a file, click Preview All Columns.
The file shows the first ten rows of the source.
9. Click Next.
Configuring Multiple Database Tables as the Source
You can configure multiple database tables as the source of a Data Synchronization task.
1. On the Source page, select a database connection.
To create a connection, click New. To edit a connection, click View and then click Edit.
2. Select Multiple.
The Source Objects table displays.
3. Click Add.
4. In the Select Source Objects dialog box, select the objects you want to use.
The dialog box displays up to 200 objects. If the objects that you want to use do not display, enter a search string to reduce the number of objects that display.
When you select an object, it appears in the Selected Objects list. To remove an object from the Selected Objects list, press Delete.
5. Click Select.
The selected sources display in the Source Objects table. To remove a source, in the Actions column, click Remove.
6. To display source fields in alphabetical order, select Display source fields in alphabetical order.
By default, source fields appear in the order returned by the source system.
7. Create source relationships or create a user-defined join, and click OK.
To create a relationship, perform the following steps:
- a. Select a database table and click Create Relationship.
- b. Select the source key for the table and then select the related source object and matching object key.
- c. Click OK.
- d. Match the primary key of the source table to the corresponding foreign key of the related database table.
- e. Create relationships as necessary to include all sources in the task.
To create a user-defined join to join all database tables, perform the following steps:
- a. Select User Defined Join and define the join.
- b. Any existing relationships are added to the join condition. To ensure that you enter field names correctly, use the Object list and Fields list to add field names to the join statement.
- c. To save the user-defined join, click OK.
8. To preview source data, select the source in the Source Objects table. If preview data does not appear automatically, click Show Data Preview.
The Data Preview area shows the first ten rows of the first five columns in the source. It also displays the total number of columns in the source.
The Data Preview area does not display certain Unicode characters as expected. If the data contains binary data, the Data Preview area shows the following text:
BINARY DATA
9. To preview all source columns in a file, select the source in the Source Objects table and click Preview All Columns.
The file shows the first ten rows of the source.
10. Click Next.
Step 3. Configure the Target
You can configure a single target for the Data Synchronization task. The options that display depend on task type and target type that you select for the task.
1. On the Target page, enter the following information:
Field | Description |
---|
Connection | Select a connection. The list of available connections depends on the task operation defined for the task. To create a connection, click New. To edit a connection, click View, and in the View Connection dialog box, click Edit. |
Target Object | If the connection includes less than 200 objects, select a target object or click Select. If the connection includes more than 200 objects, click Select. The Select Target Object dialog box displays up to 200 objects. If the object you want to use does not appear, enter a search string to reduce the number of objects that display. Select a target object and click OK. |
Display Technical Field Names Instead of Labels | Displays technical names instead of business names. Not available for all connection types. |
Display Target Fields in Alphabetical Order | Displays target fields in alphabetical order instead of the order returned by the target system. |
Formatting Options | For Flat File and FTP/SFTP connections only. Select a delimiter and text qualifier. Optionally, select an escape character. If you choose Other for the delimiter, the delimiter cannot be an alphanumeric character or a double quotation mark. If you choose a delimiter for an FTP/SFTP flat file, Informatica Cloud applies the delimiter to the local file, not the remote file, when previewing and reading data. If the remote and local files are not synchronized, you might see unexpected results. |
Create Target | Flat File connection only. Creates a target file. You can create a target file when the source connection is Salesforce, database, or ODBC, and the source object is Single or Custom. Enter a file name and select the source fields that you want to use. By default, all source fields are used. |
Truncate Target | Database targets with the Insert task operation only. Truncates a database target table before inserting new rows. - - True. Truncates the target table before inserting all rows.
- - False. Inserts new rows without truncating the target table.
Default is False. |
Enable Target Bulk Load | Select this option to write data in bulk mode. The default value is false. |
2. If preview data does not appear automatically, click Show Data Preview to preview data.
The Data Preview area shows the first ten rows of the first five columns in the target. It also shows the total number of columns in the target.
The Data Preview area does not display certain Unicode characters correctly. If the data contains binary data, the Data Preview area shows the following text:
BINARY DATA
3. To preview all target columns in a file, click Preview All Columns.
The file shows the first ten rows of the target.
4. Click Next.
Step 4. Configure the Data Filters
Use a data filter to reduce the number of source rows that the Data Synchronization task reads for the task. By default, the Data Synchronization task reads all source rows.
You can also configure the sort order for the task.
1. On the Data Filters page, choose whether to read all rows in sources or to read the first set of rows in sources.
- - To read all rows, select Process all rows.
- - To read the first set of rows, select Process only the first and enter a number.
2. To create a data filter, click New.
- - To create a simple data filter, select a source column and operator. Enter the value you want to use, and click OK.
- - To create an advanced data filter, click Advanced. Enter the field expression you want to use and click OK.
You can use parameters defined in a parameter file in data filters. When you use a parameter in a data filter, start the data filter with the parameter. For example, use $$Sales < 100000 instead of 100000 > $$Sales.
To delete a data filter, click the Delete icon.
3. To configure sort criteria, configure the following sort options:
Data Sorting Option | Description |
---|
Object | Source object. |
Sort By | Source field to use to sort data. |
Sort Direction | Sort direction: - - ASC. Ascending order.
- - DESC. Descending order.
|
To add additional sort criteria, click Add. Use the Move Up and Move Down arrows to define the order of the sort criteria.
To remove a sort criteria, click the Delete icon.
4. Click Next.
Step 5. Configure the Field Mapping
Configure field mappings to define the data that the Data Synchronization task writes to the target.
1. On the Field Mapping page, configure field mappings.
2. To view and configure field mappings in full screen mode, click Full Screen View.
To exit full screen mode, in the upper-right corner of the screen, click the Return to Normal View icon.
3. If you included multiple source objects in the task, you can select each source object in the Sources field to display the fields for the selected object. Or, you can view all source object fields.
When displaying all source object fields, the Sources table displays field names grouped by source object. You can place the cursor over the Status icon for a source field to determine the following information:
4. Some source types allow you to configure field datatypes. To configure field datatypes for a source, click Edit Types.
If the task includes more than one source, first select the source you want to edit.
In the Edit Field Datatypes dialog box, configure the following datatype attributes and click OK:
Datatype Attribute | Description |
---|
Datatype | The datatype of data in the column. |
Precision | Total number of digits in a number. For example, the number 123.45 has a precision of 5. The precision must be greater than or equal to 1. |
Scale | Number of digits to the right of the decimal point of a number. For example, the number 123.45 has a scale of 2. Scale must be greater than or equal to 0. The scale of a number must be less than its precision. The maximum scale for a Numeric datatype is 65535. |
5. To add a mapplet, click Add Mapplet.
- a. In the Add Mapplet dialog box, select the mapplet.
- b. To display technical names instead of business names, select Display technical field names instead of labels.
- c. To display fields in alphabetical order, click Display mapplet fields in alphabetical order.
By default, fields appear in the order specified by the mapplet.
- d. If necessary, select a connection for the mapplet.
- e. Click OK.
To configure field mappings, for Mapping Selection, select one of the following options.
- - Source to Target. Displays the source and target. Map source fields to the applicable target fields.
- - Source to Mapplet. Displays the source and the input fields of the mapplet. Map the source fields to the applicable input fields of the mapplet.
- - Mapplet to Target. Displays the output fields of the mapplet and the target fields. Map the output fields of the mapplet to the applicable target fields.
You cannot configure Mapping Selection options in full screen mode. If necessary, exit full screen mode to configure a Mapping Selection option.
The Clear Mapping, Automatch, and Validate Mapping buttons apply to the selected area of the field mapping.
6. To match fields with similar names, click Automatch.
Or, you can select and drag the source fields to the applicable target fields.
Informatica Cloud caches field metadata. If the fields do not appear correctly, click Refresh Fields to update the cache and view the latest field attributes.
7. To configure field datatypes for a target, click Edit Types.
Not available for all target types. If the task includes more than one target, first select the target you want to edit.
In the Edit Field Datatypes dialog box, configure the following datatype attributes and click OK:
Datatype Attribute | Description |
---|
Datatype | The datatype of data in the column. |
Precision | Total number of digits in a number. For example, the number 123.45 has a precision of 5. The precision must be greater than or equal to 1. |
Scale | Number of digits to the right of the decimal point of a number. For example, the number 123.45 has a scale of 2. Scale must be greater than or equal to 0. The scale of a number must be less than its precision. The maximum scale for a Numeric datatype is 65535. |
8. To create an expression to transform data, click the Add or Edit Expression icon in the Actions column.
In the Field Expressions dialog box, enter the expression you want to use and click OK.
You can use parameters defined in a parameter file in expressions.
9. To create a lookup, click the Add or Edit Lookup icon.
In the Field Lookup dialog box, configure the following properties and click OK:
Option | Description |
---|
Lookup Connection | Connection for the lookup object. |
Lookup Object | Object on which you want to lookup a value. |
Display Technical Field Names Instead of Labels | Displays technical names instead of business names. Not available for all connection types. |
Display Fields in Alphabetical Order | Displays lookup fields in alphabetical order. By default, fields appear in the order returned by the lookup system. |
Source Fields | Source column to use in the lookup condition. |
Lookup Fields | The column in the lookup table to use in the lookup condition. |
Output Field | The column in the lookup table that contains the output value. |
Multiplicity | Determines how to handle cases when a lookup returns multiple values: - - Error If More Than 1 Output Value. Select if the Data Synchronization task should display an error when the lookup condition returns multiple values. The application rejects rows when multiple matches are found, writing them to the error rows file. This is the default.
- - Randomly Pick 1 Output Value. Select if the Data Synchronization task should choose the first returned value when a lookup condition returns multiple values. Different systems might use different orders to return lookup values.
|
Expression | A simple expression that uses the $OutputField variable to represent the lookup return value. Enter a simple expression, such as $OutputField*100. You can use parameters defined in a parameter file in lookup expressions. To return the lookup return value without an additional expression, use $OutputField. |
10. To clear an expression or lookup and delete the field mapping, click the Clear Expression/Lookup icon next to the target field.
11. To clear all field mappings, click Clear Mapping.
12. To validate a mapping, click Validate Mapping.
13. Click Next.
Step 6. Configure a Schedule and Advanced Options
You can run a Data Synchronization task manually or schedule it to run at a specific time or interval. You can create a schedule or use an existing schedule.
You can also configure other advanced options for the task.
1. On the Schedule page, choose whether to run the task on a schedule or without a schedule.
2. To run a task on a schedule, click Run this task on schedule and select the schedule you want to use.
To create a new schedule, click New. Enter schedule details and click OK.
To remove the task from a schedule, click Do not run this task on a schedule.
3. If necessary, select a runtime environment to run the task.
4. Optionally, configure email notification options.
You can configure the following email notification options:
Field | Description |
---|
Use Default Email Notification Options for my Organization | Use the email notification options configured for the organization. |
Use Custom Email Notification Options for this Task | Use the email notification options configured for the task. You can send email to different addresses based on the status of the task: - - Failure Email Notification. Sends email to the listed addresses when the task fails to complete.
- - Warning Email Notification. Sends email to the listed addresses when the task completes with errors.
- - Success Email Notification. Sends email to the listed addresses when the task completes without errors.
Use commas to separate a list of email addresses. When you select this option, email notification options configured for the organization are not used. |
5. Optionally, configure advanced options.
You can configure the following advanced options:
Advanced Options | Description |
---|
Preprocessing Commands | Command to run before the task. |
Postprocessing Commands | Command to run after the task completes. |
Parameter File Name | Name of the file that contains the definitions and values of user-defined parameters used in the task. |
Maximum Number of Log Files | Number of session log files, error log files, and import log files to retain. By default, Informatica Cloud stores each type of log file for 10 runs before it overwrites the log files for new runs. |
Update Columns | Database targets only. Temporary primary key columns to update target data. If the database target does not include a primary key column, and the task performs an update, upsert, or delete task operation, click Add to add a temporary key. |
Upsert Field Name | The target field to use to perform upserts. |
6. Optionally, configure advanced source properties.
7. Optionally, configure advanced target properties.
8. Choose whether to run the task in standard or verbose execution mode.
If you select Verbose mode, the mapping generates additional data in the logs that you can use for troubleshooting. It is recommended that you select verbose execution mode only for troubleshooting purposes. Verbose execution mode impacts performance because of the amount of data it generates.
9. Click Save.