Load Types
The load type determines the type of operation to use when the Data Replication task replicates data from the source to the target.
Use one of the following load types when you replicate data:
- Incremental loads after initial full load
- The first time the Data Replication task runs, it performs a full load, replicating all rows of the source. For each subsequent run, the Data Replication task performs an incremental load. In an incremental load, the Data Replication task uses an upsert operation to replicate rows that changed since the last time the task ran. You can specify this load type when the task uses a Salesforce source and a database target.
- Incremental loads after initial partial load
- The Data Replication task always performs an incremental load with this load type. The first time the Data Replication task runs, the Data Replication task processes rows created or modified after a specified point in time. For each subsequent run, the Data Replication task replicates rows that changed since the last time the task ran. You can specify this load type when the task uses a Salesforce source and a database target.
- Full load each run
- The Data Replication task replicates all rows of the source objects in the task during each run. You can specify this load type when the task uses a Salesforce or database source and a database or flat file target.
For information about incremental load, see the Informatica Cloud Salesforce Connector Guide.
Full Load
For a full load, the Data Replication task replicates the data for all rows of the source objects in the task. Each time the task runs, the Data Replication task truncates the target database tables or flat file and performs a full data refresh from the source.
Run a full load in the following situations:
- •The Data Replication task uses a database source.
- •A Salesforce object in the Data Replication task is configured to be nonreplicateable within Salesforce.
If you run an incremental load on a Data Replication task that contains nonreplicateable objects, the Data Replication task runs a full load on the object. Contact the Salesforce administrator to get a list of replicateable Salesforce objects.
- •The data type of a Salesforce field changed.
If the Data Replication task detects a data type change, you might need to reset the target table to create a table that matches the updated Salesforce object. Then run the Data Replication task with full load to reload the data for all Salesforce objects included in the Data Replication task. Alternatively, you can set the AutoAlterColumnType custom configuration property so that the target table column updates to match the Salesforce object. The AutoAlterColumnType property does not apply in certain situations, such as when the source and target data types are not compatible. For more information about the AutoAlterColumnType property, see the Salesforce Connection Properties section.
Incremental Loads
You can use incremental loads when you replicate Salesforce data to a database target.
You can use the following types of incremental loads:
- •Incremental load after initial full load. The first time you run the Data Replication task, it runs a full load and replicates data from all the rows.
- •Incremental load for a specific period in time. The first time you run the Data Replication task, it performs an upsert to replicate data based on changes made to the source for a specified time period.
After the initial run, both incremental load types replicate data in the same manner.
The Data Replication task performs an upsert operation to replicate data for new rows and the rows that have changed since the last run of the task. The time of the last run is determined by the time that the last record is replicated from Salesforce.
Each replication session occurs in a single transaction. If you use the default commit interval and errors occur, the entire transaction rolls back. You can set the commit interval to a smaller value so that if a roll back occurs, only the last batch rolls back. To optimize performance, do not use an incremental load if you schedule the Data Replication task to run at long intervals, such as weeks or months. Instead, run a full load to avoid rollback of all data that was replicated during the schedule interval.
The Data Replication task never truncates target tables in an incremental run. To truncate target tables, you must run a full load Data Replication task.
When the Data Replication task compares Salesforce source objects and target tables to find inconsistencies, it completes the following tasks:
- 1. Compares field names in the source and target.
- 2. Inserts and deletes fields in the target table to match the source.
- 3. Runs a query to determine if values have been inserted or updated.
- 4. Replicates new and changed rows.
You cannot run an incremental load with the Salesforce Bulk API. When you select the incremental load option in a task, you disable the Salesforce Bulk API advanced option.
Rules and Guidelines for Running Incremental Loads
Use the following rules and guidelines when you run a Data Replication task as an incremental load:
- • You can add a column to a target. Because the Data Replication task reconciles field name inconsistencies before it runs the query to find changed data, changes to field names do not cause rows to be replicated. Replication occurs only if the row data changes.
For example, the Salesforce source contains a field named New that does not exist in the target. If you run an incremental load and no value exists in row 1 for the New field, the Data Replication task adds the New field to the target table, but it does not replicate the data in row 1. If the New field contains a value in row 1, then the Data Replication task replicates the data in row 1.
- • The incremental load might fail if the data type, precision, or scale for the source column and target column are inconsistent. The incremental load fails when the source and target columns have inconsistent data types or the source column has a higher precision or scale than the target column.
To resolve this issue, you can set the AutoAlterColumnType custom configuration property so that the target table column updates to match the Salesforce object.
Alternatively, you can reset the target to re-create the target tables to match the corresponding source objects. Next, run the Data Replication task with full load to reload the data for all source objects included in the Data Replication task.
- •The Data Replication task cannot run an incremental load on a Salesforce object that is non-replicateable or that does not have Created By and Last Modified By dates. Each Salesforce object can be configured to be non-replicateable in Salesforce. Contact the Salesforce administrator to get a list of replicateable Salesforce objects.
Salesforce does not track the Created By or Last Modified By dates for all objects.
- •If you run a task with an incremental load multiple times, the time period between the end time of the previous run and the start time of the next run must be at least 60 seconds. The task fails if you try to run the task before the 60-second waiting period.
- •You cannot run a task with an incremental load when the target is a flat file.