Task Details: Configure how to replicate data from the source

Source Tables

• •On the Selected Tables tab, individually select the check box for each source table and column you want to include. Clear the check box for any tables or columns you do not want to include. To select all tables and columns, select the Table check box at the top.
The Column count for a table shows the total number of columns in the table.
Note:
If you deselect one or more columns for a table, a minus (-) appears next to the table name in the list.
If you select the tables you want to include, all the selected and unselected tables are displayed by default. To view the selected tables only, use the filter next to the table selection count and change the view from All to Selected.
To find tables or columns, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns tables and columns that include CDC anywhere in their names. To narrow the search to only table names or column names, select Table Name or Columns in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for tables and columns.
• - To add a table rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
If you want to include database views as sources, select the Include Views check box to the left of the plus sign. This check box is available for initial load tasks.
• - To add a column rule, click the plus (+) sign at the end of the row at the table level. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
• - Validate the rule. The validation warns if your rule excludes a primary key column.
Note:
If you define multiple table or column rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a table rule specifies "Exclude CDC" followed by "Include C", all tables with names beginning with "C" are selected, including the CDC tables.
Tips and guidelines for rules:
• - You can copy an existing table rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a table or column rule directly in its row by clicking the Type or Match Condition value.
• - To view the tables and columns that match a single table rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - The task wizard is case sensitive. Enter the table or column names or masks in the case with which they were defined.
• - Key columns are selected by default and can't be excluded.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a table or column name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.

Advanced Source Properties

1. 1Complete the following optional advanced properties as needed:

Property	Load Type	Description
CDC Script	Incremental loads and combined initial and incremental loads	Generate a script for enabling CDC on source tables and then run or download the script. The only available option is Enable CDC for all columns. For Db2 for i sources, the script enables journaling on the source tables. Click Execute to run the script if you have the required privileges. Or click the Download icon to download the script so that you can give it to your DBA to run.
List Tables by Rule Type	All	Generate and download a list of the source tables that match the table selection criteria. If you used rule-based table selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Select the Include Columns check box to include columns in the list, regardless of which table selection method you used. Click the Download icon to download the list.
Enable Persistent Storage	Incremental loads and combined initial and incremental loads Note: The property is not available if you are using a serverless runtime environment.	Select this check box to enable persistent storage of transaction data in a disk buffer so that the data can be consumed continually, even when the writing of data to the target is slow or delayed. Benefits of using persistent storage are faster consumption of the source transaction logs, less reliance on log archives or backups, and the ability to still access the data persisted in disk storage after restarting a database ingestion job. Persisted data is stored on the Secure Agent. It is not encrypted. The Secure Agent's files and directories are expected to be secured from unwanted access by using native file system access permissions or file system support of encryption natively. If you select this check box and also select Stage CDC Data, this check box is cleared and becomes unavailable.
Job Exit Token	Incremental loads and combined initial and incremental loads	A unique identifier that's sent to the journal receiver exit on the IBM i machine to prevent the journal receivers from being deleted while change data capture is processing them. The exit program locks the journal receivers for the duration of time that they're in use for CDC.
Initial Start Point for Incremental Load	Incremental loads only	If you want to customize the position in the source logs from which the database ingestion and replication job starts reading change records the first time it runs, select one of the following options: - Earliest Available. The earliest available position in the database log or structure where changes are stored. For Db2 for i, the start of the current journal. - Latest Available. The latest available position in the database log or structure. - Position. The Db2 for i sequence in the change stream from which you want the database ingestion job to start retrieving change records. This value must be equal to or less than the current position value. An invalid value will cause the job to fail. For Db2 for i, do not use the default value of 0. If you select the Stage CDC Data option, this option is not available. - Specific Date and Time. A date and time, in the format MM/DD/YYYY hh:mm AM|PM, that Database Ingestion and Replication uses to determine the position in the change stream from which to start retrieving change records. Database Ingestion and Replication retrieves only the changes that were started after this date and time. If you enter a date and time earlier than the earliest date and time in the available archived logs, the job will fail. If you select the Stage CDC Data option, this option is not available. The default is Latest Available. Note: - This initial start point setting pertains only to the initial run of a job. Thereafter, if you resume a stopped or aborted job, the job begins propagating source data from where it last left off. - If you select the Stage CDC Data option, this field shows only the Earliest Available and Latest Available options. - For combined initial and incremental load jobs, initial loading is not performed until the incremental processing of change data reaches the end of the current transaction log.
Stage CDC Data	Incremental loads and combined initial and incremental loads	If you want to read data from the source database in a single pass and then write the data to common storage so that that it can be read by multiple tasks that process the same database, select this check box. The staged data can then be read by the tasks that are in the staging group. For a log-based source, the tasks can process different tables with different schemas.
Staging Group	Incremental loads and combined initial and incremental loads	If you want to use a CDC staging group to stage CDC data for multiple tasks, either select a previously defined group from the list or click New to create one.

If you chose to create a new staging group, complete the following fields on the Overview tab of the New Staging Group dialog box:

Property	Description
Group Name	Accept the generated group name, which has the format Log_group_cdc_yyyymmdd<number>, or enter a custom name.
Location	The project in which you want to store the staging group definition, if you don't want to use the Default location.
Staging Location Connection	Select a connection to the cloud staging location, which can be in Amazon S3, Google Cloud Storage, or Microsoft Azure Data Lake Storage Gen2.
Runtime Environment	Select the runtime environment to use for running the CDC staging task. For a log-based source, ensure that the Secure Agent on which the task will run can access all of the source logs.
Enable Alternate Connection for Reading Logs	Do not select this option for Db2 for i sources even though it's available.
Alternate Connection for Reading Logs	Available after you select Enable Alternate Connection for Reading Logs but does not pertain to Db2 for i sources.
Journal Name	Enter the name of the journal that records the changes. You must use the same journal name for all tasks in the staging group.
Job Exit Token	A unique identifier for each CDC job in the group that's sent to the journal receiver exit on the IBM i machine to prevent the journal from being deleted while the reader is still reading it. The exit program locks the journal for the duration of time that it's in use by the reader. You must use the same job exit token value for all tasks in the staging group.
Include LOBs	Not available.
Row Flush Threshold	The maximum number of rows that can be written to the files that store data temporarily before the data is transferred to cloud storage. The data is flushed to cloud storage either when this number of rows is reached or when the flush interval expires. Default is 50000 rows.
Flush Interval	During periods of low change activity on the source, the number of minutes and seconds that a job waits for more change data before flushing the data in the temporary files to cloud storage. The data is flushed either when this interval expires or when the row flush threshold is met. Default is 30 seconds.
Log Start Point	The Db2 for i journal from which the CDC staging job starts reading change records the first time it runs. Options are: - Earliest Available - Latest Available - Position.Then enter an appropirate position in the change stream for your source type. - Specific Date and Time. Then specify the date and time. Note: These options are similar to those described for Initial Start Point for Incremental Load in the source properties. Default is Latest Available.
Staging Data Retention Period	The number of days to retain data in cloud storage. Valid values are 0-365. Default is 14 days. After the retention period expires, the data is purged and is no longer available for subsequent restarts.

If you want to add source and target custom properties for the CDC staging jobs in the group, click the Custom Properties tab. Then enter each property name and value. Typically, you use custom properties only at the direction of Informatica Global Customer Support or Informatica technical staff.
When done, click Save to save the staging group and return to the Task Details, Step 1 page.
Note:
If you edit the staging group later from a deployed task in the group, you can change only the Row Flush Threshold, Flush Interval, Staging Data Retention Period, and custom property values.
2. 2Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available, depending on the load type:

Propery	Description
Read Event Batch Size	The number of payload events written in batch to the internal event queue during CDC processing. When the event queue is implemented as an internal ring buffer, this value is the number of payload events that the reader writes to a single internal buffer slot. Note: A batch size that's too small might increase contention between threads. A larger batch size can provide for more parallelism but consume more memory.
Reader Helper Thread Count	The number of reader helper threads used during CDC processing to convert change data into a canonical format that can be passed to the target. Default value is 3. You can enter a larger value to allow more threads to be available for performing conversion processing in parallel.
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal thread count. You can override the value from auto-tuning by setting this custom property.
Enable Partitioning	Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Select this property to use multiple partitions with one query per partition. By default, this property is not selected. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine whether to enable partitioning. You can override the value from auto-tuning by setting this custom property.
Unload Source Partition Count	The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected. Default is 5. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partition count. You can override the value from auto-tuning by setting this custom property.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
If you configured a CDC staging group, any custom properties that you enter here for the source are ignored. Instead, enter custom properties for the staging group.
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.

Source Tables

• •On the Selected Tables tab, individually select the check box for each source table you want to include. Clear the check box for any tables you do not want to include. To select all tables, select the Table check box at the top.
The Column count for a table shows the total number of columns in the table.
If you select the tables you want to include, all the selected and unselected tables are displayed by default. To view the selected tables only, use the filter next to the table selection count and change the view from All to Selected.
To find tables or columns, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns tables and columns that include CDC anywhere in their names. To narrow the search to only table names or column names, select Table Name or Columns in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for tables.
To add a table rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
If you want to include database views as sources, select the Include Views check box to the left of the plus sign. This check box is available for initial load tasks and combined initial and incremental load tasks that use the Query-based CDC method.
Note:
If you define multiple table rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a table rule specifies "Exclude CDC" followed by "Include C", all tables with names beginning with "C" are selected, including the CDC tables.
Tips and guidelines for using rules:
• - You can copy an existing table rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a table rule directly in its row by clicking the Type or Match Condition value.
• - To view the tables and columns that match a single table rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - Enter the table names or masks in the case with which they were defined. The task wizard is case sensitive.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a table name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.

Advanced Source Properties

1. 1Under Change Data Capture Method, select one of the following CDC methods if you're defining an incremental load or combined initial and incremental load task:
• - Query-based. Captures Inserts and Updates by using a SQL WHERE clause that points to a CDC query column. The query column is used to identify the rows that contain the changes made to the source tables since the beginning of the CDC interval.
• - Log-based. Captures Inserts, Updates, and Deletes in near real time from Db2 transaction logs for selected source tables on Linux or Windows.
Note:
In the current release, access to the Log-based CDC method is controlled by a flag on your organization. If you do not have access to this CDC method, contact Informatica Global Customer Support.
Default is Query-based.
2. 2If you selected Query-based as the capture method, complete the following fields:
• - CDC Query Column Type. The column data type of the CDC query column in the source table. The column data type must combine the date and time. For Db2 for LUW sources, the only available option is Timestamp.
• - CDC Query Column Name. The case-sensitive name of the CDC query column in the source table. The column must be present in the source table. Maximum length is 70 characters.
• - CDC Interval. The frequency of a query-based change data capture cycle, expressed in days, hours, and minutes. You must enter a positive number in at least one of the interval fields. Otherwise, an error will be issued when you try to save the task. The default value is 5 minutes.
3. 3If you selected Log-based as the capture method, complete the following fields, unless you plan to use a CDC staging group:
• - Catalog Table Name. The name of the capture catalog table in the format <schema>.<table_name>. The table stores information about the Db2 source tables, column definitions, and valid Db2 log positions. When you first run the job, it generates the capture catalog table in the same database that contains the source tables. Each task must have its own capture catalog table.
• - Capture Threads. The number of threads to use for capturing change data from Db2. Valid values are 1 - 9. The default value is DEFAULT, which indicates that the capture program chooses the number of threads to use.
• - Log Read Buffer Size. The buffer size, in kilobytes, that the capture process uses for change data read from the Db2 log records.
If you plan to use a CDC staging group, specify this information for the staging group instead.
4. 4Complete the following optional advanced properties as needed:

Property	Load Type	Description
CDC Script	Incremental loads and combined initial and incremental loads that use Log-based CDC	Generates a script that enables Db2 DATA CAPTURE CHANGES for all columns in each selected source table. The only available option is Enable CDC for all columns. You can execute the script, or if you don't have permissions to do so, download the script and give it to your DBA to run.
List Tables by Rule Type	All	Generate and download a list of the source tables that match the table selection criteria. If you used rule-based table selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Select the Include Columns check box to include columns in the list, regardless of which table selection method you used. Click the Download icon to download the list.
Include LOBs	All load types to Microsoft Azure Data Lake Storage Gen 2, Microsoft Azure Synapse Analytics, Microsoft Fabric OneLake, or Snowflake targets. Incremental load and combined load tasks must use the Query-based CDC method for this field to be displayed.	Select this check box if the source contains the large-object (LOB) columns from which you want to replicate data to a target. LOB data types for Db2 for LUW source: BLOB, CLOB, DBCLOB, LONG VARCHAR, LONG VARCHAR FOR BIT, LONG VARGRAPHIC, and XML LOB data might be truncated on the target. For more information, see About LOB truncation.
Initial Start Point for Incremental Load	Incremental loads only	If you want to customize the position in the source logs from which the database ingestion and replication job starts reading change records the first time it runs, select one of the following options: - Earliest Available. The earliest available position in the database log or structure where changes are stored. Not available for Query-based CDC method is selected. - Latest Available. The latest available position in the database log or structure. - Specific Date and Time. A date and time, in the format MM/DD/YYYY hh:mm AM|PM, that Database Ingestion and Replication uses to determine the position in the change stream from which to start retrieving change records. Database Ingestion and Replication retrieves only the changes that were started after this date and time. If you enter a date and time earlier than the earliest date and time in the available archived logs, the job will fail. The default is Latest Available. Note: - This initial start point setting pertains only to the initial run of a job. Thereafter, if you resume a stopped or aborted job, the job begins propagating source data from where it last left off. - For combined initial and incremental load jobs, initial loading is not performed until the incremental processing of change data reaches the end of the current transaction log.
Enable Persistent Storage	Incremental loads and combined initial and incremental loads that use Log-based CDC. For Query-based CDC, this field is not displayed because persistent storage is enabled by default and cannot be changed.	If you're using log-based CDC and do not plan to stage CDC data, select this check box to enable persistent storage of transaction data in a disk buffer so that the data can be consumed continually, even when the writing of data to the target is slow or delayed. Benefits of using persistent storage are faster consumption of the source transaction logs, less reliance on log archives or backups, and the ability to still access the data persisted in disk storage after restarting a database ingestion job. If you choose to stage CDC data, this option is deselected and becomes unavailable.
Stage CDC Data	Incremental loads and combined initial and incremental loads that use Log-based CDC	If you want to read data from the source database in a single pass and then write the data to common storage so that it can be read by multiple tasks that process the same database, select this check box. The staged data can then be read by the tasks that are in the staging group. For a log-based source, the tasks can process different tables with different schemas.
Staging Group	Incremental loads and combined initial and incremental loads that use Log-based CDC	If you want to use a CDC staging group to stage CDC data for multiple tasks, either select a previously defined group from the list or click New to create one.

If you use Log-based CDC and chose to create a new staging group, complete the following fields on the Overview tab of the New Staging Group dialog box:

Property	Description
Group Name	Accept the generated group name, which has the format Log_group_cdc_yyyymmdd<number>, or enter a custom name.
Location	The project in which you want to store the staging group definition, if you don't want to use the Default location.
Staging Location Connection	Select a connection to the cloud staging location, which can be in Amazon S3, Google Cloud Storage, or Microsoft Azure Data Lake Storage Gen2.
Runtime Environment	Select the runtime environment to use for running the CDC staging task. For a log-based source, ensure that the Secure Agent on which the task will run can access all of the source logs.
CDC Method	The only available option is Log-based. CDC staging groups can't be used with Query-based CDC.
Catalog Table Name	The name of the capture catalog table that stores information about the Db2 source tables, column definitions, and valid Db2 log positions. This table must be created in the same database that contains the source tables from which change data is captured.
Capture Threading	The number of threads that the CDC staging task uses for capturing change data from Db2. Valid values are 1 - 9. The default value is DEFAULT.
Log Read Buffer Size	The buffer size, in kilobytes, that the CDC staging task uses for change data read from the Db2 log records. Default is 0.
Row Flush Threshold	The maximum number of rows that can be written to the files that store data temporarily before the data is transferred to cloud storage. The data is flushed to cloud storage either when this number of rows is reached or when the flush interval expires. Default is 50000 rows.
Flush Interval	During periods of low change activity on the source, the number of minutes and seconds that a job waits for more change data before flushing the data in the temporary files to cloud storage. The data is flushed either when this interval expires or when the row flush threshold is met. Default is 30 seconds.
Log Start Point	The position in the source logs from which the CDC staging job starts reading change records the first time it runs. Options are: - Earliest Available - Latest Available - Specific Date and Time. The date and time you specify. Note: These options are similar to those described for Initial Start Point for Incremental Load in the source properties. Default is Latest Available.
Staging Data Retention Period	The number of days to retain data in cloud storage. Valid values are 0-365. Default is 14 days. After the retention period expires, the data is purged and is no longer available for subsequent restarts.

If you want to add source and target custom properties for the CDC staging jobs in the group, click the Custom Properties tab. Then enter each property name and value. Typically, you use custom properties only at the direction of Informatica Global Customer Support or Informatica technical staff.
When done, click Save to save the staging group and return to the Task Details, Step 1 page.
Note:
If you edit the staging group later from a deployed task in the group, you can change only the Row Flush Threshold, Flush Interval, Staging Data Retention Period, and custom property values.
5. 5Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available, depending on the load type:

Propery	Description
Read Event Batch Size	The number of payload events written in batch to the internal event queue during CDC processing. When the event queue is implemented as an internal ring buffer, this value is the number of payload events that the reader writes to a single internal buffer slot. Note: A batch size that's too small might increase contention between threads. A larger batch size can provide for more parallelism but consume more memory.
Reader Helper Thread Count	The number of reader helper threads used during CDC processing to convert change data into a canonical format that can be passed to the target. Default value is 3. You can enter a larger value to allow more threads to be available for performing conversion processing in parallel.
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal thread count. You can override the value from auto-tuning by setting this custom property.
Enable Partitioning	Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Select this property to use multiple partitions with one query per partition. By default, this property is not selected. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine whether to enable partitioning. You can override the value from auto-tuning by setting this custom property.
Unload Source Partition Count	The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected. Default is 5. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partition count. You can override the value from auto-tuning by setting this custom property.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
If you configured a CDC staging group, any custom properties that you enter here for the source are ignored. Instead, enter custom properties for the staging group.
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.

Source Tables

• •On the Selected Tables tab, individually select the check box for each source table you want to include. Clear the check box for any tables you do not want to include. To select all tables, select the Table check box at the top.
The Column count for a table shows the total number of columns in the table.
If you select the tables you want to include, all the selected and unselected tables are displayed by default. To view the selected tables only, use the filter next to the table selection count and change the view from All to Selected.
To find tables or columns, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns tables and columns that include CDC anywhere in their names. To narrow the search to only table names or column names, select Table Name or Columns in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for tables.
• - To add a table rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
Tips and guidelines for rules:
• - You can copy an existing table rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a table rule directly in its row by clicking the Type or Match Condition value.
• - To view the tables and columns that match a single table rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - The task wizard is case sensitive. Enter the table names or masks in the case with which they were defined.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a table name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.
Note:
If you define multiple table or column rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a table rule specifies "Exclude CDC" followed by "Include C", all tables with names beginning with "C" are selected, including the CDC tables.

Advanced Source Properties

1. 1Complete the following optional advanced properties as needed:

Property	Load Type	Description
CDC Script	Incremental loads and combined initial and incremental loads	Generate a script for enabling CDC on source tables and then run or download the script. The only available option is Enable CDC for all columns. For Db2 for z/OS sources, the script sets DATA CAPTURE CHANGES for source tables and certain Db2 catalog tables needed for CDC. After DATA CAPTURE CHANGES is set for one job, all other jobs recognize that the attribute is enabled in Db2 for the catalog tables needed because the Db2 catalog tables are a set of tables shared by all users of Db2. Click Execute to run the script if you have the required privileges. Or click the Download icon to download the script so that you can give it to your DBA to run.
List Tables by Rule Type	All	Generate and download a list of the source tables that match the table selection criteria. If you used rule-based table selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Select the Include Columns check box to include columns in the list, regardless of which table selection method you used. Click the Download icon to download the list.
Enable Persistent Storage	Incremental loads and combined initial and incremental loads Note: The property is not available if you are using a serverless runtime environment.	Select this check box to enable persistent storage of transaction data in a disk buffer so that the data can be consumed continually, even when the writing of data to the target is slow or delayed. Benefits of using persistent storage are faster consumption of the source transaction logs, less reliance on log archives or backups, and the ability to still access the data persisted in disk storage after restarting a database ingestion job. Persisted data is stored on the Secure Agent. It is not encrypted. The Secure Agent's files and directories are expected to be secured from unwanted access by using native file system access permissions or file system support of encryption natively. If you select this check box and also select Stage CDC Data, this check box is cleared and becomes unavailable.
Initial Start Point for Incremental Load	Incremental loads only	If you want to customize the position in the source logs from which the database ingestion and replication job starts reading change records the first time it runs, select one of the following options: - Latest Available. The latest available position in the database log or structure. - Position. The Db2 for z/OS LRSN in the change stream from which you want the database ingestion job to start retrieving change records. This value must be equal to or less than the current position value. An invalid value will cause the job to fail. The value of 0 is displayed by default, which causes the latest available point to be used. If you select the Stage CDC Data option, this option is not available. - Specific Date and Time. A date and time, in the format MM/DD/YYYY hh:mm AM|PM, that Database Ingestion and Replication uses to determine the position in the change stream from which to start retrieving change records. Database Ingestion and Replication retrieves only the changes that were started after this date and time. If you enter a date and time earlier than the earliest date and time in the available archived logs, the job will fail. If you select the Stage CDC Data option, this option is not available. The default is Latest Available. Note: - This initial start point setting pertains only to the initial run of a job. Thereafter, if you resume a stopped or aborted job, the job begins propagating source data from where it last left off. - If you select the Stage CDC Data option, this field shows only the Earliest Available and Latest Available options. - For combined initial and incremental load jobs, initial loading is not performed until the incremental processing of change data reaches the end of the current transaction log.
Stage CDC Data	Incremental loads and combined initial and incremental loads	If you want to read data from the source database in a single pass and then write the data to common storage so that that it can be read by multiple tasks that process the same database, select this check box. The staged data can then be read by the tasks that are in the staging group. For a log-based source, the tasks can process different tables with different schemas.
Staging Group	Incremental loads and combined initial and incremental loads	If you want to use a CDC staging group to stage CDC data for multiple tasks, either select a previously defined group from the list or click New to create one.

If you chose to create a new staging group, complete the following fields on the Overview tab of the New Staging Group dialog box:

Property	Description
Group Name	Accept the generated group name, which has the format Log_group_cdc_yyyymmdd<number>, or enter a custom name.
Location	The project in which you want to store the staging group definition, if you don't want to use the Default location.
Staging Location Connection	Select a connection to the cloud staging location, which can be in Amazon S3, Google Cloud Storage, or Microsoft Azure Data Lake Storage Gen2.
Runtime Environment	Select the runtime environment to use for running the CDC staging task. For a log-based source, ensure that the Secure Agent on which the task will run can access all of the source logs.
Enable Alternate Connection for Reading Logs	Do not select this option for Db2 for z/OS sources even though it's available.
Alternate Connection for Reading Logs	Available after you select Enable Alternate Connection for Reading Logs but does not pertain to Db2 for z/OS sources.
Include LOBs	Not available.
Row Flush Threshold	The maximum number of rows that can be written to the files that store data temporarily before the data is transferred to cloud storage. The data is flushed to cloud storage either when this number of rows is reached or when the flush interval expires. Default is 50000 rows.
Flush Interval	During periods of low change activity on the source, the number of minutes and seconds that a job waits for more change data before flushing the data in the temporary files to cloud storage. The data is flushed either when this interval expires or when the row flush threshold is met. Default is 30 seconds.
Log Start Point	The position in the source logs from which the CDC staging job starts reading change records the first time it runs. Options are: - Latest Available - Position.Then enter an appropirate position in the change stream for your source type. - Specific Date and Time. Then specify the date and time. Note: These options are similar to those described for Initial Start Point for Incremental Load in the source properties. Default is Latest Available.
Staging Data Retention Period	The number of days to retain data in cloud storage. Valid values are 0-365. Default is 14 days. After the retention period expires, the data is purged and is no longer available for subsequent restarts.

If you want to add source and target custom properties for the CDC staging jobs in the group, click the Custom Properties tab. Then enter each property name and value. Typically, you use custom properties only at the direction of Informatica Global Customer Support or Informatica technical staff.
When done, click Save to save the staging group and return to the Task Details, Step 1 page.
Note:
If you edit the staging group later from a deployed task in the group, you can change only the Row Flush Threshold, Flush Interval, Staging Data Retention Period, and custom property values.
2. 2Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available, depending on the load type:

Propery	Description
Read Event Batch Size	The number of payload events written in batch to the internal event queue during CDC processing. When the event queue is implemented as an internal ring buffer, this value is the number of payload events that the reader writes to a single internal buffer slot. Note: A batch size that's too small might increase contention between threads. A larger batch size can provide for more parallelism but consume more memory.
Reader Helper Thread Count	The number of reader helper threads used during CDC processing to convert change data into a canonical format that can be passed to the target. Default value is 3. You can enter a larger value to allow more threads to be available for performing conversion processing in parallel.
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal thread count. You can override the value from auto-tuning by setting this custom property.
Enable Partitioning	Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Select this property to use multiple partitions with one query per partition. By default, this property is not selected. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine whether to enable partitioning. You can override the value from auto-tuning by setting this custom property.
Unload Source Partition Count	The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected. Default is 5. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partition count. You can override the value from auto-tuning by setting this custom property.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
If you configured a CDC staging group, any custom properties that you enter here for the source are ignored. Instead, enter custom properties for the staging group.
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.

Source Tables

• •On the Selected Tables tab, individually select the check box for each source table and column you want to include. Clear the check box for any tables or columns you do not want to include. To select all tables and columns, select the Table check box at the top.
The Column count for a table shows the total number of columns in the table.
Note:
If you deselect one or more columns for a table, a minus (-) appears next to the table name in the list.
If you select the tables you want to include, all the selected and unselected tables are displayed by default. To view the selected tables only, use the filter next to the table selection count and change the view from All to Selected.
To find tables or columns, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns tables and columns that include CDC anywhere in their names. To narrow the search to only table names or column names, select Table Name or Columns in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for tables and columns.
• - To add a table rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
If you want to include database views as sources, select the Include Views check box to the left of the plus sign. This check box is available for initial load tasks.
• - To add a column rule, click the plus (+) sign at the end of the row at the table level. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
• - Validate the rule. The validation warns if your rule excludes a primary key column.
Tips and guidelines for rules:
• - You can copy an existing table rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a table or column rule directly in its row by clicking the Type or Match Condition value.
• - To view the tables and columns that match a single table rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - Enter the table or column names or masks in the case with which they were defined. The task wizard is case sensitive.
• - Key columns are selected by default and can't be excluded.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a table or column name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.
Note:
If you define multiple table or column rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a table rule specifies "Exclude CDC" followed by "Include C", all tables with names beginning with "C" are selected, including the CDC tables.

Advanced Source Properties

1. 1Under Change Data Capture Method, select one of the following CDC methods if you're defining an incremental load or combined initial and incremental load task:
• - CDC Tables. Read data changes directly from the SQL Server CDC tables.
This method is the default option for SQL Server because it provides the best replication performance and highest reliability of results.
• - Log-based. Capture Inserts, Updates, and Deletes in near real time by reading the database transaction logs. Also capture DDL changes based on schema drift settings.
For SQL Server sources, data changes are read from the SQL Server transaction log and the enabled SQL Server CDC tables. Exception: For Azure SQL Database sources, data changes are read from CDC tables only.
• - Query-based. Capture Inserts and Updates by using a SQL WHERE clause that points to a CDC query column. The query column is used to identify the rows that contain the changes made to the source tables since the beginning of the CDC interval.
Note:
If you are using a serverless runtime environment, you can use this option for a task that has an Amazon Redshift, Databricks, Google BigQuery, Microsoft Azure Synapse Analytics, Oracle, PostgreSQL, SQL Server, or Snowflake target. If you are using a non-serverless environment, you can select this option for a task with a Kafka target or with any supported cloud data warehouse, data lake, or database target type.
Note:
If you plan to use a CDC staging group to stage change data for multiple tasks in cloud storage, you must use the CDC Tables or Log-based method. You can override the CDC method when you define the staging group.
If you selected the Query-based option, also complete the following fields:
• - CDC Query Column Type. The column data type of the CDC query column in the source table. The column data type must combine the date and time. For SQL Server, the only available option is Datetime.
• - CDC Query Column Name. The case-sensitive name of the CDC query column in the source table. The column must be present in the source table. Maximum length is 70 characters.
• - CDC Interval. The frequency of a query-based change data capture cycle, expressed in days, hours, and minutes. You must enter a positive number in at least one of the interval fields. Otherwise, an error will be issued when you try to save the task. The default value is 5 minutes.
2. 2Complete the following optional advanced properties as needed:

Property	Load Type	Description
CDC Script	Incremental loads and combined initial and incremental loads that use the CDC Tables or Log-based method	Generate a script for enabling CDC on source tables and then run or download the script. Options are: - Enable CDC for all columns. Enables CDC for all columns in the selected source tables. You must use this option for a SQL Server source. Note: For source tables without a primary key, including any tables with unique indexes, CDC is enabled for all columns by default, regardless of which option is selected. - Enable CDC for primary key columns. Enables CDC only for primary key columns in the selected source tables. For Microsoft SQL Server sources, the script runs the sys.sp_cdc_enable_db and sys.sp_cdc_enable_table stored procedures to enable CDC on the source database and tables. For RDS for SQL Server, the script runs the msdb.dbo.rds_cdc_enable_db procedure to enable CDC on the source database, and runs the sys.sp_cdc_enable_table script to track CDC for tables. Click Execute to run the script if you have the required privileges. Or click the Download icon to download the script so that you can give it to your DBA to run. Note: This field is not displayed if you selected the Query-based CDC method:
Capture Filegroup	Incremental loads and combined initial and incremental loads	The name of the filegroup to be used for the change table that is created for the capture. If you leave this field empty, the change table is located in the default filegroup of the database.
Gating Role	Incremental loads and combined initial and incremental loads	The name of the database role that is used to gate access to change data. If you leave this field empty, the database does not use the gating role.
List Tables by Rule Type	All	Generate and download a list of the source tables that match the table selection criteria. If you used rule-based table selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Select the Include Columns check box to include columns in the list, regardless of which table selection method you used. Click the Download icon to download the list.
Include LOBs	Initial loads to Amazon Redshift, Amazon S3, Databricks, Google BigQuery, Google Cloud Storage, Microsoft Azure Data Lake Storage Gen 2, Microsoft Azure Synapse Analytics, Microsoft Fabric OneLake, Oracle, Oracle Cloud Object Storage, PostgreSQL, Snowflake, or SQL Server targets. Incremental loads to Kafka-enabled Azure Event Hubs, Databricks, Microsoft Fabric OneLake, PostgreSQL, Snowflake, and SQL Server targets. Disabled if you selected the Query-based CDC method. Combined initial and incremental loads to Databricks, Microsoft Fabric OneLake, PostgreSQL, Snowflake, and SQL Server targets. Disabled if you selected the Query-based CDC method.	Select this check box if the source contains the large-object (LOB) columns from which you want to replicate data to a target. LOB data types for SQL Server sources: GEOGRAPHY, GEOMETRY, IMAGE, NTEXT, NVARCHAR(MAX), TEXT, VARBINARY(MAX), VARCHAR(MAX), and XML LOB data might be truncated on the target. For more information, see About LOB truncation.
Enable Persistent Storage	Incremental loads and combined initial and incremental loads. Note: The property is not available if you are using a serverless runtime environment. For SQL Server sources that use the query-based CDC method, this field is not displayed because persistent storage is enabled by default and cannot be changed.	Select this check box to enable persistent storage of transaction data in a disk buffer so that the data can be consumed continually, even when the writing of data to the target is slow or delayed. Benefits of using persistent storage are faster consumption of the source transaction logs, less reliance on log archives or backups, and the ability to still access the data persisted in disk storage after restarting a database ingestion job. Persisted data is stored on the Secure Agent. It is not encrypted. The Secure Agent's files and directories are expected to be secured from unwanted access by using native file system access permissions or file system support of encryption natively. If you select this check box and also select Stage CDC Data, this check box is cleared and becomes unavailable.
Initial Start Point for Incremental Load	Incremental loads only	If you want to customize the position in the source logs from which the database ingestion and replication job starts reading change records the first time it runs, select one of the following options: - Earliest Available. The earliest available position in the database log or structure where changes are stored. For SQL Server, the earliest available record in the active transaction log. This option is not available if the CDC Method is set to Query-based. - Latest Available. The latest available position in the database log or structure. - Position. The SQL Server LSN in the change stream from which you want the database ingestion job to start retrieving change records. This value must be equal to or less than the current position value. An invalid value will cause the job to fail. The value of 0 is displayed by default, which causes the earliest available point to be used. A non-zero LSN that predates the beginning of the active transaction log causes data to be read from the CDC tables instead of from the transaction log. This option is not available if the CDC Method is set to Query-based. If you select the Stage CDC Data option, this option is not available. - Specific Date and Time. A date and time, in the format MM/DD/YYYY hh:mm AM|PM, that Database Ingestion and Replication uses to determine the position in the change stream from which to start retrieving change records. Database Ingestion and Replication retrieves only the changes that were started after this date and time. If you enter a date and time earlier than the earliest date and time in the available archived logs, the job will fail. If you select the Stage CDC Data option, this option is not available. The default is Latest Available. Note: - This initial start point setting pertains only to the initial run of a job. Thereafter, if you resume a stopped or aborted job, the job begins propagating source data from where it last left off. - If you select the Stage CDC Data option, this field shows only the Earliest Available and Latest Available options. - For SQL Server incremental load jobs that use log-based CDC with the transaction log, data changes are read from the active portion of the transaction log if the requested LSN is available there. If the LSN predates the active transaction log, the data changes are read from previously enabled CDC tables. Ensure that SQL Server CDC is enabled on the source tables. - For combined initial and incremental load jobs, initial loading is not performed until the incremental processing of change data reaches the end of the current transaction log.
Stage CDC Data	Incremental loads and combined initial and incremental loads	If you want to read data from the source database in a single pass and then write the data to common storage so that that it can be read by multiple tasks that process the same database, select this check box. The staged data can then be read by the tasks that are in the staging group. For a log-based source, the tasks can process different tables with different schemas.
Staging Group	Incremental loads and combined initial and incremental loads	If you want to use a CDC staging group to stage CDC data for multiple tasks, either select a previously defined group from the list or click New to create one.

If you chose to create a new staging group, complete the following fields on the Overview tab of the New Staging Group dialog box:

Property	Description
Group Name	Accept the generated group name, which has the format Log_group_cdc_yyyymmdd<number>, or enter a custom name.
Location	The project in which you want to store the staging group definition, if you don't want to use the Default location.
Staging Location Connection	Select a connection to the cloud staging location, which can be in Amazon S3, Google Cloud Storage, or Microsoft Azure Data Lake Storage Gen2.
Runtime Environment	Select the runtime environment to use for running the CDC staging task. For a log-based source, ensure that the Secure Agent on which the task will run can access all of the source logs.
Enable Alternate Connection for Reading Logs	Select this check box if you want to use a source connection other than the one selected on the Source page to read data from source logs. For example, if you perform initial loads and CDC, you might want to select a connection with additional privileges required for CDC.
Alternate Connection for Reading Logs	Select the alternate connection for reading logs.
CDC Method	For the tasks in the staging group, you can use either CDC Tables or Log-based method for a SQL Server source. You cannot use the Query-based method. If you previously set a CDC Method under source properties, you can use this field to override it for the CDC staging job.
Include LOBs	Select this check box if you want all CDC tasks in the staging group to capture and stage LOB data for their tables that are in the source database. Note: If you previously selected this option under Advanced source properties, initial load jobs will read the LOB data too. If you want to enable LOB data capture after running the CDC staging task, either Redeploy the task or create another staging group with this option enabled.
Row Flush Threshold	The maximum number of rows that can be written to the files that store data temporarily before the data is transferred to cloud storage. The data is flushed to cloud storage either when this number of rows is reached or when the flush interval expires. Default is 50000 rows.
Flush Interval	During periods of low change activity on the source, the number of minutes and seconds that a job waits for more change data before flushing the data in the temporary files to cloud storage. The data is flushed either when this interval expires or when the row flush threshold is met. Default is 30 seconds.
Log Start Point	The position in the source logs from which the CDC staging job starts reading change records the first time it runs. Options are: - Earliest Available. - Latest Available - Position.Then enter an appropirate position in the change stream for your source type. - Specific Date and Time. Then specify the date and time. Note: These options are similar to those described for Initial Start Point for Incremental Load in the source properties. Default is Latest Available.
Staging Data Retention Period	The number of days to retain data in cloud storage. Valid values are 0-365. Default is 14 days. After the retention period expires, the data is purged and is no longer available for subsequent restarts.

If you want to add source and target custom properties for the CDC staging jobs in the group, click the Custom Properties tab. Then enter each property name and value. Typically, you use custom properties only at the direction of Informatica Global Customer Support or Informatica technical staff.
When done, click Save to save the staging group and return to the Task Details, Step 1 page.
Note:
If you edit the staging group later from a deployed task in the group, you can change only the Row Flush Threshold, Flush Interval, Staging Data Retention Period, and custom property values.
3. 3Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available, depending on the load type:

Property	Description
Read Event Batch Size	The number of payload events written in batch to the internal event queue during CDC processing. When the event queue is implemented as an internal ring buffer, this value is the number of payload events that the reader writes to a single internal buffer slot. Note: A batch size that's too small might increase contention between threads. A larger batch size can provide for more parallelism but consume more memory.
Reader Helper Thread Count	The number of reader helper threads used during CDC processing to convert change data into a canonical format that can be passed to the target. Default value is 3. You can enter a larger value to allow more threads to be available for performing conversion processing in parallel.
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal thread count. You can override the value from auto-tuning by setting this custom property.
Unload JDBC Partitioning Technique	The technique to use for partitioning source table rows for initial load processing or the unload phase of a combined load job. Valid values are: - uniform. Select all of the primary key values from the source table and distribute those keys across the partitions evenly, producing partitions of equal or near equal size. - heuristic. Estimate the primary key ranges to assign to each partition by calculating the difference between the low and high primary key values and using the number of partitions requested. With this technique, ranges are calculated more quickly because the query to select primary key values is avoided. However, the sizes of the partitions might vary widely if primary keys are not distributed evenly between high and low values. For example, if 26 partitions are requested and the low primary key starts with A and the high primary key starts with Z, you might expect one partition for each set of rows with a primary key from A through Z. However, if 90% of the primary keys start with Q, R,or S, 90% of the rows are assigned to just three partitions. Default value is uniform. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partitioning technique. You can override the value from auto-tuning by setting this custom property.
Unload Source Max Parallel Partition	The maximum number of partition reader threads that can be used to query the source in parallel during initial load processing or the unload phase of combined jobs. Use this property to control the number of source partition queries that can be executed against the source at the same time. For example, if a table contains data in 100 partitions, all 100 partitions are queried at the same time by default. However, you can use this property to reduce the number of concurrent queries. Default value is equal to the total number of partitions. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal number of partition reader threads. You can override the value from auto-tuning by setting this custom property.
Enable Partitioning	Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Select this property to use multiple partitions with one query per partition. By default, this property is not selected. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine whether to enable partitioning. You can override the value from auto-tuning by setting this custom property.
Unload Source Partition Count	The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected. Default is 5. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partition count. You can override the value from auto-tuning by setting this custom property.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
If you configured a CDC staging group, any custom properties that you enter here for the source are ignored. Instead, enter custom properties for the staging group.
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.

Source Collections

• •On the Selected Collections tab, individually select the check box for each source collection you want to include. Clear the check box for any collections you do not want to include. To select all collections, select the Collection check box at the top.
The Field count for a collection shows the total number of fields in the collection.
If you select the collections you want to include, all the selected and unselected collections are displayed by default. To view the selected collections only, use the filter next to the collection selection count and change the view from All to Selected.
To find collections or fields, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns collections and fields that include CDC anywhere in their names. To narrow the search to only collection names or field names, select Collection Name or Fields in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for collections.
• - To add a collection rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
Tips and guidelines for rules:
• - You can copy an existing collection rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a collection rule directly in its row by clicking the Type or Match Condition value.
• - To view the collections and fields that match a single collection rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - The task wizard is case sensitive. Enter the collection names or masks in the case with which they were defined.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a collection name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.
Note:
If you define multiple collection rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a collection rule specifies "Exclude CDC" followed by "Include C", all collections with names beginning with "C" are selected, including the CDC collections.

Advanced Source Properties

1. 1Complete the following optional advanced properties as needed:

Property	Load Type	Description
List Collections by Rule Type	All	Generate and download a list of the source collections that match the collection selection criteria. If you used rule-based collection selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Click the Download icon to download the list.
Fetch Size	All	The number of records that a database ingestion and replication job must read at a single time from the MongoDB source. Valid values are 1 to 2147483647. The default is 5000.
Initial Start Point for Incremental Load	Incremental loads only	If you want to customize the position in the source logs from which the database ingestion and replication job starts reading change records the first time it runs, select one of the following options: - Latest Available. The latest available position in the database log or structure. - Specific Date and Time. A date and time, in the format MM/DD/YYYY hh:mm AM|PM, that Database Ingestion and Replication uses to determine the position in the change stream from which to start retrieving change records. Database Ingestion and Replication retrieves only the changes that were started after this date and time. If you enter a date and time earlier than the earliest date and time in the available archived logs, the job will fail. The default is Latest Available. Note: - This initial start point setting pertains only to the initial run of a job. Thereafter, if you resume a stopped or aborted job, the job begins propagating source data from where it last left off. - For combined initial and incremental load jobs, initial loading is not performed until the incremental processing of change data reaches the end of the current transaction log.

2. 2Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available, depending on the load type:

Propery	Description
Read Event Batch Size	The number of payload events written in batch to the internal event queue during CDC processing. When the event queue is implemented as an internal ring buffer, this value is the number of payload events that the reader writes to a single internal buffer slot. Note: A batch size that's too small might increase contention between threads. A larger batch size can provide for more parallelism but consume more memory.
Reader Helper Thread Count	The number of reader helper threads used during CDC processing to convert change data into a canonical format that can be passed to the target. Default value is 3. You can enter a larger value to allow more threads to be available for performing conversion processing in parallel.
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.

Source Tables

• •On the Selected Tables tab, individually select the check box for each source table you want to include. Clear the check box for any tables you do not want to include. To select all tables, select the Table check box at the top.
The Column count for a table shows the total number of columns in the table.
If you select the tables you want to include, all the selected and unselected tables are displayed by default. To view the selected tables only, use the filter next to the table selection count and change the view from All to Selected.
To find tables or columns, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns tables and columns that include CDC anywhere in their names. To narrow the search to only table names or column names, select Table Name or Columns in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for tables.
• - To add a table rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
If you want to include database views as sources, select the Include Views check box to the left of the plus sign. This check box is available for initial load tasks.
Tips and guidelines for rules:
• - You can copy an existing table rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a table rule directly in its row by clicking the Type or Match Condition value.
• - To view the tables and columns that match a single table rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - The task wizard is case sensitive. Enter the table names or masks in the case with which they were defined.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a table name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.
Note:
If you define multiple table rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a table rule specifies "Exclude CDC" followed by "Include C", all tables with names beginning with "C" are selected, including the CDC tables.

Advanced Source Properties

1. 1Complete the following optional advanced properties as needed:

Property	Load Type	Description
List Tables by Rule Type	All	Generate and download a list of the source tables that match the table selection criteria. If you used rule-based table selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Select the Include Columns check box to include columns in the list, regardless of which table selection method you used. Click the Download icon to download the list.
Include LOBs	All	Select this check box if the source contains columns with large-object (LOB) data types from which you want to replicate data to the target. LOB data types for a MySQL source are: BLOB, LONGBLOB, MEDIUMBLOB, TINYBLOB, JSON, TEXT, LONGTEXT, MEDIUMTEXT, and TINYTEXT. LOB data might be truncated on the target. For more information, see About LOB truncation. If you do not select this option and the source contains LOB columns, corresponding LOB columns are still be created on the target but nulls are replicated to them.
Enable Persistent Storage	Incremental loads and combined initial and incremental loads Note: The property is not available if you are using a serverless runtime environment.	Select this check box to enable persistent storage of transaction data in a disk buffer so that the data can be consumed continually, even when the writing of data to the target is slow or delayed. Benefits of using persistent storage are faster consumption of the source transaction logs, less reliance on log archives or backups, and the ability to still access the data persisted in disk storage after restarting a database ingestion job. Persisted data is stored on the Secure Agent. It is not encrypted. The Secure Agent's files and directories are expected to be secured from unwanted access by using native file system access permissions or file system support of encryption natively.
Initial Start Point for Incremental Load	Incremental loads only	If you want to customize the position in the source logs from which the database ingestion and replication job starts reading change records the first time it runs, select one of the following options: - Earliest Available. The earliest available position in the database log or structure where changes are stored. For MySQL, the earliest available record in the first binlog file. - Latest Available. The latest available position in the database log or structure. The default is Latest Available. Note: - This initial start point setting pertains only to the initial run of a job. Thereafter, if you resume a stopped or aborted job, the job begins propagating source data from where it last left off. - For combined initial and incremental load jobs, initial loading is not performed until the incremental processing of change data reaches the end of the current transaction log.

2. 2Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available, depending on the load type:

Propery	Description
Read Event Batch Size	The number of payload events written in batch to the internal event queue during CDC processing. When the event queue is implemented as an internal ring buffer, this value is the number of payload events that the reader writes to a single internal buffer slot. Note: A batch size that's too small might increase contention between threads. A larger batch size can provide for more parallelism but consume more memory.
Reader Helper Thread Count	The number of reader helper threads used during CDC processing to convert change data into a canonical format that can be passed to the target. Default value is 3. You can enter a larger value to allow more threads to be available for performing conversion processing in parallel.
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal thread count. You can override the value from auto-tuning by setting this custom property.
Enable Partitioning	Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Select this property to use multiple partitions with one query per partition. By default, this property is not selected. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine whether to enable partitioning. You can override the value from auto-tuning by setting this custom property.
Unload Source Partition Count	The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected. Default is 5. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partition count. You can override the value from auto-tuning by setting this custom property.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.

Source Tables

• •On the Selected Tables tab, individually select the check box for each source table you want to include. Clear the check box for any tables you do not want to include. To select all tables, select the Table check box at the top.
The Column count for a table shows the total number of columns in the table.
If you select the tables you want to include, all the selected and unselected tables are displayed by default. To view the selected tables only, use the filter next to the table selection count and change the view from All to Selected.
To find tables or columns, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns tables and columns that include CDC anywhere in their names. To narrow the search to only table names or column names, select Table Name or Columns in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for tables.
• - To add a table rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
Tips and guidelines for rules:
• - You can copy an existing table rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a table rule directly in its row by clicking the Type or Match Condition value.
• - To view the tables and columns that match a single table rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - The task wizard is case sensitive. Enter the table names or masks in the case with which they were defined.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a table name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.
Note:
If you define multiple table rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a table rule specifies "Exclude CDC" followed by "Include C", all tables with names beginning with "C" are selected, including the CDC tables.

Advanced Source Properties

1. 1Complete the following optional advanced properties as needed:

Property	Load Type	Description
List Tables by Rule Type	All	Generate and download a list of the source tables that match the table selection criteria. If you used rule-based table selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Select the Include Columns check box to include columns in the list, regardless of which table selection method you used. Click the Download icon to download the list.

2. 2Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available for initial load jobs with this source:

Property	Description
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time.
Enable Partitioning	Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Select this property to use multiple partitions with one query per partition. By default, this property is not selected.
Unload Source Partition Count	The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected. Default is 5.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.

Source Tables

• •On the Selected Tables tab, individually select the check box for each source table and column you want to include. Clear the check box for any tables or columns you do not want to include. To select all tables and columns, select the Table check box at the top.
The Column count for a table shows the total number of columns in the table.
Note:
If you deselect one or more columns for a table, a minus (-) appears next to the table name in the list.
If you select the tables you want to include, all the selected and unselected tables are displayed by default. To view the selected tables only, use the filter next to the table selection count and change the view from All to Selected.
To find tables or columns, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns tables and columns that include CDC anywhere in their names. To narrow the search to only table names or column names, select Table Name or Columns in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for tables and columns.
• - To add a table rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
If you want to include database views as sources, select the Include Views check box to the left of the plus sign. This check box is available for initial load tasks.
• - To add a column rule, click the plus (+) sign at the end of the row at the table level. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
• - Validate the rule. The validation warns if your rule excludes a primary key column.
Note:
If you define multiple table or column rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a table rule specifies "Exclude CDC" followed by "Include C", all tables with names beginning with "C" are selected, including the CDC tables.
Tips and guidelines for rules:
• - You can copy an existing table rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a table or column rule directly in its row by clicking the Type or Match Condition value.
• - To view the tables and columns that match a single table rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - The task wizard is case sensitive. Enter the table or column names or masks in the case with which they were defined.
• - Key columns are selected by default and can't be excluded.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a table or column name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.

Advanced Source Properties

1. 1Under Change Data Capture Method, select one of the following CDC methods if you're defining an incremental load or combined initial and incremental load task:
• - Log-based. Capture Inserts, Updates, and Deletes in near real time by reading the database transaction logs. Also capture DDL changes based on schema drift settings.
For Oracle sources, data changes are read from the Oracle redo logs.
Note:
If you plan to use a CDC staging group to stage change data for multiple tasks in cloud storage, you must use the Log-based method.
• - Query-based . Capture Inserts and Updates by using a SQL WHERE clause that points to a CDC query column. The query column is used to identify the rows that contain the changes made to the source tables since the beginning of the CDC interval.
Note:
If you are using a serverless runtime environment, you can use this option for a task that has an Amazon Redshift, Databricks, Google BigQuery, Microsoft Azure Synapse Analytics, Oracle, PostgreSQL, SQL Server, or Snowflake target. If you are using a non-serverless environment, you can select this option for a task with a Kafka target or with any supported cloud data warehouse, data lake, or database target type.
If you selected the Query-based option, complete the following fields:
• - CDC Query Column Type. The column data type of the CDC query column in the source table. The column data type must combine the date and time. For Oracle, the only available option is Timestamp.
• - CDC Query Column Name. The case-sensitive name of the CDC query column in the source table. The column must be present in the source table. Maximum length is 70 characters.
• - CDC Interval. The frequency of a query-based change data capture cycle, expressed in days, hours, and minutes. You must enter a positive number in at least one of the interval fields. Otherwise, an error will be issued when you try to save the task. The default value is 5 minutes.
2. 2Complete the following optional advanced properties as needed:

Property	Load Type	Description
CDC Script	Incremental loads and combined initial and incremental loads that use the Log-based method	Generate a script for enabling Oracle supplemental logging on the source tables to log additional information, which is required for CDC, in the redo log. You can then run or download the script. This field is displayed for the Log-based CDC method only. Options are: - Enable CDC for all columns. Enables supplemental logging for all columns in the selected source tables. This option provides the most accurate support for CDC across all target types and configurations. Note: For source tables without a primary key, including any tables with unique indexes, supplemental logging is enabled for all columns by default, regardless of which option is selected. - Enable CDC for primary key columns. Enables supplemental logging only for primary key columns in the selected source tables. For Update operations, the jobs replicate values only for the primary key columns and the columns updated as part of the DML Update. The jobs write nulls for all other columns, which can cause unexpected results for downstream consumers when the columns actually have not-null values. Note: Do not use this option if you have a Google BigQuery target. To run the script from this wizard, page click Execute. You must have the privileges required to run the script against the database. Otherwise, click the Download icon to download the script so that you can give it to your DBA to run.
List Tables by Rule Type	All	Generate and download a list of the source tables that match the table selection criteria. If you used rule-based table selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Select the Include Columns check box to include columns in the list, regardless of which table selection method you used. Click the Download icon to download the list.
Disable Flashback	Initial loads	Select this check box to disable Database Ingestion and Replication use of Oracle Flashback when fetching data from the database. This check box is selected by default for new initial load tasks. For existing initial load tasks, this check box is cleared by default, which causes Oracle Flashback to remain enabled. For tasks that have partitioning enabled, this check box is automatically selected and unavailable for editing.
Include LOBs	All load types to Amazon Redshift, Amazon S3, Databricks, Google BigQuery, Google Cloud Storage, Microsoft Azure Data Lake Storage Gen 2, Microsoft Azure Synapse Analytics, Microsoft Fabric OneLake, Oracle, Oracle Cloud Object Storage, PostgreSQL, Snowflake, or SQL Server targets. Incremental loads and combined loads can use either the Log-based or Query-based CDC method. However, jobs that use the Log-based CDC method do not replicate data from LONG, LONG RAW, and XML columns to the generated target columns.	Select this check box if the source contains the large-object (LOB) columns from which you want to replicate data to a target. LOB data types for Oracle sources: BLOB, CLOB, NCLOB, LONG, LONG RAW, and XML LOB data might be truncated on the target. For more information, see About LOB truncation.
Enable Persistent Storage	Incremental loads and combined initial and incremental loads. Note: This property is not available if you are using a serverless runtime environment. For sources that use the query-based CDC method, this field is not displayed because persistent storage is enabled by default and cannot be changed.	Select this check box to enable persistent storage of transaction data in a disk buffer so that the data can be consumed continually, even when the writing of data to the target is slow or delayed. Benefits of using persistent storage are faster consumption of the source transaction logs, less reliance on log archives or backups, and the ability to still access the data persisted in disk storage after restarting a database ingestion job. Persisted data is stored on the Secure Agent. It is not encrypted. The Secure Agent's files and directories are expected to be secured from unwanted access by using native file system access permissions or file system support of encryption natively. If you select this check box and also select Stage CDC Data, this check box is cleared and becomes unavailable.
Initial Start Point for Incremental Load	Incremental loads only	If you want to customize the position in the source logs from which the database ingestion and replication job starts reading change records the first time it runs, select one of the following options: - Earliest Available. The earliest available position in the database log or structure where changes are stored. For Oracle, this option is not available unless you select Stage CDC Data. - Latest Available. The latest available position in the database log or structure. - Position. An Oracle SCN in the change stream from which you want the database ingestion job to start retrieving change records. This value must be equal to or less than the current position value. An invalid value will cause the job to fail. The value of 0 is displayed by default, which causes the latest available point to be used. This option is not available if the CDC Method is set to Query-based. If you select the Stage CDC Data option, this option is not available. - Specific Date and Time. A date and time, in the format MM/DD/YYYY hh:mm AM|PM, that Database Ingestion and Replication uses to determine the position in the change stream from which to start retrieving change records. Database Ingestion and Replication retrieves only the changes that were started after this date and time. If you enter a date and time earlier than the earliest date and time in the available archived logs, the job will fail. If you select the Stage CDC Data option, this option is not available. The default is Latest Available. Note: - This initial start point setting pertains only to the initial run of a job. Thereafter, if you resume a stopped or aborted job, the job begins propagating source data from where it last left off. - If you select the Stage CDC Data option, this field shows only the Earliest Available and Latest Available options. - For combined initial and incremental load jobs, initial loading is not performed until the incremental processing of change data reaches the end of the current transaction log.
Stage CDC Data	Incremental loads and combined initial and incremental loads	If you want to read data from the source database in a single pass and then write the data to common storage so that that it can be read by multiple tasks that process the same database, select this check box. The staged data can then be read by the tasks that are in the staging group. For a log-based source, the tasks can process different tables with different schemas.
Staging Group	Incremental loads and combined initial and incremental loads	If you want to use a CDC staging group to stage CDC data for multiple tasks, either select a previously defined group from the list or click New to create one.

If you chose to create a new staging group, complete the following fields on the Overview tab of the New Staging Group dialog box:

Property	Description
Group Name	Accept the generated group name, which has the format Log_group_cdc_yyyymmdd<number>, or enter a custom name.
Location	The project in which you want to store the staging group definition, if you don't want to use the Default location.
Staging Location Connection	Select a connection to the cloud staging location, which can be in Amazon S3, Google Cloud Storage, or Microsoft Azure Data Lake Storage Gen2.
Runtime Environment	Select the runtime environment to use for running the CDC staging task. For a log-based source, ensure that the Secure Agent on which the task will run can access all of the source logs.
Enable Alternate Connection for Reading Logs	Select this check box if you want to use a source connection other than the one selected on the Source page to read data from source logs. For example, if you perform initial loads and CDC, you might want to select a connection with additional privileges required for CDC.
Alternate Connection for Reading Logs	Select the alternate connection for reading logs.
CDC Method	The only option is Log-based.
Include LOBs	Select this check box if you want all CDC tasks in the staging group to capture and stage LOB data for their tables that are in the source database. Note: If you previously selected this option under Advanced source properties, initial load jobs will read the LOB data too. If you want to enable LOB data capture after running the CDC staging task, either Redeploy the task or create another staging group with this option enabled.
Row Flush Threshold	The maximum number of rows that can be written to the files that store data temporarily before the data is transferred to cloud storage. The data is flushed to cloud storage either when this number of rows is reached or when the flush interval expires. Default is 50000 rows.
Flush Interval	During periods of low change activity on the source, the number of minutes and seconds that a job waits for more change data before flushing the data in the temporary files to cloud storage. The data is flushed either when this interval expires or when the row flush threshold is met. Default is 30 seconds.
Log Start Point	The position in the source logs from which the CDC staging job starts reading change records the first time it runs. Options are: - Latest Available - Position.Then enter an appropirate position in the change stream for your source type. - Specific Date and Time. Then specify the date and time. Note: These options are similar to those described for Initial Start Point for Incremental Load in the source properties. Default is Latest Available.
Staging Data Retention Period	The number of days to retain data in cloud storage. Valid values are 0-365. Default is 14 days. After the retention period expires, the data is purged and is no longer available for subsequent restarts.

If you want to add source and target custom properties for the CDC staging jobs in the group, click the Custom Properties tab. Then enter each property name and value. Typically, you use custom properties only at the direction of Informatica Global Customer Support or Informatica technical staff.
When done, click Save to save the staging group and return to the Task Details, Step 1 page.
Note:
If you edit the staging group later from a deployed task in the group, you can change only the Row Flush Threshold, Flush Interval, Staging Data Retention Period, and custom property values.
3. 3Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available, depending on the load type:

Property	Description
Read Event Batch Size	The number of payload events written in batch to the internal event queue during CDC processing. When the event queue is implemented as an internal ring buffer, this value is the number of payload events that the reader writes to a single internal buffer slot. Note: A batch size that's too small might increase contention between threads. A larger batch size can provide for more parallelism but consume more memory.
Reader Helper Thread Count	The number of reader helper threads used during CDC processing to convert change data into a canonical format that can be passed to the target. Default value is 3. You can enter a larger value to allow more threads to be available for performing conversion processing in parallel.
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal thread count. You can override the value from auto-tuning by setting this custom property.
Unload JDBC Partitioning Technique	The technique to use for partitioning source table rows for initial load processing or the unload phase of a combined load job. Valid values are: - uniform. Select all of the primary key values from the source table and distribute those keys across the partitions evenly, producing partitions of equal or near equal size. - heuristic. Estimate the primary key ranges to assign to each partition by calculating the difference between the low and high primary key values and using the number of partitions requested. With this technique, ranges are calculated more quickly because the query to select primary key values is avoided. However, the sizes of the partitions might vary widely if primary keys are not distributed evenly between high and low values. For example, if 26 partitions are requested and the low primary key starts with A and the high primary key starts with Z, you might expect one partition for each set of rows with a primary key from A through Z. However, if 90% of the primary keys start with Q, R,or S, 90% of the rows are assigned to just three partitions. Default value is uniform. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partitioning technique. You can override the value from auto-tuning by setting this custom property.
Unload Source Max Parallel Partition	The maximum number of partition reader threads that can be used to query the source in parallel during initial load processing or the unload phase of combined jobs. Use this property to control the number of source partition queries that can be executed against the source at the same time. For example, if a table contains data in 100 partitions, all 100 partitions are queried at the same time by default. However, you can use this property to reduce the number of concurrent queries. Default value is equal to the total number of partitions. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal number of partition reader threads. You can override the value from auto-tuning by setting this custom property.
Enable Partitioning	Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Select this property to use multiple partitions with one query per partition. By default, this property is not selected. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine whether to enable partitioning. You can override the value from auto-tuning by setting this custom property.
Unload Source Partition Count	The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected. Default is 5. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partition count. You can override the value from auto-tuning by setting this custom property.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
If you configured a CDC staging group, any custom properties that you enter here for the source are ignored. Instead, enter custom properties for the staging group.
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.

Source Tables

• •On the Selected Tables tab, individually select the check box for each source table and column you want to include. Clear the check box for any tables or columns you do not want to include. To select all tables and columns, select the Table check box at the top.
The Column count for a table shows the total number of columns in the table.
Note:
If you deselect one or more columns for a table, a minus (-) appears next to the table name in the list.
If you select the tables you want to include, all the selected and unselected tables are displayed by default. To view the selected tables only, use the filter next to the table selection count and change the view from All to Selected.
To find tables or columns, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns tables and columns that include CDC anywhere in their names. To narrow the search to only table names or column names, select Table Name or Columns in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for tables and columns.
• - To add a table rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
If you want to include database views as sources, select the Include Views check box to the left of the plus sign. This check box is available for initial load tasks.
• - To add a column rule, click the plus (+) sign at the end of the row at the table level. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
• - Validate the rule. The validation warns if your rule excludes a primary key column.
Note:
If you define multiple table or column rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a table rule specifies "Exclude CDC" followed by "Include C", all tables with names beginning with "C" are selected, including the CDC tables.
Tips and guidelines for rules:
• - You can copy an existing table rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a table or column rule directly in its row by clicking the Type or Match Condition value.
• - To view the tables and columns that match a single table rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - The task wizard is case sensitive. Enter the table or column names or masks in the case with which they were defined.
• - Key columns are selected by default and can't be excluded.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a table or column name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.

Advanced Source Properties

1. 1Complete the following optional advanced properties as needed:

Property	Load Type	Description
CDC Script	Incremental loads and conbined initial and incremental loads	Generate a script for enabling CDC on source tables and then run or download the script. The only available option is Enable CDC for all columns. For PostgreSQL sources, the script sets REPLICATION IDENTITY FULL on the selected source tables to write all column values to the WAL file. Also creates a replication slot of the type of pgoutput or wal2json. If the slot type is pgoutput, the script also creates the publication and adds tables to it. Click Execute to run the script if you have the required privileges. Or click the Download icon to download the script so that you can give it to your DBA to run.
List Tables by Rule Type	All	Generate and download a list of the source tables that match the table selection criteria. If you used rule-based table selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Select the Include Columns check box to include columns in the list, regardless of which table selection method you used. Click the Download icon to download the list.
Include LOBs	Initial loads to Amazon Redshift, Amazon S3, Databricks, Google BigQuery, Google Cloud Storage, Microsoft Azure Data Lake Storage Gen 2, Microsoft Azure Synapse Analytics, Microsoft Fabric OneLake, Oracle, Oracle Cloud Object Storage, PostgreSQL, Snowflake, or SQL Server targets. Incremental loads to Amazon Redshift, Amazon S3, Databricks, Google BigQuery, Google Cloud Storage, Kafka-enabled Azure Event Hubs, Microsoft Azure Data Lake Storage Gen 2, Microsoft Azure Synapse Analytics, Microsoft Fabric OneLake, Oracle, Oracle Cloud Object Storage, PostgreSQL, Snowflake, or SQL Server targets. Combined initial and incremental loads to Amazon Redshift, Amazon S3, Databricks, Google BigQuery, Google Cloud Storage, Microsoft Azure Data Lake Storage Gen 2, Microsoft AzureSynapse Analytics, Microsoft Fabric OneLake, Oracle, Oracle Cloud Object Storage, PostgreSQL, Snowflake, or SQL Server targets.	Select this check box if the source contains the large-object (LOB) columns from which you want to replicate data to a target. LOB data types for PostgreSQL sources: BYTEA, TEXT, and XML plus some other potentially large types such as JSON, JSONB LOB data might be truncated on the target. For more information, see About LOB truncation.
Initial Start Point for Incremental Load	Incremental loads only	If you want to customize the position in the source logs from which the database ingestion and replication job starts reading change records the first time it runs, select one of the following options: - Earliest Available. The earliest available position in the database log or structure where changes are stored. For PostgreSQL, the earliest available record in the replication slot. - Latest Available. The latest available position in the database log or structure. - Position. A PostgreSQL LSN in the change stream from which you want the database ingestion job to start retrieving change records. This value must be equal to or less than the current position value. An invalid value will cause the job to fail. The value of 0 is displayed by default, which causes the earliest available point to be used. If you select Stage CDC Data, this option is not available unless you have a SAP HANA source and use the Log-based CDC method. - Specific Date and Time. A date and time, in the format MM/DD/YYYY hh:mm AM|PM, that Database Ingestion and Replication uses to determine the position in the change stream from which to start retrieving change records. Database Ingestion and Replication retrieves only the changes that were started after this date and time. If you enter a date and time earlier than the earliest date and time in the available archived logs, the job will fail. The default is Latest Available. Note: - This initial start point setting pertains only to the initial run of a job. Thereafter, if you resume a stopped or aborted job, the job begins propagating source data from where it last left off. - For combined initial and incremental load jobs, initial loading is not performed until the incremental processing of change data reaches the end of the current transaction log.

2. 2Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available, depending on the load type:

Propery	Description
Read Event Batch Size	The number of payload events written in batch to the internal event queue during CDC processing. When the event queue is implemented as an internal ring buffer, this value is the number of payload events that the reader writes to a single internal buffer slot. Note: A batch size that's too small might increase contention between threads. A larger batch size can provide for more parallelism but consume more memory.
Reader Helper Thread Count	The number of reader helper threads used during CDC processing to convert change data into a canonical format that can be passed to the target. Default value is 3. You can enter a larger value to allow more threads to be available for performing conversion processing in parallel.
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal thread count. You can override the value from auto-tuning by setting this custom property.
Enable Partitioning	Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Select this property to use multiple partitions with one query per partition. By default, this property is not selected. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine whether to enable partitioning. You can override the value from auto-tuning by setting this custom property.
Unload Source Partition Count	The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected. Default is 5. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partition count. You can override the value from auto-tuning by setting this custom property.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.

Source Tables

• •On the Selected Tables tab, individually select the check box for each source table you want to include. Clear the check box for any tables you do not want to include. To select all tables, select the Table check box at the top.
The Column count for a table shows the total number of columns in the table.
If you select the tables you want to include, all the selected and unselected tables are displayed by default. To view the selected tables only, use the filter next to the table selection count and change the view from All to Selected.
To find tables or columns, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns tables and columns that include CDC anywhere in their names. To narrow the search to only table names or column names, select Table Name or Columns in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for tables. To add a table rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
Note:
If you define multiple table rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a table rule specifies "Exclude CDC" followed by "Include C", all tables with names beginning with "C" are selected, including the CDC tables.
Tips and guidelines for rules:
• - You can copy an existing table rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a table rule directly in its row by clicking the Type or Match Condition value.
• - To view the tables and columns that match a single table rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - The task wizard is case sensitive. Enter the table names or masks in the case with which they were defined.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a table name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.

Advanced Source Properties

1. 1If you're defining an incremental load or combined initial and incremental load task, the Change Data Capture Method field shows the selected CDC method. For SAP HANA sources, this field is read only because the capture type is set in the SAP HANA Database Ingestion connection properties. Options are:
• - Log-based. The CDC Staging Task reads data from SAP HANA online and archived transaction logs.
• - Trigger-based. The job uses triggers to get before images and after images of DML changes and to write entries for the changes to internal tables.
2. 2Complete the following optional advanced properties as needed:

Property	Load Type	Description
CDC Method	Incremental loads and combined initial and incremental loads	Displays either Log-based or Trigger-based in read only mode. This CDC method is derived from the capture type you set in the SAP HANA Database Ingestion connection you're using.
CDC Script	Incremental loads and combined initial and incremental loads	Generate a script for enabling CDC on source tables and then run or download the script. The only available option is Enable CDC for all columns. For SAP HANA and SAP HANA Cloud sources that use Trigger-based CDC, the script creates the required PKLOG, PROCESSED, and _CDC shadow tables. Also creates three triggers and a sequence for each selected source table. For SAP HANA sources that use Log-based CDC, creates the TRANSACTIONS and ROWCACHE tables in the database cache that stages the change data captured by the CDC Staging Task. Click Execute to run the script if you have the required privileges. Or click the Download icon to download the script so that you can give it to your DBA to run.
List Tables by Rule Type	All	Generate and download a list of the source tables that match the table selection criteria. If you used rule-based table selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Select the Include Columns check box to include columns in the list, regardless of which table selection method you used. Click the Download icon to download the list.
Include LOBs	Incremental loads and combined initial and incremental loads	Not available.
Enable Persistent Storage	Incremental loads and combined initial and incremental loads Note: The property is not available if you are using a serverless runtime environment.	For sources that use the Log-based CDC method, select this check box to enable persistent storage of transaction data in a disk buffer so that the data can be consumed continually, even when the writing of data to the target is slow or delayed. For sources that use the Trigger-based CDC method, this option is selected by default and can't be changed. Benefits of using persistent storage are faster consumption of the source transaction logs, less reliance on log archives or backups, and the ability to still access the data persisted in disk storage after restarting a database ingestion job. Persisted data is stored on the Secure Agent. It is not encrypted. The Secure Agent's files and directories are expected to be secured from unwanted access by using native file system access permissions or file system support of encryption natively.
Initial Start Point for Incremental Load	Incremental loads (Log-based and trigger-based CDC)	If you want to customize the position in the source logs from which the database ingestion and replication job starts reading change records the first time it runs, select one of the following options: - Earliest Available. The earliest available position in the database log or structure where changes are stored. For trigger-based CDC, the earliest available record in the PKLOG table. For log-based CDC, the earliest point in the TRANSACTIONS table. - Latest Available. The latest available position in the database log or structure. - Position. An SAP HANA sequence value in the change stream from which you want the database ingestion job to start retrieving change records. This value must be equal to or less than the current position value. An invalid value will cause the job to fail. The default value of 0 is displayed by default, which causes the earliest available point to be used. - Specific Date and Time. A date and time, in the format MM/DD/YYYY hh:mm AM|PM, that determines the position in the change stream from which to start retrieving change records. Database Ingestion and Replication retrieves only the changes that were made after this date and time. Database Ingestion and Replication determines the Position/Sequence Number (PSN) based on the earliest available PSN after the specified date and time. For trigger-based CDC, the time zone of the date and time will be the same as the SAP HANA database time zone. For log-based CDC, enter this value in UTC. The log-based CDC method stores all timestamps in UTC. The default is Latest Available. Note: - This initial start point setting pertains only to the initial run of a job. Thereafter, if you resume a stopped or aborted job, the job begins propagating source data from where it last left off. - For combined initial and incremental load jobs, initial loading is not performed until the incremental processing of change data reaches the end of the current transaction log.
Stage CDC Data	Incremental loads and combined initial and incremental loads that use the Log-based CDC method	This option is selected by default for log-based CDC tasks and can't be deselected. This option enables the job to read data from the source database in a single pass and then write the data to common storage. The data can then be read from storage by multiple tasks in the staging group. The tasks must process the same database but can process different tables that have different schemas.
Staging Group	Incremental loads and combined initial and incremental loads that use the Log-based CDC method	To specify a CDC staging group to stage CDC data for multiple log-based CDC tasks, either select a previously defined group from the list or click New to create one, provided that the connection you're using is not already associated with a staging group. If another log-based CDC task that uses the same connection is already associated with a deployed staging group, this field displays that group name in read only mode and the New button is disabled. You can edit the group properties if you want to add custom properties for the group. Other group properties are provided by the connection and can't be edited. If the source connection isn't already associated with a CDC Staging Task that's been deployed, you must create a new group.

If you need to create a new staging group because one is not already associated with the connection, complete the following fields on the Overview tab of the New Staging Group dialog box:

Property	Description
Group Name	Accept the generated group name, which has the format Log_group_cdc_yyyymmdd<number>, or enter a custom name.
Location	The project in which you want to store the staging group definition, if you don't want to use the Default location.
Runtime Environment	Select the runtime environment to use for running the CDC staging task. For a log-based source, ensure that the Secure Agent on which the task will run can access all of the source logs.
Staging Data Retention Period	The number of days to retain data in cloud storage. For SAP HANA (log-based) sources, this field is read only. It displays the number of days set in the Log Clear property of the SAP HANA Database Ingestion connection. After this period elapses, the data staged in the cache database is compacted.

If you want to add source and target custom properties for the CDC staging jobs in the group, click the Custom Properties tab. Then under Source Properties or Target Properties, click the Add new row + icon and enter a property name and value. Typically, you use custom properties at the direction of Informatica Global Customer Support or Informatica technical staff.
When done, click Save to save the staging group and return to the Task Details, Step 1 page.
Note:
If you edit the staging group later from a deployed task in the group, you can change only the Row Flush Threshold, Flush Interval, Staging Data Retention Period, and custom property values.
3. 3Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available, depending on the load type:

Property	Description
Read Event Batch Size	The number of payload events written in batch to the internal event queue during CDC processing. When the event queue is implemented as an internal ring buffer, this value is the number of payload events that the reader writes to a single internal buffer slot. Note: A batch size that's too small might increase contention between threads. A larger batch size can provide for more parallelism but consume more memory.
Reader Helper Thread Count	The number of reader helper threads used during CDC processing to convert change data into a canonical format that can be passed to the target. Default value is 3. You can enter a larger value to allow more threads to be available for performing conversion processing in parallel.
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal thread count. You can override the value from auto-tuning by setting this custom property.
Unload JDBC Partitioning Technique	The technique to use for partitioning source table rows for initial load processing or the unload phase of a combined load job. Valid values are: - uniform. Select all of the primary key values from the source table and distribute those keys across the partitions evenly, producing partitions of equal or near equal size. - heuristic. Estimate the primary key ranges to assign to each partition by calculating the difference between the low and high primary key values and using the number of partitions requested. With this technique, ranges are calculated more quickly because the query to select primary key values is avoided. However, the sizes of the partitions might vary widely if primary keys are not distributed evenly between high and low values. For example, if 26 partitions are requested and the low primary key starts with A and the high primary key starts with Z, you might expect one partition for each set of rows with a primary key from A through Z. However, if 90% of the primary keys start with Q, R,or S, 90% of the rows are assigned to just three partitions. Default value is uniform. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partitioning technique. You can override the value from auto-tuning by setting this custom property.
Unload Source Max Parallel Partition	The maximum number of partition reader threads that can be used to query the source in parallel during initial load processing or the unload phase of combined jobs. Use this property to control the number of source partition queries that can be executed against the source at the same time. For example, if a table contains data in 100 partitions, all 100 partitions are queried at the same time by default. However, you can use this property to reduce the number of concurrent queries. Default value is equal to the total number of partitions. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal number of partition reader threads. You can override the value from auto-tuning by setting this custom property.
Enable Partitioning	Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Select this property to use multiple partitions with one query per partition. By default, this property is not selected. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine whether to enable partitioning. You can override the value from auto-tuning by setting this custom property.
Unload Source Partition Count	The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected. Default is 5. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partition count. You can override the value from auto-tuning by setting this custom property.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
If you configured a CDC staging group, any custom properties that you enter here for the source are ignored. Instead, enter custom properties for the staging group.
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.

Source Tables

• •On the Selected Tables tab, individually select the check box for each source table you want to include. Clear the check box for any tables you do not want to include. To select all tables, select the Table check box at the top.
The Column count for a table shows the total number of columns in the table.
If you select the tables you want to include, all the selected and unselected tables are displayed by default. To view the selected tables only, use the filter next to the table selection count and change the view from All to Selected.
To find tables or columns, you can type all or part of a name in the Find box and click Search. This value is case-sensitive. If you type the beginning of a name only, a wildcard isn't required to represent the remainder. For example, CDC, CD, and CD* return the same results. However, if the search string is within the name, include the wildcard * at the beginning. For example, *CDC returns tables and columns that include CDC anywhere in their names. To narrow the search to only table names or column names, select Table Name or Columns in the drop-down list adjacent to the Find box.
• •On the Selection Rules tab, you can create inclusion and exclusion rules for tables. To add a table rule, click the plus (+) sign in the upper right corner. In the Type field, select Include or Exclude as the rule type. Then enter a string, with or without wildcards, as the condition.
If you want to include database views as sources, select the Include Views check box to the left of the plus sign.
Note:
If you define multiple table rules, they're processed in the order in which they're listed (top to bottom). Be sure to define them in the correct order of processing. For example, if a table rule specifies "Exclude CDC" followed by "Include C", all tables with names beginning with "C" are selected, including the CDC tables.
Tips and guidelines for rules:
• - You can copy an existing table rule to use as a starting point for creating another rule. Click the Copy icon at the right end of the row.
• - You can edit a table rule directly in its row by clicking the Type or Match Condition value.
• - To view the tables and columns that match a single table rule, click View Objects in the row for the rule. For an Include rule, it shows the objects to be included. For an Exclude rule, it shows the objects to be excluded.
When you define a condition in a rule:
• - The task wizard is case sensitive. Enter the table names or masks in the case with which they were defined.
• - A mask can contain one or more wildcards. Supported wildcards are: an asterisk (*), which represents one or more characters, and a question mark (?), which represents a single character. A wildcard can occur multiple times in a mask value and can occur anywhere in the value.
• - Delimiters such as quotation marks or brackets are not allowed, even if the source database uses them. For example, Oracle requires quotation marks around lowercase and mixed-case names to force the names to be stored in lowercase or mixed case. However, in the task wizard, you must enter the lowercase or mixed-case names without quotation marks.
• - If a table name includes special characters such as a backslash (\), asterisk(*), dollar sign ($), caret (^), or question mark (?), escape each special character with a backslash (\) when you enter the rule.

Advanced Source Properties

1. 1Complete the following optional advanced properties as needed:

Property	Load Type	Description
List Tables by Rule Type	All	Generate and download a list of the source tables that match the table selection criteria. If you used rule-based table selection, you can select the type of selection rules to use. Options are: - Include Rules Only - Exclude Rules Only - Include And Exclude Rules Select the Include Columns check box to include columns in the list, regardless of which table selection method you used. Click the Download icon to download the list.

2. 2Under Custom Properties, you can specify one or more custom properties that Informatica provides to improve performance or to meet your special requirements. To add a property, click the + icon to add a row. In the Property Name field, select a property and enter the value, or select the Custom option and manuallly enter both the property name and value.
The following table describes the properties that are available for initial load jobs with this source:

Property	Description
Unload Helper Thread Count	The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer. Default value is 2. If two threads can’t keep up with the incoming volume, you can increase the number of threads. Consider increasing the number of threads in the following situations: 1) parallel SQL result sets are open, which usually occurs when source partitioning is enabled, or 2) some rows are very large or wide, which increases conversion time. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal thread count. You can override the value from auto-tuning by setting this custom property.
Enable Partitioning	Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Select this property to use multiple partitions with one query per partition. By default, this property is not selected. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine whether to enable partitioning. You can override the value from auto-tuning by setting this custom property.
Unload Source Partition Count	The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected. Default is 5. Note: If you select the Auto-Tune option on the last wizard page, auto-tuning can determine the optimal partition count. You can override the value from auto-tuning by setting this custom property.
Custom	Select this option to manually enter the name of a property and its value. Use this option to enter properties that Informatica Global Customer Support or a technical staff member has provided to you for a special case. Available for any supported load type.

Custom properties are intended to address performance or special processing needs. A property name can contain only alphanumeric characters and the following special characters: periods (.), hyphens (-), and underscores (_).
Tip:
To delete a custom property after you've entered it, click the Delete icon at the right end of the property row.