You can select a single source object to transfer data from or define a custom query.
The type of source and the formatting and advanced options that you can configure for the source depend on the source connection that you select. For example, you can create a custom query for database sources that support SQL queries. For a flat file source, you can configure formatting options such as the formatting type. For Salesforce sources, you can configure advanced options such as the SOQL filter condition, row limit, and bulk query.
For information about the options that you can configure for a source connection, see the help for the appropriate connector.
You can preview the data in the source. The preview returns the first 10 rows. To preview rows in alphabetical order, select Display fields in alphabetical order. Data Integration does not change the order of rows in the actual source. You can also download the preview results as a CSV file.
Source filters
Apply filter conditions to filter the source data that you transfer to the target.
You can apply the following types of filter conditions:
Simple
Select the source field, and configure the operator and value to use in the filter.
When you define more than one filter condition, the task evaluates them in the order that you specify. The task evaluates the filter conditions using the AND logical operator to join the conditions. It returns rows that match all the filter conditions.
Advanced
Create a filter expression using the expression editor. You enter one expression that contains all filters. You can use source fields and built-in functions in the expression.
For certain source types, you can sort the source data to provided sorted data to the target.
When you sort data, you select one or more source fields to sort by. If you apply more than one sort condition, Data Integration sorts fields in the listed order.
To see if a connector supports sorting, see the help for the appropriate connector.
Custom queries
Create a custom query when you want to use a database source that you can't configure with a single object and lookup source. You might create a custom query to join multiple source tables.
To use a custom query as a source, select Query as the source type and then click Select. When you define the query, use SQL that is valid for the source database. You can use database-specific functions in the query.
When you create a custom query, enter an SQL SELECT statement to select the source columns you want to use. Data Integration uses the SQL statement to retrieve source column information.
Data Integration ensures that custom query column names are unique. If an SQL statement returns a duplicate column name, Data Integration adds a number to the duplicate column name as follows:
<column_name><number>
Tip: Test the SQL statement you want to use on the source database before you create a custom query. Data Integration doesn't display specific error messages for invalid SQL statements.