Dynamic Sources
A dynamic source is a source that can change at run time. You can configure flat file, relational, Amazon S3, and complex file dynamic sources in a mapping.
Note: Dynamic mapping support for Amazon S3 sources and complex file sources is available for technical preview. Technical preview functionality is supported but is unwarranted and is not production-ready. Informatica recommends that you use these features in non-production environments only.
You can configure dynamic run-time functionality for a source in the following ways:
- Get columns from the data source.
- When you expect small changes to a source at run time, you can configure the Read transformation to get flat file or relational object columns at run time. You can update the ports in a Read transformation at run time based on the structure of the relational data source or the flat file data source.
- Assign a parameter to determine the flat file name and directory of source.
- When the flat file sources are similar, you can assign a parameter to a file name or directory. When you use a parameter, you do not need to create a data object for each source.
- Assign a parameter to determine the resource, table owner, or directory of a relational data object.
- When the relational sources are similar, you can assign a parameter to get the resource, connection, and table owner properties.
- Assign a parameter to determine the data object to use for a file or relational source.
When you expect small changes to a source, you can update the ports in a Read transformation at run time based on the structure of the relational data source or the flat file data source.
The following table shows where you can configure the dynamic run-time functionality of a source:
Dynamic Run-time Source Functionality | Configuration |
|---|
Get columns from the data source. | Configure the Data Object tab on the Read transformation for the following source types: |
Assign a parameter to determine the flat file name and directory. | Configure the Advanced tab on the physical data object for the following source type: |
Assign a parameter to determine the connection, owner, or resource. | Configure the Run-Time tab on the Read transformation for the following source type: |
Assign a parameter to determine the data object. | Configure the Data Object tab on the Read transformation for the following source types: |
Get Columns from the Data Source
You can update the ports in a Read transformation at run time based on the structure of the relational or flat file data source. You might use this method to update the run-time instance of the Read transformation.
Update the columns at run time when you expect small changes to the source. For example, you need to process a source from another organization, but the organization cannot guarantee the order of the columns in the source file. When you enable the option to update the data object columns at run time, the Data Integration Service changes the ports of the Read transformation based on the structure of the source data. The Read transformation passes the data to the downstream transformations in the dynamic mapping for processing.
When you update the data object columns at run time, the Data Integration Service updates a run-time instance of the Read transformation. It does not update metadata in the Model repository. To see the changes in the Developer tool, view the mapping with resolved parameters. To update the physical data object definition in the Model repository, use the synchronize option in the Developer tool. The Developer tool re-imports the physical data object metadata and changes the metadata.
Note: If you create a custom SQL query for a Read or Lookup transformation, the metadata from the custom query might conflict with the refreshed metadata from data source columns. If the query is inconsistent with the query that the Data Integration Service would generate by default, the mapping fails.
The following image shows where to enable the option on the Data Object tab:
The Data Integration Service determines the structure of a relational source by the schema. It examines the schema of the resource that appears on the Run-time tab. The Data Integration Service then updates the columns in the transformation data object based on the schema.
The Data Integration Service determines the structure of a flat file source based on the way you configure the flat file physical data object. You configure the data object to generate column names at run time.
Configure this functionality on the Data Object tab of the Read transformation for a flat file or relational source.
For more information about setting the flat file physical data object properties, see the Informatica Developer Tool Guide.
Assign a Parameter to a Flat File Name
To run a dynamic mapping with similar flat file sources, you can assign a parameter to a file name or directory. When you use a parameter, you do not need to create a data object for each source.
You can parameterize the file name and the directory in a flat file physical data object. You can parameterize the properties before you create a transformation from the data object. Configure the parameters on the Advanced tab of the physical data object properties. When you create the transformation from the physical data object, you can use mapping parameters to override the parameter default values.
The following image shows the Advanced tab of a physical data object:
Configure this functionality on the Advanced tab of the physical data object for a flat file source.
Assign a Parameter to Relational Source Properties
To run a dynamic mapping with similar relational sources, you can assign a parameter the resource, connection, and table owner properties in the Read transformation.
Use a parameter for the resource to run a mapping with different but similar tables in the same database. When you use a parameter for the resource, you do not need to create a data object for each source. Use a parameter for the connection to access a different database. You might need to run a unique SQL query against multiple relational sources.
Configure the relational table parameters on the Run-Time tab of the transformation properties. You cannot parameterize these properties in the relational physical data object. When you create parameters for the properties in the Read transformation, you create mapping parameters. Once you configure the run-time parameters for the Read transformation, the data source that the Read transformation uses is updated based on the data stored in the relational database.
By default, you create a connection type parameter for the connection. You configure a resource type parameter for the table name and a string parameter for the table owner.
The following image shows the Run-Time tab of the Read transformation:
Configure this functionality on the Run-Time tab of the Read transformation for a relational source.
Assign a Parameter to the Source Data Object
You can assign a parameter to the data object and change the source for the Read transformation at run time.
Parameterize the data object when you have a different physical data object in the Model repository for each data source. You might parameterize the data object if you need to configure the same transformation for a flat file or for a database table. When you parameterize the data object, you enable the transformation to use data objects that have different properties or unique SQL queries.
When you create a Read transformation from a physical data object, information about the data object appears on the Data Object tab of the transformation properties. You can click the data object name to view the physical data object definition from the Model repository.
To parameterize the data object, create a resource type parameter or browse for a resource parameter. The parameter default value is the name of physical data object in the Model repository. When you create a default parameter value, you select a physical data object name from a list of data objects in the repository.
When you change the data object, the transformation ports change. You can view the ports on the Ports tab in the transformation properties.
The following image shows the Data Object tab when you specify the data object by a parameter:
The following table describes the parameter options on the Data Object tab:
Parameter Options | Description |
|---|
Parameter | The name of a resource parameter that you configured as the data object. Read-only. |
Description | The description of the parameter. Read-only. |
New | Create a resource parameter. Browse for and select a data object in the Model repository for the parameter default value. |
Browse | Browse for a resource parameter and select the parameter. |
Default Value | The default value of the resource parameter that you configured for the data object. The default value is a physical data object name. Read-only. |
Configure this functionality on the Data Object tab of the Read transformation for a flat file or relational source.
