Developing PowerCenter Real-time Workflows
You develop PowerCenter workflows for Data Integration Hub in the same way that you develop other PowerCenter workflows. Data Integration Hub transformations interact directly with Data Integration Hub.
Developing PowerCenter Real-time Workflows Rules and Guidelines
When you develop PowerCenter real-time workflows to use in publications with custom mappings, consider the following rules and guidelines:
- •Make sure to name and store PowerCenter entities for custom mappings in different folders with a different naming convention from the naming convention of PowerCenter entities for automatic mappings.
- •You must enable the property Enable high precision in the session that writes data to the Data Integration Hub publication repository.
- •To prevent naming conflicts, do not use _DIH__ in the parameter names, and do not use workflow and mapping parameters with the same names as workflow and mapping parameters that Data Integration Hub uses in workflows for publications with automatic mappings.
- •The workflows do not generate error messages. Maintaining the workflows is the responsibility of the Data Integration Hub developer.
- •If you have multiple tables in a topic and you want to ensure that the records for all tables are included in the same publication, change the value of the system property dih.realtime.time.window. For information, see the Data Integration Hub Administrator Guide.
- •If you edit the PowerCenter workflow or mapping, you must update the associated Data Integration Hub workflow by re-selecting the PowerCenter workflow or the XML file in the Data Integration Hub Operation Console.
For general rules and guidelines on developing PowerCenter workflows to use with Data Integration Hub, see .
Developing Publication Real-time Workflows Process
To develop a workflow in PowerCenter that processes Data Integration Hub real time publications, perform the following tasks:
- 1. Create the source and target definitions.
- 2. Create the mapping and add transformations.
- 3. Create the workflow and the session and save the workflow.
- 4. In the Data Integration Hub Operation Console, create a topic to which the application will publish the real-time data. Base the topic structure on the PowerCenter workflow or on the workflow definition file. Data Integration Hub creates the topic in PowerCenter and names the target according to the Data Integration Hub naming conventions.
- 5. In the workflow that you created in step Developing Publication Real-time Workflows Process, rename the target to the name of the topic target in PowerCenter and save the workflow. Optionally, export the workflow to an XML workflow definition file.
- 6. In the Data Integration Hub Operation Console, create a publication real-time workflow and select the PowerCenter workflow that you modified in step Developing Publication Real-time Workflows Process.
Step 1. Create the Source and Target Definitions
When you develop a publication real-time workflow, you create the source based on real-time data that you want to publish. You set the target of the publication real-time workflow to the Data Integration Hub publication repository. You can also add properties to store other Data Integration Hub values that you want to send from PowerCenter.
You define the source of a publication real-time workflow based on the following real-time data:
- •Data in the Apache Kafka stream
- •Structure of the JMS
- •Web service source
You create the source and the target definitions in the PowerCenter Designer in the same way that you create source and target definitions for other mappings. For general information about source and target definitions, see the PowerCenter Designer Guide.
When you create the source and the target definitions, consider the following guidelines:
- •Use the DIH__STAGING connection in the target definition.
- •The database target type must match the database type of the Data Integration Hub publication repository.
- •The target must contain DIH__PUBLICATION_INSTANCE_ID and DIH__PUBLICATION_INSTANCE_DATE columns.
The following image shows an example of a target with the DIH__PUBLICATION_INSTANCE_ID and DIH__PUBLICATION_INSTANCE_DATE columns:
Step 2. Create the Mapping
Create a mapping that contains the source definition, the target definition, and the transformations that you want to use in the real-time publication workflow.
You create a mapping for Data Integration Hub in the same way that you build other PowerCenter mappings.
When you create the source and the mapping, consider the following guidelines:
- •Add the DX_Publication_Parameters transformation to the mapping. You can also add other PowerCenter transformation to the mapping.
- •In the DXPublicationName port of the DX_Publication_Parameters transformation enter the name of the publication to which the workflow publishes. The DX_Publication_Parameters transformation adds placeholders for the publication instance ID and for the publication date in the Data Integration Hub publication repository.
- •Connect the port DXPublicationInstanceId from the DX_Publication_Parameters transformation to the DIH__PUBLICATION_INSTANCE_ID column in the target.
- •Connect the port DXPublicationInstanceDate from the DX_Publication_Parameters transformation to the DIH__PUBLICATION_INSTANCE_DATE column in the target.
The following image shows an example mapping that includes the DX_Publication_Parameters transformation. The mapping also includes an expression that sets the publication name:
Step 3. Create and Save the PowerCenter Workflow and Session
Create the real-time workflow and the session that runs the mapping. You create the workflow in the same way that you create other PowerCenter workflows.
1. In the Workflow Designer, create the workflow and the session object. You must configure the session as a real-time session. For general information about creating workflows and sessions, see the PowerCenter Advanced Workflow Guide.
2. Save the session object.
3. Test the workflow to ensure that it works correctly.
4. Save the workflow to the PowerCenter repository. Optionally, use the Repository Manager to export the workflow from PowerCenter to an XML file.
Note: Do not save the workflow to a folder where the folder name is prefixed by DIH__pub or by DIH__sub.
Step 4. Create the Topic
Create the topic to which the application will publish the real-time data. The topic defines the data structure and additional data definitions such as the data retention period.
You create the topic in the Data Integration Hub Operation Console.
When you create the topic, consider the following guidelines:
- •You define the topic structure in the Structure page of the Topic wizard.
- •From the Add Tables list, choose From PowerCenter.
- •In the Add Tables from PowerCenter Workflow dialog box, choose one of the following options:
- - Choose Select a workflow from the PowerCenter repository to add tables from the real-time workflow.
- - Choose Select a workflow definition file (.xml) to add tables from the XML file that you exported from the real-time workflow.
For more information, see the Data Integration Hub Operator Guide.
Step 5. Rename the Workflow Target
Rename the target of the PowerCenter real-time workflow to the name of the topic target in PowerCenter. You rename the target in the PowerCenter Designer.
1. In the navigation pane, access the folder of the topic that you created and expand the folder tree.
2. Expand the Targets subfolder and copy the target name into the clipboard.
3. Access the folder of the workflow and expand the folder tree.
4. Expand the Targets subfolder and replace the target name with the name that you copied from the topic folder.
5. Save the session object.
6. Test the workflow to ensure that it works correctly.
7. Save the PowerCenter workflow to the PowerCenter repository. Optionally, use the Repository Manager to export the workflow from PowerCenter to an XML file.
Note: Do not save the workflow to a folder where the folder name is prefixed by DIH__pub or by DIH__sub.
Step 6. Create the Publication Real-time Workflow
Create the Data Integration Hub real-time workflow that will publish the real-time data. The Data Integration Hub operator can use the workflow to create a real-time publication.
You create the workflow in the Data Integration Hub Operation Console, in the Workflows page.
When you create the workflow, consider the following guidelines:
- •You define the topic structure in the Structure page of the Topic wizard.
- •In Flow Type, choose PowerCenter Real-time workflow.
- •In Type, choose Publication.
The Data Integration Hub operator selects the workflow in the Create Custom Real Time Publication wizard, in the Mapping page.
