The Union transformation is an active transformation that you use to merge data from multiple pipelines into a single pipeline.
For data integration patterns, it is common to combine two or more data sources into a single stream that includes the union of all rows. The data sources often do not have the same structure, so you cannot freely join the data streams. The Union transformation enables you to make the metadata of the streams alike so that you can combine the data sources in a single target.
The Union transformation merges data from multiple sources similar to the UNION ALL SQL statement. For example, you might use the Union transformation to merge employee information from ADP with data from a Workday employee object.
You can add, change, or remove specific fields when you merge data sources with a Union transformation.
At run time, the mapping task processes input groups in parallel. It concurrently reads the sources connected to the Union transformation and pushes blocks of data into the input groups of the transformation. As the mapping runs, it merges data into a single output group based on the field mappings.