User Guide > Streaming Mappings > Streaming Mappings Overview
  

Streaming Mappings Overview

Use the Developer tool to create and run streaming mappings in the Hadoop run-time environment and process data that is in JSON, XML, CSV, or Avro format.
Develop a mapping to read, transform, and write data according to your business needs. When you create a streaming mapping, select the Hadoop environment and the Spark engine. When you run a streaming mapping, the Data Integration Service pushes the processing to nodes on a Spark engine in the Hadoop cluster.
Use the following steps as a guideline when you develop a streaming mapping:
  1. 1. Create connections that you want to use to access streaming data.
  2. 2. Create input, output, and reusable objects that you want to use in the mapping. Create physical data objects to use as mapping input or output.
  3. 3. Create reusable transformations that you want to use.
  4. 4. Create rules.
  5. 5. Create the streaming mapping.
  6. 6. Add objects to the mapping.
  7. 7. Link ports between mapping objects to create a flow of data from sources to targets, through transformations.
  8. 8. Configure the mapping
  9. 9. Validate the mapping to identify errors.
  10. 10. Save the mapping and run it to see the mapping output.