User Guide > Streaming Mappings > Data Objects
  

Data Objects

Based on the type of source you are reading from or target you are writing to you can create physical data objects to access the different types of data.
The following table lists the data objects that you can include in streaming mappings and the read and write data object operations for each:
Data Object
Source
Data Object Operation
Target
Data Object Operation
AmazonKinesis
Amazon Kinesis Stream
Read
Amazon Kinesis Firehose Delivery Stream
Write
Azure Data Lake Store
-
-
Microsoft Azure Data Lake Store
Write
Azure Event Hub
Azure Event Hub Service
Read
Azure Event Hub Service
Write
Complex File
-
-
HDFS sequence file or binary file
Write
HBase
-
-
HBase tables
Write
JMS
JMS server
Read
JMS server
Write
Kafka
Kafka broker
Read
Kafka broker
Write
MapRStreams
MapR Stream
Read
MapR Stream
Write
Relational
-
-
Hive table
JDBC-compliant database
Write

Creating a Data Object

Create a data object to add to a streaming mapping.
    1. Select a project or folder in the Object Explorer view.
    2. Click File > New > Data Object.
    3. Select the data object that you want to add t the streaming mapping and click Next.
    The data object dialog box appears.
    4. Enter a name for the data object.
    5. Click Browse next to the Location option and select the target project or folder.
    6. Click Browse next to the Connection option and select the connection that you want to use
    7. To add a resource, click Add next to the Selected Resources option.
    The Add Resource dialog box appears.
    8. Select the check box next to the resource you want to add and click OK.
    9. Click Finish.
    The data object appears under Data Objects in the project or folder in the Object Explorer view.

Creating a Data Object Operation

You can create the data object read or write operation for a data object depending on the source that you read from or target that you write to. You can then add the object operation to a streaming mapping.
    1. Select the data object in the Object Explorer view.
    2. Right-click and select New > Data Object Operation.
    The Data Object Operation dialog box appears.
    3. Enter a name for the data object operation.
    4. Select the type of data object operation. You can choose to create a StreamRead or StreamWrite operation.
    The Data Integration Service uses read operation properties when it reads data and write operation properties when it writes data.
    5. Click Add.
    The Select Resources dialog box appears.
    6. Select the data object for which you want to create the data object operation and click OK.
    7. Click Finish.
The Developer tool creates the data object operation for the selected data object.