Big Data Streaming
This section describes new Big Data Streaming features in version 10.2.1.
Sources and Targets
Effective in version 10.2.1, you can read from or write to the following sources and targets in streaming mappings:
- •Azure Event Hubs. Create an Azure EventHub data object to read from or write to Event Hub events. You can use an Azure EventHub connection to access Microsoft Azure Event Hubs as source or target. You can create and manage an Azure Eventhub connection in the Developer tool or through infacmd.
- •Microsoft Azure Data Lake Store. Create an Azure Data Lake store data object to write to Azure Data Lake Store. You can use an Azure Data Lake Store connection to access Microsoft Azure Data Lake Store tables as targets. You can create and manage a Microsoft Azure Data Lake Store connection in the Developer tool.
- •JDBC-compliant database. Create a relational data object with a JDBC connection.
For more information, see the "Sources in a Streaming Mapping" and "Targets in a Streaming Mapping" chapters in the Informatica Big Data Streaming 10.2.1 User Guide.
Stateful Computing in Streaming Mappings
Effective in 10.2.1, you can use window functions in an Expression transformation to perform stateful calculations in streaming mappings.
For more information, see the "Streaming Mappings" chapter in the Informatica Big Data Streaming 10.2.1 User Guide.
Transformation Support
Effective in version 10.2.1, you can use the following transformations in streaming mappings:
- •Data Masking
- •Normalizer
- •Python
You can perform an uncached lookup on HBase data in streaming mappings with a Lookup transformation.
For more information, see the "Streaming Mappings" chapter in the Informatica Big Data Streaming 10.2.1 User Guide.
Truncate Partitioned Hive Target Tables
Effective in version 10.2.1, you can truncate an external or managed Hive table with or without partitions.
For more information about truncating Hive targets, see the "Targets in a Streaming Mapping" chapter in the Informatica Big Data Streaming 10.2.1 User Guide.