Big Data Streaming
This section describes changes to Big Data Streaming in version 10.2.1.
Configuring the Hadoop Distribution
Effective in version 10.2.1, you configure the Hadoop distribution in cluster configuration properties.
The Distribution Name and Distribution Version properties are populated when you import a cluster configuration from the cluster. You can edit the distribution version after you finish the import process.
Previously, the Hadoop distribution was identified by the path to the distribution directory on the machine that hosts the Data Integration Service.
For more information about the Distribution Name and Distribution Version properties, see the Informatica Big Data Management 10.2.1 Administration Guide.
Developer Tool Configuration
Effective in version 10.2.1, you can create a Metadata Access Service. The Metadata Access Service is an application service that allows the Developer tool to access Hadoop connection information to import and preview metadata.
The following sources and targets use Metadata Access Service at design time to extract the metadata:
- •HBase
- •HDFS
- •Hive
- •MapR-DB
- •MapRStreams
Previously, you performed the following steps manually on each Developer tool client machine to establish communication between the Developer tool machine and Hadoop cluster at design time:
- •Extracted cluster configuration files.
- •Ran krb5.ini file to import metadata from Hive, HBase, and complex file sources from a kerberos-enabled Hadoop cluster.
The Metadata Access Service eliminates the need to configure each Developer tool machine for design-time connectivity to Hadoop cluster.
For more information, see the "Metadata Access Service" chapter in the Informatica 10.2.1 Application Service Guide.
Kafka Connection Properties
Effective in version 10.2.1, properties for the Kafka connection have changed.
You can now configure the Kafka broker version in the connection properties.
Previously, you configured this property in the hadoopEnv.properties file and the hadoopRes.properties file.
For more information about the Kafka connection, see the "Connections" chapter in the Informatica Big Data Streaming 10.2.1 User Guide.