New Features (10.1.1 Update 2)

Big Data Management

Truncate Hive table partitions on mappings that use the Blaze run-time engine

Effective in version 10.1.1 Update 2, you can truncate Hive table partitions on mappings that use the Blaze run-time engine.

For more information about truncating partitions in a Hive target, see the Informatica 10.1.1 Update 2 Big Data Management User Guide.

Filters for partitioned columns on the Blaze engine

Effective in version 10.1.1 Update 2, the Blaze engine can push filters on partitioned columns down to the Hive source to increase performance.

When a mapping contains a Filter transformation on a partitioned column of a Hive source, the Blaze engine reads only the partitions with data that satisfies the filter condition. To enable the Blaze engine to read specific partitions, the Filter transformation must be the next transformation after the source in the mapping.

For more information, see the Informatica 10.1.1 Update 2 Big Data Management User Guide.

OraOop support on the Spark engine

Effective in version 10.1.1 Update 2, you can configure OraOop to run Sqoop mappings on the Spark engine. When you read data from or write data to Oracle, you can configure the direct argument to enable Sqoop to use OraOop.

OraOop is a specialized Sqoop plug-in for Oracle that uses native protocols to connect to the Oracle database. When you configure OraOop, the performance improves.

For more information, see the Informatica 10.1.1 Update 2 Big Data Management User Guide.

Sqoop support for native Teradata mappings on Cloudera clusters

Effective in version 10.1.1 Update 2, if you use a Teradata PT connection to run a mapping on a Cloudera cluster and on the Blaze engine, the Data Integration Service invokes the Cloudera Connector Powered by Teradata at run time. The Data Integration Service then runs the mapping through Sqoop.

For more information, see the Informatica 10.1.1 Update 2 PowerExchange for Teradata Parallel Transporter API User Guide.

Scheduler support on Blaze and Spark engines

Effective in version 10.1.1 Update 2, the following schedulers are valid for Hadoop distributions on both Blaze and Spark engines:

• - Fair Scheduler. Assigns resources to jobs such that all jobs receive, on average, an equal share of resources over time.
• - Capacity Scheduler. Designed to run Hadoop applications as a shared, multi-tenant cluster. You can configure Capacity Scheduler with or without node labeling. Node label is a way to group nodes with similar characteristics.

For more information, see the Mappings in the Hadoop Environment chapter of the Informatica 10.1.1 Update 2 Big Data Management User Guide.

Support for YARN queues on Blaze and Spark engines

Effective in version 10.1.1 Update 2, you can direct Blaze and Spark jobs to a specific YARN scheduler queue. Queues allow multiple tenants to share the cluster. As you submit applications to YARN, the scheduler assigns them to a queue. You configure the YARN queue in the Hadoop connection properties.

For more information, see the Mappings in the Hadoop Environment chapter of the Informatica 10.1.1 Update 2 Big Data Management User Guide.

Hadoop security features on IBM BigInsights 4.2

Effective in version 10.1.1 Update 2, you can use the following Hadoop security features on the IBM BigInsights 4.2 Hadoop distribution:

• - Apache Knox
• - Apache Ranger
• - HDFS Transparent Encryption

For more information, see the Informatica 10.1.1 Update 2 Big Data Management Security Guide.

SSL/TLS security modes

Effective in version 10.1.1 Update 2, you can use the SSL and TLS security modes on the Cloudera and HortonWorks Hadoop distributions, including the following security methods and plugins:

• - Kerberos authentication
• - Apache Ranger
• - Apache Sentry
• - Name node high availability
• - Resource Manager high availability

For more information, see the Informatica 10.1.1 Update 2 Big Data Management Installation and Configuration Guide.

Hive sources and targets on Amazon S3

Effective in version 10.1.1 Update 2, Big Data Management supports reading and writing to Hive on Amazon S3 buckets for clusters configured with the following Hadoop distributions:

• - Amazon EMR
• - Cloudera
• - HortonWorks
• - MapR
• - BigInsights

For more information, see the Informatica 10.1.1 Update 2 Big Data Management User Guide.

Enterprise Information Catalog

File System resource

Effective in version 10.1.1 Update 2, you can create a File System resource to import metadata from files in Windows and Linux file systems.

For more information, see the Informatica 10.1.1 Update 2 Live Data Map Administrator Guide.

Apache Ranger-enabled clusters

Effective in version 10.1.1 Update 2, you can deploy Enterprise Information Catalog on Apache Ranger-enabled clusters. Apache Ranger provides a security framework to manage the security of the clusters.

Enhanced SSH support for deploying Informatica Cluster Service

Effective in version 10.1.1 Update 2, you can deploy Informatica Cluster Service on hosts where Centrify is enabled. Centrify integrates with an existing Active Directory infrastructure to manage user authentication on remote Linux hosts.

Intelligent Data Lake

Hadoop ecosystem

Effective in version 10.1.1 Update 2, you can use following Hadoop distributions as a Hadoop data lake:

• - Cloudera CDH 5.9
• - Hortonworks HDP 2.3, 2.4, and 2.5
• - Azure HDInsight 3.5
• - Amazon EMR 5.0
• - IBM BigInsights 4.2

Using MariaDB for the Data Preparation Service

Effective in version 10.1.1 Update 2, you can use MariaDB 10.0.28 for the Data Preparation Service repository.

Viewing column-level lineage

Effective in version 10.1.1 Update 2, data analysts can view lineage of individual columns in a table corresponding to activities such as data asset copy, import, export, publication, and upload.

SSL/TLS support

Effective in version 10.1.1 Update 2, you can integrate Intelligent Data Lake with Cloudera 5.9 clusters that are SSL/TLS enabled.

PowerExchange Adapters for Informatica

PowerExchange for Amazon Redshift