Hadoop Files Connector > Introduction to Hadoop Files Connector > ComplexFileProcessor Connector Administration

ComplexFileProcessor Connector Administration

Before you use Hadoop Files Connector in tasks, an administrator must verify the following prerequisites:

•You must have a license to use Cloudera CDH, Amazon Elastic MapReduce (EMR), or Hortonworks HDP packages. When you use a Microsoft HDInsight package, you can read data only from or write data only to a HDFS file system. To read data from or write data to HDFS running on Microsoft HDInsight, you need to use Hortonworks HDP package.
•You must use Hadoop Files Connector on Linux 64-bit operating system.

ComplexFileProcessor Connector Migration Task

To migrate ComplexFileProcessor Connector from the Informatica Cloud Spring 2017 release to the Informatica Intelligent Cloud Services Data Integration Spring 2018 April release, you must perform the following tasks:

1If you used multiple licenses within an organization in the Informatica Cloud Spring 2017 release, you must create separate sub-organizations for each Hadoop distribution that you want to use and assign license for the Hadoop distribution.

2Copy the required configuration files to the following directory:

<Secure Agent installation directory>/downloads/package-<distribution_name>/package/<distribution_name>_<version>/conf

3Restart the Secure Agent for the changes to take effect.

Access Kerberos Enabled Hadoop Cluster

Configure the /etc/hosts file and copy the Kerberos configuration file for HDFS instances that use Kerberos authentication.

1Open the /etc/hosts file, located in the /etc directory on the machine where Secure Agent is installed.
2To configure the Secure Agent to work with the Kerberos Key Distribution Center (KDC), make an entry of the KDC hosts in the /etc/hosts file.
3Copy the krb5.conf configuration file from the /etc directory in the Hadoop cluster node to the following location:

<Secure Agent installation directory>/jdk/jre/lib/security

4If the cluster is SSL enabled, import the certificate alias file to the following location:

<Secure Agent installation directory>/jdk/jre/lib/security/cacerts

5Restart the Secure Agent.

Access the non-Kerberos Enabled Hadoop Cluster

If you have a Cloudera, Amazon EMR, Hortonworks, or Microsoft HDInsight instance that does not use Kerberos authentication and runs in a Hadoop cluster environment, perform the following steps:

1Copy the hdfs-site.xml file with valid name node configuration to the appropriate <package>/conf folder.
2Restart the Secure Agent.