Administration of Complex File Connector
Before you use Complex File Connector in tasks, an administrator must verify the following prerequisites:
- •You must have a license to use Cloudera CDH, Amazon Elastic MapReduce (EMR), or Hortonworks HDP packages. When you use a Microsoft HDInsight package, you can read data only from or write data only to a HDFS file system. To read data from or write data to HDFS running on Microsoft HDInsight, you need to use Hortonworks HDP package.
- •You must use Complex File Connector on Linux 64-bit operating system.
Access Kerberos Enabled Hadoop Cluster
Configure the /etc/hosts file and copy the Kerberos configuration file for HDFS instances that use Kerberos authentication.
- 1. Open the /etc/hosts file, located in the /etc directory on the machine where Secure Agent is installed.
- 2. To configure the Secure Agent to work with the Kerberos Key Distribution Center (KDC), make an entry of the KDC hosts in the /etc/hosts file.
- 3. Copy the krb5.conf configuration file from the /etc directory in the Hadoop cluster node to the following location:
<Secure Agent installation directory>/jre/lib/security
- 4. Download the hdfs-site.xml and core-site.xml files from the Hadoop cluster to the following location:
<Secure Agent Installation Directory>/downloads/package-<hadoop distribution_version>/package/<hadoop distribution>/conf
- 5. Restart the Secure Agent.
Access non-Kerberos Enabled Hadoop Cluster
If you have a Cloudera CDH, Amazon EMR, Hortonworks HDP, or Microsoft HDInsight HDFS instance, that does not use Kerberos authentication, running in a Hadoop cluster environment, perform the following steps:
- 1. Copy the hdfs-site.xml file with valid name node configuration to the appropriate <package>/conf folder.
- 2. Restart the Secure Agent.