Hadoop Files Connector > Mappings and Mapping Tasks with Hadoop Files > Hadoop Files Sources in Mappings
  

Hadoop Files Sources in Mappings

To read data from a complex file, configure a Hadoop Files object as the Source transformation in a mapping. You can configure a Source transformation to represent a single complex file source.
Specify the name and description of the Hadoop Files source. Configure the source and advanced properties for the source object.
The following table describes the Hadoop Files source properties that you can configure in a Source transformation:
Source Property
Description
Connection
Name of the source connection or create a connection parameter.
If you choose to create a parameter for the connection, provide the connection type as Toolkit Application or leave it blank.
Source Type
Type of source object. Select Single or Parameter as the source type.
Object
Select the source object from which you want to read data. Though selecting a source object is mandatory, the agent ignores this object. The agent processes the source object specified in File Path in advanced source properties.
Parameter
The parameter for the source object. Create or select the parameter for the source object.
Note: The parameter property appears if you select parameter as the source type.
The following table describes the Hadoop Files source advanced properties that you can configure in a Source transformation:
Advanced Property
Description
File Path
Mandatory. Location of the file or directory from which you want to read data. Maximum length is 255 characters. If the path is a directory, all the files in the directory must have the same file format.
If the file or directory is in HDFS, enter the path without the node URI. For example, /user/lib/testdir specifies the location of a directory in HDFS. The path must not contain more than 512 characters.
If the file or directory is in the local system, enter the fully qualified path. For example, /user/testdir specifies the location of a directory in the local system.
File Format
File format. Default is Binary.
Compression Format
Optional. Compression format of the source files. The compression format for binary files. Select one of the following options:
  • - AUTO
  • - DEFLATE
  • - gzip
  • - bzip2
Select AUTO to detect the compression format based on the source file extension.
Tracing Level
Sets the amount of detail that appears in the log file.
You can choose terse, normal, verbose initialization, or verbose data. Default is normal.