Hadoop Files V2 Connector > Mappings and tasks with Hadoop Files V2 Connector > File partitioning to read from complex files
  

File partitioning to read from complex files

You can configure fixed partitioning in a mapping that reads data from complex files of the Parquet and Avro file formats.
Enable partitioning when you configure the Source transformation in the Mapping Designer.
On the Partitions tab for the Source transformation, you select fixed partitioning and enter the number of partitions based on the amount of data that you want to read.
The following image shows the configured partitioning:
On the Partitions tab of the Source transformation, the partitioning type is Fixed and the number of partitions is set to 2.
When you run the mapping, the Secure Agent distributes the source data across the partitions to process parallely based on the number of partitions that you specify in the Source transformation. The maximum number of partitions must not exceed 64.