Microsoft Azure Data Lake Storage Gen2 Connector > Troubleshooting > Troubleshooting a mapping
  

Troubleshooting a mapping

Time zone for the Date and Timestamp data type fields in Parquet or Avro file formats defaults to the Secure Agent host machine time zone.  
When you run a mapping to read from or write to fields of the Date and Timestamp data types in the Parquet or Avro file formats, the time zone defaults to the Secure Agent host machine time zone.
To change the Date and Timestamp to the UTC time zone, configure the JVMOptions in the Secure Agent.  
Perform the following steps to configure the JVM options in the Secure Agent:
  1. 1Select Administrator > Runtime Environments.
  2. 2On the Runtime Environments page, select the Secure Agent for which you want to configure the JVMOptions.
  3. 3In the upper-right corner, click Edit.
  4. 4In the System Configuration Details section, select Data Integration Server as the service and DTM as the type.
  5. 5Edit the JVMOption field and set the value to -Duser.timezone=UTC.
  6. 6Click Save.
Mapping failed with a Java heap space error
When you read from or write to large data sets in Microsoft Azure Data Lake Storage Gen2, certain mappings might fail with the following error:
[ERROR] java.lang.OutOfMemoryError: Java heap space
You must increase the heap size to run the mappings successfully. The recommended heap size is 1 GB.
Perform the following steps to configure the JVM options in the Secure Agent to increase the memory for the Java heap size:
  1. 1Select Administrator > Runtime Environments.
  2. 2On the Runtime Environments page, select the Secure Agent for which you want to increase memory from the list of available Secure Agents.
  3. 3In the upper-right corner, click Edit.
  4. 4In the System Configuration Details section, select Data Integration Server as the service and DTM as the type.
  5. 5Edit the JVMOption field and set the value to -Xmx1024m.
    6. Note: The recommended heap size is 1 GB. You can increase the heap size based on the data you want to process.
  6. 6Click Save.
Unicode character strings are written incorrectly to the target
When you run a Microsoft Azure Data Lake Storage Gen2 mapping on a Linux machine, parameterize the source connection and object, and append data to the existing target, the Unicode character strings in the data are not written correctly to the target.
To resolve this issue, set the environment variable LC_ALL="en_US.UTF-8" in the Secure Agent on the Linux machine, and restart the Secure Agent.
Mapping fails if the directory name or subdirectory name contains Unicode characters
When you read from or write to a flat file in Microsoft Azure Data Lake Storage Gen2 and if the directory name or subdirectory name contains Unicode characters, the mapping fails.
To resolve this issue, set the environment variable LC_ALL="en_US.UTF-8" in the Secure Agent, and restart the Secure Agent.
Non-English characters in the source are incorrectly written to the target when you use the Append write strategy
When you append data to a flat file in a Microsoft Azure Data Lake Storage Gen2 target, the non-English characters in the source are incorrectly written to the target.
To write the non-English characters correctly to the target, configure the JVMOptions in the Secure Agent.  
Perform the following steps to configure the JVM options in the Secure Agent:
  1. 1Select Administrator > Runtime Environments.
  2. 2On the Runtime Environments page, select the Secure Agent for which you want to configure the JVMOptions.
  3. 3In the upper-right corner, click Edit.
  4. 4In the System Configuration Details section, select Data Integration Server as the service and DTM as the type.
  5. 5Edit the JVMOption field and set the value to -Dfile.encoding=UTF-8.
  6. 6Click Save.
Mapping fails due to failure to access the /tmp directory
When you run a Microsoft Azure Data Lake Storage Gen2 mapping, the Secure Agent stages the files in a temporary staging folder. By default, the folder for staging data is /tmp.
Ensure that you have the read and write permissions to the /tmp folder.
Perform the following steps to change the temporary staging folder:
  1. 1Select Administrator > Runtime Environments.
  2. 2On the Runtime Environments page, select the Secure Agent for which you want to configure the JRE_OPTS field.
  3. 3In the upper-right corner, click Edit.
  4. 4In the System Configuration Details section, select Data Integration Server as the service and DTM as the type.
  5. 5Edit the JRE_OPTS field and set the value to -Djava.io.tmpdir=<DIR>.
  6. 6Click Save.