Rules and guidelines for mappings

• •To read and write complex files and to avoid java heap space error, set the JVM options for type DTM to increase the -Xms and -Xmx values in the system configuration details of the Secure Agent. The recommended -Xms and -Xmx values are 512 MB and 1024 MB respectively.
• •When the source data contains special characters, the Secure Agent replaces the special characters with underscore (_) character in the target file.
• •When you use the Snappy compression format to write data to Microsoft Fabric OneLake, the mapping retains a snappy-1.1.8****-libsnappyjava.so file in the /tmp directory on the agent machine after it runs successfully.
• •When you run a mapping that reads from and writes to Microsoft Fabric OneLake and the source file name contains spaces along with special characters such as & and #, the data preview and mapping fails.
• •When you read from a file or directory in Microsoft Fabric OneLake, ensure that the file name or directory name does not contain any emoji character.
• •If the file name or lakehouse path has percentage (%), the data preview and mapping fails at runtime.
• •To use the authenticated proxy server, add dns_v4_first on entry in /etc/squid/squid.conf file in the proxy server and restart the proxy server.
• •If you configure the non-proxy host with HTTPS to access complex files and add dfs endpoints in the non-proxy host, you cannot bypass the proxy server. The endpoint addresses remain listed in the proxy logs.
• •When you use the JVM options to configure the HTTPS proxy server and read and write complex files, the mapping fails at runtime.
• •When you create a target at runtime, consider the following rules for the object name:
• - Always prefix the object name with the workspace name. For example, <Workspace_Name>/<Lakehouse_Path>/Files/source.csv.
• - If you specify a forward slash (/) in the Lakehouse Path connection property, you must specify the object name as <Workspace_Name>/<Lakehouse_Path>/Files/source.csv
• - If you specify the complete path in the Lakehouse Path connection property, you must specify the object name as <Workspace_Name>/Files/source.csv
• •When you create a new target and select Handle Special Characters to append time stamp to the file name, the file name override is not honored in a mapping.
• •When you parameterize a target object using a parameter file, ensure that the lakehouse name has a lakehouse extension. Else, the mapping fails.
• •When you configure partial parameterization of lakehouse override in a mapping in advanced mode, the mapping fails.
• •When you parameterize the connection and object and select the Allow parameter to be overridden at runtime option, you can only use a parameter file to override the default object and connection.
• •When you partially parameterize the File Name Override value, specify the format of the parameter name as $$<parameter_name> in a mapping and ${$parameter_name} in a mapping in advanced mode.
• •When you write data to complex files where some of the fields are unmapped, the unmapped fields are written to the target with null data. However, when you write data to flat files, only the mapped fields are written to the target.
• •When you run a mapping in advanced mode to read from Microsoft Fabric OneLake and write to Microsoft Azure Data Lake Storage Gen2, or read from Microsoft Azure Data Lake Storage Gen2 and write to Microsoft Fabric OneLake, ensure to configure different authentication types to access the source and target endpoints. For example, use Service Principal authentication to access Microsoft Fabric OneLake and Shared Key authentication to access the Microsoft Azure Data Lake Storage Gen2 account.
• •When you create a mapping in advanced mode, you cannot preview data for the individual transformations. However, you can preview the data object when you create a mapping in advanced mode.
• •When you run a parameterized mapping where you specify a relative lakehouse path in the target connection and root in the lakehouse path in the parameter file, the file path is incorrectly formed at runtime. To overcome this, replace the <lakehouse_name/Files> in the lakehouse path with the workspace name in the parameterized target object. For example, if the lakehouse path in the target object path is Lakehouse/Files/csv/dir/062_Onelake, replace Lakehouse/Files with the workspace name in the parameter file.
• •You cannot read zero-byte files when you run mappings in advanced mode.
• •You cannot use AVRO file format when you run mappings in advanced mode.
• •You can use the following special characters in a directory name or a file name:
`, ~, !, @, #, $, ^, &, *, -, _, =, +, and ()
• •When you read objects from directories or subdirectories in Microsoft Fabric OneLake, ensure that the selected object exists in the source. If the object is deleted, the mapping fails with an object not found error.