Rules and guidelines for mappings and mapping tasks
Use the following guidelines when you create mappings:
•When you use the Snappy compression format to write data to Google Cloud Storage, the mapping retains a snappy-1.1.8****-libsnappyjava.so file in the temp directory on the agent machine after it runs successfully.
•When you override the source or target file using the Source File Name or Target File Name advanced property, ensure that the file contains valid header columns.
•When you create a target based on a Avro source file and the source contains fields of Null or BigInt data type, the Secure Agent creates a field of the Integer data type in the Avro target file.
•When you create a Google Cloud Storage target at runtime and the source contains an empty table or view, the mapping fails, but the Secure Agent creates an empty file in Google Cloud Storage.
•When you read data from a Google Cloud Storage source and write data to a Google Cloud Storage target and the source and target connections use different project IDs associated with the same Google service account, the mapping fails.
•When you parameterize the object, you can override it with a file located within the specified object path and its subfolders. For example, if you define the object path as bucket name/folder1/file1.txt in a mapping task, you can replace the file with another one from the directory bucket name/folder1 or any of its subfolders. However, if you want to override it with a file from a different directory, you need to parameterize the Google Cloud storage path and file name in the advanced properties of the source or target, and then provide a distinct object path in the parameter file.
•When you enable the Optimize Object Metadata Import in the Google Cloud Storage connection property and use a parameterized object in the mapping to create a new target at runtime, where you specify only the folder name or the file name without the bucket name in the parameter file, the mapping fails. You need to specify the entire path including the bucket name in the parameter file.
•When you write data to a target in CSV or JSON format, you might encounter name collision errors with the local staging file. To prevent this issue, add the following parameter in any JVMOption field in the System Configuration Details section of the Secure Agent:
Service
Data Integration Server
Type
DTM
Name
JVMOption
Value
-DGoogleStorage_RandomizeLocalFileName=true
•You cannot perform the following tasks when the Use Parallel Composite Upload target advanced property is enabled in a mapping:
- Use Avro or Parquet file format to upload the files to Google Cloud Storage.
- Compress the data when you write to the target.
- Encrypt the data when you write to the target.
•When you use Iceberg file format in a mapping, you can’t reorder the fields in a table.