The April 2025 release of Data Integration includes the following new features and enhancements.
Hierarchical mappers
You can use the following new features and enhancements for hierarchical mappers:
•Use an intelligent structure model as the source schema.
•Search for elements in the source and target schemas.
•Run hierarchical mappers on the Cloud Server.
If you run the hierarchical mapper on the Cloud Server, the hierarchical mapper runs natively and you don't need to configure a runtime environment in Administrator.
For more information, see Components.
Intelligent structure models
This release includes the following enhancements for intelligent structure models:
•You can configure a Large Language Model connection to connect to Azure OpenAI and use the chat model to process and interpret unstructured data within an intelligent structure model.
•You can combine multiple output groups into a single group when you generate normalized hierarchy relationships. This allows the data to be written to a single target resulting in streamlined data processing.
•You can use an intelligent structure model to parse scanned PDF files.
•You can use a custom AI engine to determine the underlying patterns and structures of the input file and create a model that you can use to transform, parse, and generate output groups.
For more information, see Components.
Source control
This release includes the following source control enhancements:
•You can compare additional source-controlled assets in your Git repository. When you commit the following Data Integration assets to your Git repository, Data Integration generates a JSON file with a vc.json extension that you can use to compare the versions:
- Data transfer task
- Dynamic mapping task
- Linear taskflow
- Replication task
- Synchronization task
- Mapping task
- Mapping
- Mapplet
- User-defined function
- Fixed-width component
- Saved query
- Shared sequence
•You can use a cloud-hosted GitLab repository for source control.
For more information, see Asset Management.
String functions
Use the string function UUID_String to create globally-unique IDs for vectors that you write to a vector database.
For more information, see Functions.
Taskflows
You can now use the following types of Data Ingestion and Replication tasks to invoke a taskflow:
•Application Ingestion and Replication tasks
•Database Ingestion and Replication tasks
This feature is available for initial load tasks and incremental load tasks that have any supported source type and a Snowflake target that doesn't use the Superpipe option. When you define an ingestion and replication task, you can select the Execute in Taskflow option on the last wizard page to make the task available to add to taskflows in Data Integration. For incremental load jobs, you can also optionally select the Add Cycle ID option on the page for the target to include cycle ID metadata in the target table. When you configure the taskflow in Data Integration, you can select an ingestion and replication task as an event source and add any appropriate transformation type to transform the ingested data. The taskflow is automatically triggered to start when either the initial load task successfully completes or after each CDC cycle in an incremental load operation. If a CDC cycle ends but the previous taskflow run is still running, the data is queued and waits for the previous taskflow to complete.
For more information, see Taskflows.
Transformations
The following transformations have enhancements this release:
•When you configure a Hierarchy Builder transformation in a mapping, you can assign the decimal data type field as a primary key or foreign key.
•When you configure a Normalizer transformation in a mapping, you can create group-level fields with nested levels to normalize multiple-occurring groups of fields in each source row.