The July 2024 release of Data Integration includes the following new features and enhancements.
Watch the What's New video to learn about new features and enhancements in the July 2024 release.
Dynamic mapping tasks
When you configure a source, target, or lookup object in a dynamic mapping task, you can edit the field metadata with certain connector types. You can edit the field data type, precision, and scale.
For more information about dynamic mapping tasks, see Tasks.
Guided and unified Home page
The Home page provides a guided and unified experience for Data Integration users and Data Ingestion and Replication users.
The panels on the Home page are customized for each user based on criteria like your organization's licenses, the user's roles, and whether the user loads data to a particular cloud data warehouse. For example, the Home page for an Admin user that loads data to Amazon Redshift might look like the following image:
For more information about the new Home page, see Introduction.
Hierarchical mappers
You can use the h2h resource to run a hierarchical mapper that converts hierarchical data to a different hierarchical structure.
Use the following URI to run the hierarchical mapper:
/h2h/api/v1/service/run/<hierarchical mapper ID>
For more information, see REST API Reference.
Hierarchy Builder transformation
In a Hierarchy Builder transformation, you can enable performance mode in output settings to optimize the mapping performance at run time.
For more information, see Transformations.
Hierarchy Parser transformation
In a Hierarchy Parser transformation, you can enable performance mode in input settings to optimize the mapping performance at run time.
For more information, see Transformations.
Intelligent structure models
You can simplify the Excel spreadsheets for better performance to consume less heap memory usage and CPU cores when you parse Excel spreadsheet in Intelligent Structure Discovery. The simplify Excel mode applies only to Excel 2007.
For more information, see Components.
Message Discovery Service in the Data Services transformation
The Message Discovery Service is available in the Data Services transformation. It detects the name of a parser data service from the upstream transformation and invokes the parser data service from the data services repository.
For more information, see Transformations.
Simultaneous task runs
You can run multiple instances of a mapping task based on a mapping in advanced mode at the same time.
For more information about mapping tasks, see Tasks. For more information about running a mapping task using the REST API, see REST API Reference.
Source partitioning in mapping tasks
When you create a mapping task with a parameterized source connection or object, you can configure fixed and key range partitioning for the source.
Source partitioning in mapping tasks is available with certain connector types. To see if a connector supports source partitioning in mapping tasks, see the help for the appropriate connector.
For more information about source partitioning see, Transformations. For more information about mapping tasks, see Tasks.
SQL ELT
This release includes SQL ELT enhancements for Databricks, Google Big Query, and Snowflake Data Cloud:
•You can create a mapping in SQL ELT mode when you extract data from the following sources and load data to Databricks:
- Amazon S3
- Databricks
- Microsoft Azure Data Lake Storage Gen2
•You can create a mapping in SQL ELT mode when you extract data from the following sources and load data to Google BigQuery:
- Amazon S3
- Google BigQuery
- Google Cloud Storage
•You can push a few conditional expression, cortex, date and time, and window functions to Snowflake.
For the list of supported functions, see the Snowflake Data Cloud Connector documentation.
Target Field Group field for target transformations
You can use the new Target Field Group field on the Field Mappings tab to select tables for which you want to configure field mappings. This option appears when the transformation target type is Multi Group.
For more information, see the IMS Connector, Sequential Connector, and VSAM Connector.