This release includes enhancements to the following connectors.
Common
Effective in this release, the Connections page features a refreshed design, new table layout, and improved search and sorting. A Deprecated label appears on the connector tile if the connector is in maintenance mode. These changes are limited to the Connections user interface, with no changes in the core service functionality.
Amazon Athena Connector
You can use workgroups in Amazon Athena to isolate and manage queries for different users, teams, or applications. You can configure each workgroup with specific settings, such as default S3 output locations and encryption options, to control how queries are processed and results are stored.
You can specify a workgroup in the Amazon Athena JDBC URL.
Amazon Redshift V2 Connector
This release includes the following enhancements for Amazon Redshift V2 Connector:
•Amazon Redshift V2 Connector uses the AWS SDK version 2.30.15.
•You can read data from tables enabled for data sharing in Amazon Redshift. Data sharing allows you to share data across Amazon Redshift clusters without the need to copy or move the data or duplicate the storage.
Amazon S3 V2 Connector
This release includes the following enhancements for Amazon S3 V2 Connector:
•Amazon S3 V2 Connector uses the AWS SDK version 2.30.15.
•You can tag target objects to categorize the storage for the objects when you use Avro, Parquet, JSON, and ORC file formats.
Databricks Connector
This release includes the following enhancements for Databricks Connector:
•SQL ELT optimization enhancements
- You can push the median aggregate function to Databricks.
- You can use a filter argument in an aggregate function.
•You can read from and write to Unity Catalog-managed Apache Iceberg tables in a mapping.
•You can configure the Secure Agent to extend the timeout duration for the temporary credentials issued by AWS Security Token Service when you stage data in Amazon S3.
•You can configure the Secure Agent to retry the connection to the Databricks SQL Warehouse cluster up to 3 times, with a 4.5-minute wait between each attempt if the connection fails.
•You can edit the precision or scale of a field in the source or lookup object in a Databricks mapping in SQL ELT mode that reads from Amazon S3, Google Cloud Storage, or Microsoft Azure Data Lake Storage Gen2.
Google BigQuery V2 Connector
This release includes the following enhancements for Google BigQuery V2 Connector:
•You can read from and write to Apache Iceberg tables in a mapping.
•When a mapping enabled with SQL ELT optimization contains multiple pipelines, you can define the flow run order to load the targets from the pipelines in a specific order.
Google Cloud Storage V2 Connector
You can use the delta file format to read data in a mapping.
JDBC V2 Connector
You can configure a connection and mapping in one environment and then migrate and run the mapping in another environment using a different connection.
Microsoft Azure Data Lake Storage Gen2 Connector
This release includes the following enhancements for Microsoft Azure Data Lake Storage Gen2 Connector:
•When you run a mapping to read from to Apache Iceberg files, you can use the Iceberg format.
•When you run a mapping in advanced mode, you can use the None format type to read files in binary format.
Microsoft Fabric Data Warehouse Connector
You can sort records based on the conditions you specify when you read data from Microsoft Fabric Data Warehouse.
Microsoft SQL Server Connector
When you use Kerberos authentication to connect to Microsoft SQL Server on the Linux platform, multiple users can run mappings without restarting the Data Integration Server.
ODBC Connector
This release includes the following enhancements for ODBC Connector:
•You can configure SCRAM-SHA-256 authentication when you connect to a Greenplum database using an ODBC connection.
•When you use Kerberos authentication to connect to a DB2 database on the Linux platform, multiple users can run mappings without restarting the Data Integration Server.
Open Table Connector
This release includes the following enhancements for Open Table Connector:
•You can read from and write to Apache Iceberg tables that are managed by the AWS Glue Catalog and stored in Amazon S3 in a mapping.
•You can drop the existing target table and then recreate the target table at run time when you use Apache Iceberg tables managed by the AWS Glue Catalog.
Oracle Connector
This release includes the following enhancements for Oracle Connector:
•You can connect to Oracle Database version 23ai using an Oracle connection.
•You can configure the updated ODBC driver version 08.02.3863 available in the EBF folder to run all your mappings and tasks.
This release includes the following enhancements for PostgreSQL Connector:
•You can configure the updated JDBC driver version 6.0.0.001796 available in the latest folder to run all your mappings and tasks.
•You can connect to PostgreSQL database version 16.x using a PostgreSQL connection if you use the JDBC driver from the latest folder.
Salesforce Connector
You can overwrite the Salesforce source, lookup, and target objects at run time with values specified in a parameter file in a mapping.
SAP HANA Connector
You can perform the following actions on the lookup data when you look up data from SAP HANA:
•Override the default SQL query used to look up data.
•Use a custom query to reduce the number of columns to query.
•Filter data before the data enters the data flow.
Snowflake Data Cloud Connector
This release includes the following enhancements for Snowflake Data Cloud Connector:
•Snowflake Data Cloud Connector uses the JDBC driver version 3.25.0, by default, to connect to Snowflake and run your Snowflake Data Cloud jobs.
•You can use a programmatic access token generated in Snowflake when you connect to Snowflake using standard authentication.
•You can limit the number of rejected rows written to the rejected file for the target object in a mapping.
•You can write data of the TIMESTAMPNTZ data type with microsecond or nanosecond precision to a Snowflake target in a mapping enabled with the staging property for the write operation.
Teradata Connector
When you configure a KRB5 authentication and connect to Teradata from Linux, multiple users can run mappings without restarting the Data Integration Server.
Zendesk V2 Connector
You can now use an API token or OAuth 2.0 client credentials authentication to connect to Zendesk.