This release includes enhancements to the following connectors.
Amazon Athena Connector
You can configure an EC2 instance to assume an IAM role and use temporary security credentials to connect to Amazon Athena.
Amazon Redshift V2 Connector
You can push window functions to Amazon Redshift in a mapping in SQL ELT mode.
For the list of functions that you can push down to Amazon Redshift, see the Amazon Redshift V2 Connector documentation.
Amazon S3 V2 Connector
This release includes the following enhancements for Amazon S3 V2 Connector:
•You can use the document file format to read PDF files in a mapping in advanced mode.
•When you use discover structure and document file format in a mapping in advanced mode, you can incrementally load source files from a directory to read and process only the files that have changed since the last time the mapping ran.
Azure AI Connector
You can run your Azure AI Search Connector mappings and tasks in the Azure serverless environment.
Databricks Connector
This release includes the following enhancements for Databricks Connector:
•You can read delta format files from Amazon S3 and Microsoft Azure Data Lake Storage Gen2 sources in a mapping in SQL ELT mode and mapping task configured for SQL ELT optimization.
•You can use the Databricks cluster version 15.4 LTS at runtime to process Databricks mappings and tasks.
Google BigQuery V2 Connector
This release includes the following enhancements for Google BigQuery V2 Connector:
•You can use Workload Identity Federation authentication to connect to Google BigQuery.
•You can edit the precision of binary, decimal, and string data types, and the scale of the decimal data type for Google Cloud Storage or Amazon S3 sources in a SQL ELT mode mapping.
•You can read and write JSON data using Google BigQuery Storage API or hybrid connection mode.
Kafka Connector
You can write Avro data as wire format with complex schemas, such as nested and hierarchical structures, to Confluent Kafka with a schema registry by using Intelligent Structure Discovery.
Microsoft Azure Data Lake Storage Gen2 Connector
This release includes the following enhancements for Microsoft Azure Data Lake Storage Gen2 Connector:
•When you use discover structure and document file format in a mapping in advanced mode, you can incrementally load source files from a directory to read and process only the files that have changed since the last time the mapping ran.
•You can write to a partitioned target when you use the delta format to write to a Delta Lake.
Microsoft Azure Synapse SQL Connector
You can use the copy command to load data from Microsoft Azure Data Lake Storage Gen2 to Microsoft Azure Synapse SQL in a mapping task configured for SQL ELT optimization.
Microsoft Excel Connector
You can read multiple Microsoft Excel files from a directory using a single Microsoft Excel connection. You do not need to create separate connections for each file.
Microsoft Fabric Data Warehouse Connector
This release includes the following enhancements for Microsoft Fabric Data Warehouse Connector:
Mapping in SQL ELT mode
•You can edit the precision and scale of a field in the source or lookup object in a mapping in SQL ELT mode that reads from Microsoft Azure Data Lake Storage Gen2.
•You can push window functions to Microsoft Fabric Data Warehouse.
For the list of supported functions, see the Microsoft Fabric Data Warehouse Connector documentation.
Microsoft Fabric OneLake Connector
You can read from and create a Delta Lake in mappings and mappings in advanced mode.
Microsoft SQL Server Connector
This release includes the following enhancements for Microsoft SQL Server Connector:
•You can run a mapping using one of the following upgraded driver versions:
•You can configure and run the environment SQL queries in the same session within a Target transformation.
MongoDB V2 Connector
This release includes the following enhancements for MongoDB V2 Connector:
•You can configure rows for an insert, update, upsert, or delete operation when you write data to a MongoDB V2 target. This enhancement doesn't apply to mappings in advanced mode.
•You can use LDAP authentication to connect to MongoDB in a mapping. This enhancement doesn't apply to mappings in advanced mode.
•You can read data from a CDC source and write to a MongoDB V2 target in a mapping. This enhancement doesn't apply to mappings in advanced mode.
•You can read from or write to Azure CosmosDB enabled with the MongoDB API in a mapping.
NetSuite Connector
You can now use the NetSuite Web Service Description Language (WSDL) versions 2024_2 to 2022_1 to access NetSuite data.
ODBC Connector
You can connect to Teradata database version 20.x using the Teradata subtype in an ODBC connection.
Open Table Connector
This release includes the following enhancements for Open Table Connector:
•You can read from and write to Apache Iceberg tables that are managed by the Polaris catalog and stored in Amazon S3 using OAuth2 client credentials authentication.
•When you set up the Open Table connection, you can configure an EC2 role to assume a different IAM role to authenticate access to resources in Amazon S3 storage.
•When you read from and write to Apache Iceberg tables, you can run SQL commands before and after reading or writing data. You can also truncate the target table before loading data into the target.
Oracle Autonomous Database Connector
This release includes the following enhancements for Oracle Autonomous Database Connector:
•When you read from an Oracle Autonomous Database source, you can read the data directly without staging it.
•You can perform the following lookups on an Oracle Autonomous Database object in a mapping:
- Cached and uncached lookups
- Connected and unconnected lookups
•You can perform the following actions on the data when you write to an Oracle Autonomous Database target:
- Update, upsert, delete, and data-driven operations
- Override the default SQL query used to write data
- Run SQL commands before and after the write operation
- Truncate the target table before loading data into it
- Create the target table at runtime
Oracle Cloud Object Storage Connector
You can perform the following actions on objects when you read from an Oracle Cloud Object Storage source:
•Read all files directly under a specified directory, excluding files in its subdirectories.
•Read files from a directory path that is overridden during runtime.
•Use wildcard characters in file names and directory paths during runtime to select files dynamically.
•Recursively read files within all subdirectories under a specified directory.
Oracle Connector
This release includes the following enhancements for Oracle Connector:
•You can run a mapping using one of the following upgraded driver versions:
- JDBC EBF driver: 6.0.0.001506
- JDBC Latest driver: 6.0.0.001574
To use these JDBC drivers, you need to update the Secure Agent configuration. For more information about configuring these drivers, see the How to configure the JDBC driver for Oracle Connector Knowledge Base article.
•You can join tables across the schemas when you use multiple objects as the source type.
Slack Connector
You can use the Conversation_History operation to read the list of messages and events from a specific conversation.
Snowflake Data Cloud Connector
This release includes the following enhancements for Snowflake Data Cloud Connector:
•You can write to an Apache Iceberg target table created at run time.
•You can optimize performance for a mapping task by configuring pass-through partitioning when you read from Snowflake. This enhancement doesn't apply to mappings in advanced mode.
Teradata Connector
You can connect to Teradata database version 20.x with TTU version 20.x and JDBC driver version 20.x.