This release includes enhancements to the following connectors.
Amazon Athena Connector
You can configure an EC2 instance to assume an IAM role and use temporary security credentials to connect to Amazon Athena.
Amazon Redshift V2 Connector
You can push window functions to Amazon Redshift in a mapping in SQL ELT mode.
For the list of functions that you can push down to Amazon Redshift, see the Amazon Redshift V2 Connector documentation.
Amazon S3 V2 Connector
This release includes the following enhancements for Amazon S3 V2 Connector:
•You can use the document file format to read PDF files in a mapping in advanced mode.
•When you use discover structure and document file format in a mapping in advanced mode, you can incrementally load source files in a directory to read and process only the files that have changed since the last time the mapping ran.
Azure AI Connector
When you create an Azure AI Search connection, you can use the serverless runtime environment configured in Azure.
Databricks Connector
This release includes the following enhancements for Databricks Connector:
•Mapping in SQL ELT mode
You can read Delta format files from an Amazon S3 or a Microsoft Azure Data Lake Storage Gen2 source.
•You can use the Databricks cluster version 15.4 LTS at runtime to process Databricks mappings and tasks.
Google BigQuery V2 Connector
This release includes the following enhancements for Google BigQuery V2 Connector:
•You can use Workload Identity Federation authentication to connect to Google BigQuery.
•In a SQL ELT mode mapping, you can edit the precision of binary, decimal, and string data types, as well as the scale of the decimal data type, for Google Cloud Storage or Amazon S3 sources.
•You can read and write JSON data using Google BigQuery Storage or hybrid connection mode.
Kafka Connector
You can write data with complex schemas, such as nested and hierarchical structures, to Confluent Kafka with a schema registry by using Intelligent Structure Discovery.
Microsoft Azure Data Lake Storage Gen2 Connector
This release includes the following enhancements for Microsoft Azure Data Lake Storage Gen2 Connector:
•You can incrementally load source files in a directory that contains file formats such as Discover Structure and Document.
•You can write to a partitioned target when you use the delta format to write to a Delta Lake.
Microsoft Azure Synapse SQL Connector
This release includes the following enhancements for Microsoft Azure Synapse SQL Connector:
•SQL ELT optimization enhancement
You can use the copy command to load data from Microsoft Azure Data Lake Storage Gen2 to Microsoft Azure Synapse SQL.
•When you read data from Microsoft Azure Synapse SQL and the source table has a UTF-8 compatible collation, you can read data directly from Microsoft Azure Synapse SQL without staging the data.
Microsoft Excel Connector
You can read multiple Microsoft Excel files from a directory using a single Microsoft Excel connection. You do not need to create separate connections for each file.
Microsoft Fabric Data Warehouse Connector
This release includes the following enhancements for Microsoft Fabric Data Warehouse Connector:
Mapping in SQL ELT mode
•You can edit the precision and scale of a field in the source or lookup object in a mapping in SQL ELT mode that reads from Microsoft Azure Data Lake Storage Gen2.
•You can push window functions to Microsoft Fabric Data Warehouse.
For the list of supported functions, see the Microsoft Fabric Data Warehouse Connector documentation.
Microsoft Fabric OneLake Connector
You can read from or create a Delta Lake in mappings and mappings in advanced mode.
Microsoft SQL Server Connector
This release includes the following enhancements for Microsoft SQL Server Connector:
•You can run a mapping using one of the following upgraded driver versions:
•You can configure the environment SQL queries to execute in the same session within a Target transformation.
MongoDB V2 Connector
This release includes the following enhancements for MongoDB V2 Connector:
•You can perform a data driven operation on the MongoDB V2 target in a mapping. This feature doesn't apply to mappings in advanced mode.
•You can use LDAP authentication type to connect to MongoDB in a mapping. This feature doesn't apply to mappings in advanced mode.
•You can read data from a CDC source and write to a MongoDB V2 target in a mapping. This feature doesn't apply to mappings in advanced mode.
•You can read from or write to Azure CosmosDB enabled with the MongoDB API in a mapping.
NetSuite Connector
You can now use the NetSuite Web Service Description Language (WSDL) versions 2024_2 to 2022_1 to access NetSuite data.
ODBC Connector
You can connect to Teradata database version 20.x using the Teradata subtype in the ODBC connection.
Open Table Connector
This release includes the following enhancements for Open Table Connector:
•You can read from and write to Apache Iceberg tables, which are managed by the REST API based Polaris catalog using OAuth2 client credentials authentication and stored in Amazon S3.
•When you set up the Open Table connection, you can configure the EC2 role to assume a different IAM role to authenticate access to resources in Amazon S3 storage.
•When you read from and write to Apache Iceberg tables, you can run SQL commands before and after reading or writing data. You can also truncate the target table before loading data into the target.
Oracle Autonomous Database Connector
This release includes the following enhancements for Oracle Autonomous Database Connector:
•When you read from the Oracle Autonomous Database source, you can read the data directly without staging it.
•You can perform the following lookups on an Oracle Autonomous Database object in a mapping:
- Cached and uncached lookups
- Connected and unconnected lookups
•You can perform the following actions on the data when you write to the Oracle Autonomous Database target:
- Update, upsert, and delete operations
- Data-driven operations based on specified conditions
- Override the default SQL query used to write data
- Run SQL commands before and after the agent writes the data
- Truncate the target table before loading data into it
- Create the target table at runtime
Oracle Cloud Object Storage Connector
You can perform the following actions on objects when you read from the Oracle Cloud Object Storage source:
•Read all files directly under a specified directory, excluding files in its subdirectories.
•Read files from a directory path that is overridden during runtime.
•Use wildcard characters in file names and directory paths during runtime to select files dynamically.
•Recursively read files within all subdirectories under a specified directory.
Slack Connector
You can use the Conversation_History operation to read the list of messages and events from a specific conversation.
Snowflake Data Cloud Connector
This release includes the following enhancements for Snowflake Data Cloud Connector:
•Mapping in SQL ELT mode
You can write to an Apache Iceberg target table created at run time in a mapping. This feature also applies to mappings and mappings in advanced mode.
•You can skip the post-SQL statements specified for the target and get an error message in the session log incase of a mapping failure. This feature doesn't apply to mappings in advanced mode.
•You can optimize performance for a mapping task by configuring the pass-through partitioning when you read from Snowflake. This feature doesn't apply to mappings in advanced mode.
Teradata Connector
You can connect to Teradata database version 20.x with TTU version 20.x and JDBC driver version 20.x.