Connections > Open Table connection properties > Connect to Open Table
  

Connect to Open Table

Let's configure the Open Table connection properties to connect to AWS Glue Catalog or Hive Metastore.

Before you begin

Before you get started, you will need to add the Amazon Athena or Hive JDBC driver to the Secure Agent machine and configure the authentication-specific prerequisites.
Permanent IAM Credentials authentication requires the access key and secret key values of the IAM user. Keep the access key and secret key handy before creating the connection. For more information about creating an access key and secret key, see the AWS documentation.
To configure Service Principal authentication, you need the Azure account name, client secret, client ID, and tenant ID for your application registered in the Azure Active Directory. Keep the Azure account name, client secret, client ID, and tenant ID handy before creating the connection. For more information about Azure account name, client secret, client ID, and tenant ID for your application, see the Microsoft Azure documentation.
Check out Prerequisites to learn more about how to configure policies and role to access Apache Iceberg or Delta Lake tables.

Open Table formats with associated catalog and storage types

You can choose the Open Table format that you want to use and its associated catalog type and storage type to interact with data.
The following table summarizes the Open Table formats that you can use, their catalog types, storage types, and the authentication options available for each storage type:
Open Table format
Catalog type
Storage type
Authentication type
Apache Iceberg
AWS Glue Catalog
Amazon S3
Permanent IAM Credentials authentication
Hive Metastore
Microsoft Azure Delta Lake Storage Gen2
Service Principal authentication
Delta Lake
AWS Glue Catalog
Amazon S3
Permanent IAM Credentials authentication

Connection details

The following table describes the Open Table connection properties:
Property
Description
Connection Name
Name of the connection.
Each connection name must be unique within the organization. Connection names can contain alphanumeric characters, spaces, and the following special characters: _ . + -,
Maximum length is 255 characters.
Description
Description of the connection. Maximum length is 4000 characters.
Type
Open Table
Use Secret Vault
Stores sensitive credentials for this connection in the secrets manager that is configured for your organization.
This property appears only if secrets manager is set up for your organization.
This property is not supported by Data Ingestion and Replication.
When you enable the secret vault in the connection, you can select which credentials that the Secure Agent retrieves from the secrets manager. If you don't enable this option, the credentials are stored in the repository or on a local Secure Agent, depending on how your organization is configured.
For information about how to configure and use a secrets manager, see Secrets manager configuration.
Runtime Environment
The name of the runtime environment where you want to run tasks.
Select a Secure Agent, Hosted Agent, or serverless runtime environment.
You cannot run a database ingestion task on a Hosted Agent or in a serverless runtime environment.
Open Table Format
The Open Table format that you want to use to read from or write data to a catalog.
Select Apache Iceberg or Delta Lake from the list.

Catalog types

You can select AWS Glue Catalog or Hive Metastore as the catalog type to manage the metadata of the Open Table format that you selected.
Select the catalog type that your Open Table format uses and then configure the catalog specific parameters.

Storage types

You can choose Amazon S3 or Microsoft Azure Data Lake Storage Gen2 as the storage type to store the Open Table format tables.
Select the storage type and configure the storage specific authentication parameters.