Connect to Amazon S3

Property	Description
Connection Name	Name of the connection. Each connection name must be unique within the organization. Connection names can contain alphanumeric characters, spaces, and the following special characters: _ . + -, Maximum length is 255 characters.
Description	Description of the connection. Maximum length is 4000 characters.
Type	Amazon S3 V2
Use Secret Vault	Stores sensitive credentials for this connection in the secrets manager that is configured for your organization. This property appears only if secrets manager is set up for your organization. This property is not supported by Mass Ingestion. When you enable the secret vault in the connection, you can select which credentials that the Secure Agent retrieves from the secrets manager. If you don't enable this option, the credentials are stored in the repository or on a local Secure Agent, depending on how your organization is configured. For information about how to configure and use a secrets manager, see Secrets manager configuration.
Runtime Environment	The name of the runtime environment where you want to run tasks. Select a Secure Agent, Hosted Agent, or serverless runtime environment. You cannot run an application ingestion task or a database ingestion task on a Hosted Agent or serverless runtime environment.

Authentication types

Basic authentication

Basic authentication requires access key and secret key values from your AWS account.

The following table describes the basic connection properties for basic authentication:

Property	Description
Access Key	Access key to access the Amazon S3 bucket.
Secret Key	Secret key to access the Amazon S3 bucket. The secret key is associated with the access key and uniquely identifies the account.
Folder Path	Bucket name or complete folder path to the Amazon S3 objects. For example, <bucket name>/<my folder name> For application ingestion and database ingestion tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Region Name	The AWS region of the bucket that you want to access. Select one of the following regions: - Africa(Cape Town) - Asia Pacific(Mumbai) - Asis Pacific(Jakarta) - Asia Pacific (Osaka) - Asia Pacific(Seoul) - Asia Pacific(Singapore) - Asia Pacific(Sydney) - Asia Pacific(Tokyo) - Asia Pacific(Hong Kong) - AWS GovCloud(US) - AWS GovCloud(US-East) - Canada(Central) - China(Bejing) - China(Ningxia) - EU(Ireland) - EU(Frankfurt) - EU(London) - EU(Milan) - EU(Paris) - EU(Stockholm) - South America(Sao Paulo) - Middle East(Bahrain) - Middle East(UAE) - US East(N. Virginia) - US East(Ohio) - US ISO East - US ISOB East(Ohio) - US ISO West - US West(N. California) - US West(Oregon) Default is US East(N. Virginia).

IAM authentication

IAM authentication requires only the folder path to the Amazon S3 objects. The EC2 role must have access to the folder. :

The following table describes the basic connection properties for AWS IAM authentication:

Property

Description

Folder Path

Bucket name or complete folder path to the Amazon S3 objects.

For example, <bucket name>/<my folder name>

For application ingestion and database ingestion tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.

Region Name

The AWS region of the bucket that you want to access.

Select one of the following regions:

- Africa(Cape Town)
- Asia Pacific(Mumbai)
- Asis Pacific(Jakarta)
- Asia Pacific (Osaka)
- Asia Pacific(Seoul)
- Asia Pacific(Singapore)
- Asia Pacific(Sydney)
- Asia Pacific(Tokyo)
- Asia Pacific(Hong Kong)
- AWS GovCloud(US)
- AWS GovCloud(US-East)
- Canada(Central)
- China(Bejing)
- China(Ningxia)
- EU(Ireland)
- EU(Frankfurt)
- EU(London)
- EU(Milan)
- EU(Paris)
- EU(Stockholm)
- South America(Sao Paulo)
- Middle East(Bahrain)
- Middle East(UAE)
- US East(N. Virginia)
- US East(Ohio)
- US ISO East
- US ISOB East(Ohio)
- US ISO West
- US West(N. California)
- US West(Oregon)

Default is US East(N. Virginia).

AssumeRole via EC2 role authentication

AssumeRole via EC2 role authentication requires you to enable the EC2 role to assume another IAM role specified in the IAM Role ARN option:

The following table describes the basic connection properties for AssumeRole via EC2 role authentication:

Property	Description
Folder Path	Bucket name or complete folder path to the Amazon S3 objects. For example, <bucket name>/<my folder name> For application ingestion and database ingestion tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Region Name	The AWS region of the bucket that you want to access. Select one of the following regions: - Africa(Cape Town) - Asia Pacific(Mumbai) - Asis Pacific(Jakarta) - Asia Pacific (Osaka) - Asia Pacific(Seoul) - Asia Pacific(Singapore) - Asia Pacific(Sydney) - Asia Pacific(Tokyo) - Asia Pacific(Hong Kong) - AWS GovCloud(US) - AWS GovCloud(US-East) - Canada(Central) - China(Bejing) - China(Ningxia) - EU(Ireland) - EU(Frankfurt) - EU(London) - EU(Milan) - EU(Paris) - EU(Stockholm) - South America(Sao Paulo) - Middle East(Bahrain) - Middle East(UAE) - US East(N. Virginia) - US East(Ohio) - US ISO East - US ISOB East(Ohio) - US ISO West - US West(N. California) - US West(Oregon) Default is US East(N. Virginia).
IAM Role ARN	The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role assumed by the user to use the dynamically generated temporary security credentials. Enter the ARN value if you want to use the temporary security credentials to access AWS resources. This property is not applicable to an application ingestion task. Note: Even if you remove the IAM role that enables the agent to access the Amazon S3 bucket and create a connection, the test connection is successful. For more information about how to get the ARN of the IAM role, see the AWS documentation.
External ID	Provides a more secure access to the Amazon S3 bucket when the Amazon S3 bucket is in a different AWS account.
Use EC2 Role to Assume Role	Enables the EC2 role to assume another IAM role specified in the IAM Role ARN option. By default, this property is not selected. Note: The EC2 role must have a policy attached with permissions to assume an IAM role from the same or different account. Note: Enter a value for the IAM Role ARN property when you enable this property for a streaming ingestion task.

AssumeRole via IAM user authentication

AssumeRole via IAM user authentication requires the access key and secret key values of the IAM user and the ARN of the IAM role.

The following table describes the basic connection properties for AssumeRole via IAM user authentication:

Property	Description
Access Key	Access key to access the Amazon S3 bucket.
Secret Key	Secret key to access the Amazon S3 bucket. The secret key is associated with the access key and uniquely identifies the account.
Folder Path	Bucket name or complete folder path to the Amazon S3 objects. For example, <bucket name>/<my folder name> For application ingestion and database ingestion tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Region Name	The AWS region of the bucket that you want to access. Select one of the following regions: - Africa(Cape Town) - Asia Pacific(Mumbai) - Asis Pacific(Jakarta) - Asia Pacific (Osaka) - Asia Pacific(Seoul) - Asia Pacific(Singapore) - Asia Pacific(Sydney) - Asia Pacific(Tokyo) - Asia Pacific(Hong Kong) - AWS GovCloud(US) - AWS GovCloud(US-East) - Canada(Central) - China(Bejing) - China(Ningxia) - EU(Ireland) - EU(Frankfurt) - EU(London) - EU(Milan) - EU(Paris) - EU(Stockholm) - South America(Sao Paulo) - Middle East(Bahrain) - Middle East(UAE) - US East(N. Virginia) - US East(Ohio) - US ISO East - US ISOB East(Ohio) - US ISO West - US West(N. California) - US West(Oregon) Default is US East(N. Virginia).
IAM Role ARN	The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role assumed by the user to use the dynamically generated temporary security credentials. Enter the value of this property if you want to use the temporary security credentials to access the AWS resources. This property is not applicable to an application ingestion task. Note: Even if you remove the IAM role that enables the agent to access the Amazon S3 bucket and create a connection, the test connection is successful. For more information about how to get the ARN of the IAM role, see the AWS documentation.
External ID	Provides a more secure access to the Amazon S3 bucket when the Amazon S3 bucket is in a different AWS account.

Credential profile file authentication

Credential profile file authentication requires the credential profile file path and profile name.

The following table describes the basic connection properties for credential profile file authentication:

Property	Description
Folder Path	Bucket name or complete folder path to the Amazon S3 objects. For example, <bucket name>/<my folder name> For application ingestion and database ingestion tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Region Name	The AWS region of the bucket that you want to access. Select one of the following regions: - Africa(Cape Town) - Asia Pacific(Mumbai) - Asis Pacific(Jakarta) - Asia Pacific (Osaka) - Asia Pacific(Seoul) - Asia Pacific(Singapore) - Asia Pacific(Sydney) - Asia Pacific(Tokyo) - Asia Pacific(Hong Kong) - AWS GovCloud(US) - AWS GovCloud(US-East) - Canada(Central) - China(Bejing) - China(Ningxia) - EU(Ireland) - EU(Frankfurt) - EU(London) - EU(Milan) - EU(Paris) - EU(Stockholm) - South America(Sao Paulo) - Middle East(Bahrain) - Middle East(UAE) - US East(N. Virginia) - US East(Ohio) - US ISO East - US ISOB East(Ohio) - US ISO West - US West(N. California) - US West(Oregon) Default is US East(N. Virginia).
Other Authentication Type¹	Use the Credential Profile File Authentication to access the Amazon S3 credentials from a credential file that contains the access key and secret key. Select one the following authentication types: - NONE - Credential Profile File Authentication Enter the credential profile file path and profile name to connect to Amazon S3. You can use permanent IAM credentials or temporary session tokens when you configure the credential profile file authentication. Default is NONE.
Credential Profile File Path¹	The credential profile file path. If you don't enter the credential profile path, the Secure Agent uses the credential profile file present in the following default location in your home directory: ~/.aws/credentials Note: Mass Ingestion Databases has not been certified with the Credential Profile File Path and Profile Name connection properties. Mass Ingestion Databases finds AWS credentials by using the default credential provider chain that is implemented by the DefaultAWSCredentialsProviderChain class, which includes the credential profile file.
Profile Name¹	Name of the profile in the credential profile file used to get the credentials. If you don't enter the profile name, the credentials from the default profile in the credential profile file are used.
¹Applies only to mappings.

Federated single sign-on authentication

Federated user single sign-on authentication requires the user name and password of the federated user, IdP SSO URL, ARN of the SAML identity provider, and ARN of the IAM role assumed by the federated user. You can only use ADFS 3.0 (IDP) for SSO.

The following table describes the basic connection properties for federated single sign-on authentication:

Property	Description
Folder Path	Bucket name or complete folder path to the Amazon S3 objects. For example, <bucket name>/<my folder name> For application ingestion and database ingestion tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Region Name	The AWS region of the bucket that you want to access. Select one of the following regions: - Africa(Cape Town) - Asia Pacific(Mumbai) - Asis Pacific(Jakarta) - Asia Pacific (Osaka) - Asia Pacific(Seoul) - Asia Pacific(Singapore) - Asia Pacific(Sydney) - Asia Pacific(Tokyo) - Asia Pacific(Hong Kong) - AWS GovCloud(US) - AWS GovCloud(US-East) - Canada(Central) - China(Bejing) - China(Ningxia) - EU(Ireland) - EU(Frankfurt) - EU(London) - EU(Milan) - EU(Paris) - EU(Stockholm) - South America(Sao Paulo) - Middle East(Bahrain) - Middle East(UAE) - US East(N. Virginia) - US East(Ohio) - US ISO East - US ISOB East(Ohio) - US ISO West - US West(N. California) - US West(Oregon) Default is US East(N. Virginia).
Federated SSO IdP¹	SAML 2.0-enabled identity provider for the federated user single sign-on to use with the AWS account. You can only use ADFS 3.0 (IDP) for SSO. Select None if you don't want to use federated user single sign-on. Note: Federated user single sign-on is not applicable to mappings in advanced mode. Note: Federated user single sign-on is not applicable to application ingestion tasks, database ingestion tasks, and streaming ingestion tasks.
Federated User Name	User name of the federated user to access the AWS account through the identity provider.
Federated User Password	Password for the federated user to access the AWS account through the identity provider.
IdP SSO URL	Single sign-on URL of the identity provider for AWS. Doesn't apply to a streaming ingestion task.
SAML Identity Provider ARN	ARN of the SAML identity provider that the AWS administrator created to register the identity provider as a trusted provider.
Role ARN	ARN of the IAM role assumed by the federated user.

Advanced settings

Property	Description
S3 Account Type	The type of the Amazon S3 account. Select from the following options: - Amazon S3 Storage. Enables you to use the Amazon S3 services. - S3 Compatible Storage. Enables you to use the endpoint for a third-party storage provider such as Scality RING or MinIO. Default is Amazon S3 storage.
REST Endpoint	The S3 storage endpoint required for S3 compatible storage. Enter the S3 storage endpoint in HTTP or HTTPs format. For example, http://s3.isv.scality.com.
S3 VPC Endpoint Type¹	The VPC endpoint type for Amazon S3. You can enable private communication with Amazon S3 by selecting a VPC endpoint. Select one of the following options: - None - Gateway Endpoint - Interface Endpoint Default is None. Doesn't apply to an application ingestion task or database ingestion task.
Endpoint DNS Name for Amazon S3¹	The DNS name for the Amazon S3 interface endpoint. Enter the DNS name in the following format: bucket.<DNS name of the interface endpoint> Doesn't apply to an application ingestion task or database ingestion task.
STS VPC Endpoint Type¹	The VPC endpoint type for AWS STS. Applicable when you select the S3 VPC interface endpoint. When you select IAM Role ARN or Federated SSO IdP, configure the STS VPC endpoint. Doesn't apply to an application ingestion task, streaming ingestion task, or database ingestion task.
Endpoint DNS Name for AWS STS¹	The DNS name for the AWS STS interface endpoint. Doesn't apply to an application ingestion task or database ingestion task.
KMS VPC Endpoint Type¹	Applicable when you select the interface endpoint. The VPC endpoint type for the AWS KMS. When you select Customer Master Key ID, configure the KMS VPC endpoint. Doesn't apply to an application ingestion task or database ingestion task.
Endpoint DNS Name for AWS KMS¹	The DNS name for the AWS KMS interface endpoint. Doesn't apply to an application ingestion task or database ingestion task.
Master Symmetric Key	A 256-bit AES encryption key in the Base64 format when you use client-side encryption. You can generate a key using a third-party tool. Doesn't apply to an application ingestion task, database ingestion task, or streaming ingestion task.
Customer Master Key ID	The customer master key ID or alias name generated by AWS Key Management Service (AWS KMS) or the Amazon Resource Name (ARN) of your custom key for cross-account access. Note: Cross-account access is not available for mappings in advanced mode. You must generate the customer master key for the same region where the Amazon S3 bucket resides. You can specify the following master keys: - Customer generated customer master key. Enables client-side or server-side encryption. - Default customer master key. Enables client-side or server-side encryption. Only the administrator user of the account can use the default customer master key ID to enable client-side encryption. Doesn't apply to an application ingestion task, database ingestion task, or streaming ingestion task.
¹Applies only to mappings.