Connectors and Connections > Data Ingestion and Replication connection properties > Amazon S3 V2 connection properties
  

Amazon S3 V2 connection properties

When you set up an Amazon S3 V2 connection, configure the connection properties.
The following table describes the Amazon S3 V2 connection properties:
Property
Description
Connection Name
Name of the connection.
Each connection name must be unique within the organization. Connection names can contain alphanumeric characters, spaces, and the following special characters: _ . + -,
Maximum length is 255 characters.
Description
Type
The Amazon S3 V2 connection type.
Runtime Environment
Name of the runtime environment where you want to run the tasks.
You cannot run an application ingestion and replication task or a database ingestion and replication task on a Hosted Agent or serverless runtime environment.
Access Key
Access key to access the Amazon S3 bucket.
Enter the access key value based on the following authentication methods:
  • - Basic authentication. Enter the actual access key value.
  • - IAM authentication. Don't enter the access key value.
  • - Temporary security credentials using assume role. Enter the secret access key of an IAM user with no permissions to access Amazon S3 bucket.
  • - Assume role for EC2. Don't enter the access key value.
  • - Credential profile file authentication. Don't enter the access key value.
  • - Federated user single sign-on. Don't enter the secret access key value.
Secret Key
Secret access key to access the Amazon S3 bucket. The secret key is associated with the access key and uniquely identifies the account.
Enter the secret access key value based on the following authentication methods:
  • - Basic authentication. Enter the actual access secret value.
  • - IAM authentication. Don't enter the access secret value.
  • - Temporary security credentials using assume role. Enter access secret of an IAM user with no permissions to access Amazon S3 bucket.
  • - Assume role for EC2. Don't enter the access key value.
  • - Credential profile file authentication. Don't enter the access secret value.
  • - Federated user single sign-on. Don't enter the access secret value.
IAM Role ARN
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role assumed by the user to use the dynamically generated temporary security credentials.
Enter the value of this property if you want to use the temporary security credentials to access the AWS resources.
This property is not applicable to an application ingestion and replication task.
Note: Even if you remove the IAM role that enables the agent to access the Amazon S3 bucket and create a connection, the test connection is successful.
For more information about how to get the ARN of the IAM role, see the AWS documentation.
External Id
Provides a more secure access to the Amazon S3 bucket when the Amazon S3 bucket is in a different AWS account.
Use EC2 Role to Assume Role
Enables the EC2 role to assume another IAM role specified in the IAM Role ARN option.
Note: The EC2 role must have a policy attached with a permission to assume an IAM role from the same or different account.
By default, the Use EC2 Role to Assume Role check box is not selected.
Note: Enter a value for the IAM Role ARN property when you enable this property for a streaming ingestion and replication task.
Folder Path
Bucket name or complete folder path to the Amazon S3 objects.
For tasks other than application ingestion and replication and database ingestion and replication tasks, don't use a slash at the end of the folder path. For example, <bucket name>/<my folder name>.
For application ingestion and replication and database ingestion and replication tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Master Symmetric Key
A 256-bit AES encryption key in the Base64 format when you use client-side encryption. You can generate a key using a third-party tool.
Doesn't apply to a streaming ingestion and replication task.
Customer Master Key ID
The customer master key ID or alias name generated by AWS Key Management Service (AWS KMS) or the Amazon Resource Name (ARN) of your custom key for cross-account access.
You must generate the customer master key for the same region where the Amazon S3 bucket resides.
You can specify the following master keys:
  • - Customer generated customer master key. Enables client-side or server-side encryption.
  • - Default customer master key. Enables client-side or server-side encryption. Only the administrator user of the account can use the default customer master key ID to enable client-side encryption.
Doesn't apply to a streaming ingestion and replication task.
S3 Account Type
The type of the Amazon S3 account.
Select from the following options:
  • - Amazon S3 Storage. Enables you to use the Amazon S3 services.
  • - S3 Compatible Storage. Enables you to use the endpoint for a third-party storage provider such as Scality RING or MinIO.
Default is Amazon S3 storage.
REST Endpoint
The S3 storage endpoint required for S3 compatible storage.
Enter the S3 storage endpoint in HTTP or HTTPs format.
For example, http://s3.isv.scality.com.
Region Name
The AWS region of the bucket that you want to access.
Select one of the following regions:
  • - Africa(Cape Town)
  • - Asia Pacific(Mumbai)
  • - Asis Pacific(Jakarta)
  • - Asia Pacific (Osaka)
  • - Asia Pacific(Seoul)
  • - Asia Pacific(Singapore)
  • - Asia Pacific(Sydney)
  • - Asia Pacific(Tokyo)
  • - Asia Pacific(Hong Kong)
  • - AWS GovCloud(US)
  • - AWS GovCloud(US-East)
  • - Canada(Central)
  • - China(Bejing)
  • - China(Ningxia)
  • - EU(Ireland)
  • - EU(Frankfurt)
  • - EU(London)
  • - EU(Milan)
  • - EU(Paris)
  • - EU(Stockholm)
  • - South America(Sao Paulo)
  • - Middle East(Bahrain)
  • - Middle East(UAE)
  • - US East(N. Virginia)
  • - US East(Ohio)
  • - US ISO East
  • - US ISOB East(Ohio)
  • - US ISO West
  • - US West(N. California)
  • - US West(Oregon)
Default is US East(N. Virginia).
Federated SSO IdP
SAML 2.0-enabled identity provider for the federated user single sign-on to use with the AWS account.
Amazon S3 V2 connector supports only the ADFS 3.0 identity provider. Select None if you don't want to use federated user single sign-on.
Note: Federated user single sign-on is not applicable to application ingestion and replication tasks, database ingestion and replication tasks, and streaming ingestion and replication tasks.
Other Authentication Type
Select one the following authentication types:
  • - NONE
  • - Credential Profile File Authentication
Select the Credential Profile File Authentication option to access the Amazon S3 credentials from a credential file that contains the access key and secret key.
Enter the credential profile file path and the profile name to establish the connection with Amazon S3.
You can use permanent IAM credentials or temporary session tokens when you configure the Credential Profile File Authentication.
Default is NONE.
Credential Profile File Path
Specifies the credential profile file path.
If you don't enter the credential profile path, the Secure Agent uses the credential profile file present in the following default location in your home directory:
~/.aws/credentials
Note: Database Ingestion and Replication has not been certified with the Credential Profile File Path and Profile Name connection properties. Database Ingestion and Replication finds AWS credentials by using the default credential provider chain that is implemented by the DefaultAWSCredentialsProviderChain class, which includes the credential profile file.
Profile Name
Name of the profile in the credential profile file used to get the credentials.
If you don't enter the profile name, the credentials from the default profile in the credential profile file are used.
S3 VPC Endpoint Type
The VPC endpoint type for Amazon S3.
You can enable private communication with Amazon S3 by selecting a VPC endpoint.
Select one of the following VPC endpoint types:
  • - None
  • - Gateway Endpoint
  • - Interface Endpoint
Default is None.
Doesn't apply to an application ingestion and replication task or database ingestion and replication task.
Endpoint DNS Name for Amazon S3
The DNS name for the Amazon S3 interface endpoint.
Enter the DNS name in the following format:
bucket.<DNS name of the interface endpoint>
Doesn't apply to an application ingestion and replication task or database ingestion and replication task.
STS VPC Endpoint Type
Applicable when you select the S3 VPC interface endpoint.
The VPC endpoint type for AWS STS.
When you select IAM Role ARN or Federated SSO IdP, configure the STS VPC endpoint.
Doesn't apply to an application ingestion and replication task, database ingestion and replication task, or streaming ingestion and replication task.
Endpoint DNS Name for AWS STS service
The DNS name for the AWS STS interface endpoint.
Doesn't apply to an application ingestion and replication task or database ingestion and replication task.
KMS VPC Endpoint Type
Applicable when you select the interface endpoint.
The VPC endpoint type for the AWS KMS.
Doesn't apply to an application ingestion and replication task or database ingestion and replication task.
Endpoint DNS Name for AWS KMS service
The DNS name for the AWS KMS interface endpoint.
Doesn't apply to an application ingestion and replication task or database ingestion and replication task.

Federated user single sign-on connection properties

Configure the following properties when you select ADFS 3.0 in Federated SSO IdP:
Property
Description
Federated User Name
User name of the federated user to access the AWS account through the identity provider.
Federated User Password
Password for the federated user to access the AWS account through the identity provider.
IdP SSO URL
Single sign-on URL of the identity provider for AWS.
Doesn't apply to a streaming ingestion and replication task.
SAML Identity Provider ARN
ARN of the SAML identity provider that the AWS administrator created to register the identity provider as a trusted provider.
Role ARN
ARN of the IAM role assumed by the federated user.

Credential Profile File Authentication

You can provide the credentials required to establish the connection with Amazon S3 through the credential profile file that contains an access key and secret key. The credential profile file contains an access key, a secret key, and a session token when you use temporary security credentials.
You can use permanent IAM credentials or temporary security credentials with a session token when you use credential profile file authentication.
If you do not specify the credential profile file path, the default credential file path is used. If you do not specify the profile name, the credentials are used from the default profile in the credential file.
Consider the following rules for a credential profile file:
A sample credential profile file:
[default]

aws_access_key_id = 1233333

aws_secret_access_key = abcabcabc


[test-profile]

aws_access_key_id = 1233333

aws_secret_access_key = abcabcabc

aws_session_token = jahaheieomdrtflmlioerp
The aws_access_key_id and aws_secret_access_key specify the AWS access key and secret key used as part of credentials to authenticate the user.
The aws_session_token specifies an AWS session token used as part of the credentials to authenticate the user. A session token is required only if you specify temporary security credentials.

Private communication with Amazon S3

You can enable private communication with Amazon S3 by configuring a gateway endpoint or interface endpoint on AWS console and in the Amazon S3 V2 connection.
You can configure Amazon S3 V2 Connector to establish private communication with Amazon S3 without exposing your traffic to the public internet. To access Amazon S3, ensure that the Secure Agent is a part of the subnet in the AWS Virtual Private Cloud (VPC). AWS S3 VPC endpoint enables an S3 request to be routed to the Amazon S3 service, without having to connect a subnet to an internet gateway. You can create an interface endpoint or a gateway endpoint.
For more information, see Configuring private communication with Amazon S3 using the Amazon S3 V2 Connector.