Connectors and Connections > Data Ingestion and Replication connection properties > Amazon S3 V2 connection properties

Amazon S3 V2 connection properties

Create an Amazon S3 V2 connection to read from and write to Amazon S3.

Prepare for authentication

You can configure multiple authentication types to access Amazon S3.

Before you configure the connection properties, you need to keep the authentication details handy based on the authentication type that you want to use.

•Basic authentication requires access key and secret key values from your AWS account.
•IAM authentication requires attaching policies to the EC2 role to grant access to specific folder paths and access Amazon S3 objects .
•AssumeRole with EC2 role authentication requires you to enable the EC2 role to assume another IAM role specified by the IAM Role ARN.
•AssumeRole with IAM user authentication requires the access key and secret key values of the IAM user and the ARN of the IAM role.
•Credential profile file authentication requires the credential profile file path and profile name.
•Federated user single sign-on authentication requires the user name and password of the federated user, IdP SSO URL, ARN of the SAML identity provider, and ARN of the IAM role assumed by the federated user. You can only use ADFS 3.0 (IDP) for SSO.

Create a minimal Amazon IAM policy

You can configure an IAM policy through the AWS console. Use AWS IAM authentication to securely control access to Amazon S3 resources.

Use the following minimum required policies for users to read data from an Amazon S3 bucket:

•GetObject
•ListBucket

Use the following minimum required policies for users to write data to an Amazon S3 bucket:

•PutObject
•GetObject
•DeleteObject
•ListBucket
•ListBucketMultipartUploads. Applicable only for mappings in advanced mode.

The following sample policy shows the minimal Amazon IAM policy to write data to an Amazon S3 bucket:

{
"Version": "2012-10-17",
"Statement":
[
{
"Effect": "Allow",
"Action":[
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads"
],
"Resource": [
"arn:aws:s3:::<bucket_name>/*",
"arn:aws:s3:::<bucket_name>"
]
}
]
}

For bucket level access in advanced mode, you need to provide the AllowListBucketMultipartUploads permission at the bucket level in addition to the ListBucketMultipartUploads permission.

The following sample policy shows the minimal Amazon IAM policy to access the S3 bucket at the bucket level in advanced mode:

{
"Sid": "AllowListBucketMultipartUploads",
"Action": [
"s3:ListBucketMultipartUploads"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::infa.qa.minimum.access.bucket"
]
},

For mappings in advanced mode, you can use different AWS accounts within the same AWS region. Make sure that the Amazon IAM policy confirms access to the AWS accounts used in the mapping.

IAM authentication

To configure IAM authentication, the Secure Agent needs to run on an Amazon Elastic Compute Cloud (EC2) system. If you prefer not to specify the keys or use the IAM role ARN, then assign the minimum policy to the EC2 with access to the S3 bucket.

If you do not provide the access key and the secret key in the connection, Amazon S3 V2 Connector uses AWS credentials provider chain that looks for credentials in the following order:

1The AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY or AWS_ACCESS_KEY and AWS_SECRET_KEY environment variables.
2The aws.accessKeyId and aws.secretKey java system properties.
3The credential profiles file at the default location, ~/.aws/credentials.
4The instance profile credentials delivered through the Amazon EC2 metadata service.

Perform the following steps to configure IAM authentication on EC2:

1Create a minimal Amazon IAM policy.
2Create the Amazon EC2 role. The Amazon EC2 role is used when you create an EC2 system. For more information about creating the Amazon EC2 role, see the AWS documentation.
3Link the minimal Amazon IAM policy with the Amazon EC2 role.
4Create an EC2 instance. Assign the Amazon EC2 role that you created in step 2 to the EC2 instance.
5Install the Secure Agent on the EC2 system.

AssumeRole using EC2 role and IAM user

You can configure AssumeRole using EC2 role or IAM user to connect to Amazon S3.

You can use the temporary security credentials using AssumeRole to access AWS resources from the same or different AWS accounts.

When you configure AssumeRole using EC2 role or IAM user, ensure that you have the sts:AssumeRole permission and a trust relationship established within the AWS accounts to use the temporary security credentials. The trust relationship is defined in the trust policy of the IAM role when you create the role. The IAM role adds the EC2 role or IAM user as a trusted entity allowing the EC2 role or IAM user to use the temporary security credentials and access the AWS accounts.

For more information about how to establish the trust relationship, see the AWS documentation.

When the trusted EC2 role or IAM user requests for the temporary security credentials, the AWS Security Token Service (AWS STS) dynamically generates the temporary security credentials that are valid for a specified period and provides the credentials to the trusted EC2 role or IAM user.

AssumeRole using EC2 role

To configure an EC2 role to assume the IAM role provided in the IAM Role ARN connection property, select the Use EC2 Role to Assume Role check box in the Amazon S3 V2 connection properties.

The Amazon EC2 role can assume another IAM role from the same or different AWS account without requiring a permanent access key and secret key. The Amazon EC2 role can also assume another IAM role from a different region.

Consider the following prerequisites before you configure AssumeRole using EC2 role:

•Install the Secure Agent on an AWS service such as Amazon EC2.
•The EC2 role attached to the AWS EC2 service must not have access to Amazon S3 but needs to have permission to assume another IAM role.
•The IAM role that needs to be assumed by the EC2 role must have a permission policy and a trust policy attached to it.

AssumeRole using IAM user

To configure AssumeRole using IAM user, provide the value of the IAM Role ARN connection property when you create an Amazon S3 V2 connection. The IAM Role ARN uniquely identifies the AWS resources. Then, specify the time duration in seconds during which you can use the temporarily security credentials in the Temporary Credential Duration advanced source and target properties.

You need to follow some guidelines when you configure AssumeRole using IAM user. For more information, see .

External ID

You can specify the external ID of your AWS account for a more secure access to the Amazon S3 bucket when the Amazon S3 bucket is in same or different AWS account.

You can optionally specify the external ID in the AssumeRole request to the AWS Security Token Service (STS).

The external ID must be a string.

The following sample shows an external ID condition in the assumed IAM role's trust policy:

"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AWS_Account_ID : user/user_name"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "dummy_external_id"
}
}
}
]

AssumeRole policy

To use the temporary security credentials to access the AWS resources, both the IAM user and IAM role require policies.

The following section lists the policies required for the IAM user and IAM role:

IAM user: An IAM user must have the sts:AssumeRole policy to use the temporary security credentials in the same or different AWS account.; The following sample policy allows an IAM user to use the temporary security credentials in an AWS account:; The following sample policy allows an IAM user for the China region to use the temporary security credentials in an AWS account:

IAM role: An IAM role must have a sts:AssumeRole policy and a trust policy attached with the IAM role to allow the IAM user to access the AWS resource using the temporary security credentials. The policy specifies the AWS resource that the IAM user can access and the actions that the IAM user can perform. The trust policy specifies the IAM user from the AWS account that can access the AWS resource.

Credential profile file authentication

You can provide the credentials required to establish the connection with Amazon S3 through the credential profile file.

If you do not specify the credential profile file path, the default credential file path is used. If you do not specify the profile name, the credentials are used from the default profile in the credential file.

Consider the following rules for a credential profile file:

•The credential file must be on the same machine where you installed the Secure Agent.
•The credential profile file name must end with .credentials.
•If you do not specify the credential profile path, the Secure Agent uses the credential profile file present in the following default location in your home directory:

~/.aws/credentials

Note: On Windows, you can refer to your home directory by using the environment variable %UserProfile%. On Unix-like systems, you can use the environment variable $HOME.

The following sample shows a credential profile file:

[default]

aws_access_key_id = 1233333

aws_secret_access_key = abcabcabc

[test-profile]

aws_access_key_id = 1233333

aws_secret_access_key = abcabcabc

aws_session_token = jahaheieomdrtflmlioerp

The aws_access_key_id and aws_secret_access_key are the AWS access key and secret key used as part of credentials to authenticate the user.

The aws_session_token is the AWS session token used as part of the credentials to authenticate the user. A session token is required only if you specify temporary security credentials.

Connect to Amazon S3

Let's configure the Amazon S3 connection properties to connect to Amazon S3.

Before you begin

Before you get started, you'll need to get information from your Amazon S3 account based on the authentication type that you want to configure.

Check out Prepare for authentication to learn more about the authentication prerequisites.

Connection details

The following table describes the basic connection properties:

Property	Description
Connection Name	Name of the connection. Each connection name must be unique within the organization. Connection names can contain alphanumeric characters, spaces, and the following special characters: _ . + -, Maximum length is 255 characters.
Description	Description of the connection. Maximum length is 4000 characters.
Type	Amazon S3 V2
Use Secret Vault	Stores sensitive credentials for this connection in the secrets manager that is configured for your organization. This property appears only if secrets manager is set up for your organization. This property is not supported by Data Ingestion and Replication. When you enable the secret vault in the connection, you can select which credentials that the Secure Agent retrieves from the secrets manager. If you don't enable this option, the credentials are stored in the repository or on a local Secure Agent, depending on how your organization is configured. For information about how to configure and use a secrets manager, see Secrets manager configuration.
Runtime Environment	The name of the runtime environment where you want to run tasks. You cannot run an application ingestion and replication task or a database ingestion and replication on a Hosted Agent or serverless runtime environment.

Authentication types

You can configure basic, AWS Identity and Access Management (IAM), temporary security credentials, assume role for EC2, credential profile file, and federated user single sign-on authentication types to access Amazon S3.

Select the required authentication method and then configure the authentication-specific parameters.

Basic authentication

Basic authentication requires access key and secret key values from your AWS account.

The following table describes the basic connection properties for basic authentication:

Property	Description
Access Key	Access key to access the Amazon S3 bucket.
Secret Key	Secret key to access the Amazon S3 bucket. The secret key is associated with the access key and uniquely identifies the account.
Folder Path	Amazon S3 bucket name or the folder path within the Amazon S3 bucket where the Amazon S3 objects are stored. For example, <bucket name>/<my folder name> For application ingestion and replication and database ingestion and replication tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Region Name	The AWS region of the bucket that you want to access. Select one of the following regions: - Africa(Cape Town) - Asia Pacific(Mumbai) - Asis Pacific(Jakarta) - Asia Pacific (Osaka) - Asia Pacific(Seoul) - Asia Pacific(Singapore) - Asia Pacific(Sydney) - Asia Pacific(Tokyo) - Asia Pacific(Hong Kong) - AWS GovCloud(US) - AWS GovCloud(US-East) - Canada(Central) - China(Bejing) - China(Ningxia) - EU(Ireland) - EU(Frankfurt) - EU(London) - EU(Milan) - EU(Paris) - EU(Stockholm) - South America(Sao Paulo) - Middle East(Bahrain) - Middle East(UAE) - US East(N. Virginia) - US East(Ohio) - US ISO East - US ISOB East(Ohio) - US ISO West - US West(N. California) - US West(Oregon) Default is US East(N. Virginia).

IAM authentication

IAM authentication requires only the folder path to the Amazon S3 objects. The EC2 role must have access to the folder.

The following table describes the basic connection properties for AWS IAM authentication:

Property

Description

Folder Path

Amazon S3 bucket name or the folder path within the Amazon S3 bucket where the Amazon S3 objects are stored.

For example, <bucket name>/<my folder name>

For application ingestion and replication and database ingestion and replication tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.

Region Name

The AWS region of the bucket that you want to access.

Select one of the following regions:

- Africa(Cape Town)
- Asia Pacific(Mumbai)
- Asis Pacific(Jakarta)
- Asia Pacific (Osaka)
- Asia Pacific(Seoul)
- Asia Pacific(Singapore)
- Asia Pacific(Sydney)
- Asia Pacific(Tokyo)
- Asia Pacific(Hong Kong)
- AWS GovCloud(US)
- AWS GovCloud(US-East)
- Canada(Central)
- China(Bejing)
- China(Ningxia)
- EU(Ireland)
- EU(Frankfurt)
- EU(London)
- EU(Milan)
- EU(Paris)
- EU(Stockholm)
- South America(Sao Paulo)
- Middle East(Bahrain)
- Middle East(UAE)
- US East(N. Virginia)
- US East(Ohio)
- US ISO East
- US ISOB East(Ohio)
- US ISO West
- US West(N. California)
- US West(Oregon)

Default is US East(N. Virginia).

AssumeRole via EC2 role authentication

AssumeRole via EC2 role authentication requires you to enable the EC2 role to assume another IAM role specified in the IAM Role ARN option.

The following table describes the basic connection properties for AssumeRole via EC2 role authentication:

Property	Description
Folder Path	Amazon S3 bucket name or the folder path within the Amazon S3 bucket where the Amazon S3 objects are stored. For example, <bucket name>/<my folder name> For application ingestion and replication and database ingestion and replication tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Region Name	The AWS region of the bucket that you want to access. Select one of the following regions: - Africa(Cape Town) - Asia Pacific(Mumbai) - Asis Pacific(Jakarta) - Asia Pacific (Osaka) - Asia Pacific(Seoul) - Asia Pacific(Singapore) - Asia Pacific(Sydney) - Asia Pacific(Tokyo) - Asia Pacific(Hong Kong) - AWS GovCloud(US) - AWS GovCloud(US-East) - Canada(Central) - China(Bejing) - China(Ningxia) - EU(Ireland) - EU(Frankfurt) - EU(London) - EU(Milan) - EU(Paris) - EU(Stockholm) - South America(Sao Paulo) - Middle East(Bahrain) - Middle East(UAE) - US East(N. Virginia) - US East(Ohio) - US ISO East - US ISOB East(Ohio) - US ISO West - US West(N. California) - US West(Oregon) Default is US East(N. Virginia).
IAM Role ARN	The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role assumed by the user to use the dynamically generated temporary security credentials. Enter the ARN value if you want to use the temporary security credentials to access AWS resources. This property is not applicable to an application ingestion and replication task. Note: Even if you remove the IAM role that grants the agent access to the Amazon S3 bucket, the test connection is successful. For more information about how to get the ARN of the IAM role, see the AWS documentation.
External ID	The external ID of your AWS account. External ID provides a more secure access to the Amazon S3 bucket when the Amazon S3 bucket is in a different AWS account.
Use EC2 Role to Assume Role	Enables the EC2 role to assume another IAM role specified in the IAM Role ARN option. By default, this property is not selected. Note: The EC2 role must have a policy attached with permissions to assume an IAM role from the same or different account. Note: Enter a value for the IAM Role ARN property when you enable this property for a streaming ingestion and replication task.

AssumeRole via IAM user authentication

AssumeRole via IAM user authentication requires the access key and secret key values of the IAM user and the ARN of the IAM role.

The following table describes the basic connection properties for AssumeRole via IAM user authentication:

Property	Description
Access Key	Access key to access the Amazon S3 bucket.
Secret Key	Secret key to access the Amazon S3 bucket. The secret key is associated with the access key and uniquely identifies the account.
Folder Path	Amazon S3 bucket name or the folder path within the Amazon S3 bucket where the Amazon S3 objects are stored. For example, <bucket name>/<my folder name> For application ingestion and replication and database ingestion and replication tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Region Name	The AWS region of the bucket that you want to access. Select one of the following regions: - Africa(Cape Town) - Asia Pacific(Mumbai) - Asis Pacific(Jakarta) - Asia Pacific (Osaka) - Asia Pacific(Seoul) - Asia Pacific(Singapore) - Asia Pacific(Sydney) - Asia Pacific(Tokyo) - Asia Pacific(Hong Kong) - AWS GovCloud(US) - AWS GovCloud(US-East) - Canada(Central) - China(Bejing) - China(Ningxia) - EU(Ireland) - EU(Frankfurt) - EU(London) - EU(Milan) - EU(Paris) - EU(Stockholm) - South America(Sao Paulo) - Middle East(Bahrain) - Middle East(UAE) - US East(N. Virginia) - US East(Ohio) - US ISO East - US ISOB East(Ohio) - US ISO West - US West(N. California) - US West(Oregon) Default is US East(N. Virginia).
IAM Role ARN	The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role assumed by the user to use the dynamically generated temporary security credentials. Enter the value of this property if you want to use the temporary security credentials to access the AWS resources. This property is not applicable to an application ingestion and replication task. Note: Even if you remove the IAM role that enables the agent to access the Amazon S3 bucket and create a connection, the test connection is successful. For more information about how to get the ARN of the IAM role, see the AWS documentation.
External ID	The external ID of your AWS account. External ID provides a more secure access to the Amazon S3 bucket when the Amazon S3 bucket is in a different AWS account.

Credential profile file authentication

Credential profile file authentication requires the credential profile file path and profile name.

The following table describes the basic connection properties for credential profile file authentication:

Property	Description
Folder Path	Amazon S3 bucket name or the folder path within the Amazon S3 bucket where the Amazon S3 objects are stored. For example, <bucket name>/<my folder name> For application ingestion and replication and database ingestion and replication tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Region Name	The AWS region of the bucket that you want to access. Select one of the following regions: - Africa(Cape Town) - Asia Pacific(Mumbai) - Asis Pacific(Jakarta) - Asia Pacific (Osaka) - Asia Pacific(Seoul) - Asia Pacific(Singapore) - Asia Pacific(Sydney) - Asia Pacific(Tokyo) - Asia Pacific(Hong Kong) - AWS GovCloud(US) - AWS GovCloud(US-East) - Canada(Central) - China(Bejing) - China(Ningxia) - EU(Ireland) - EU(Frankfurt) - EU(London) - EU(Milan) - EU(Paris) - EU(Stockholm) - South America(Sao Paulo) - Middle East(Bahrain) - Middle East(UAE) - US East(N. Virginia) - US East(Ohio) - US ISO East - US ISOB East(Ohio) - US ISO West - US West(N. California) - US West(Oregon) Default is US East(N. Virginia).
Other Authentication Type	Determines whether you want to use the credential profile file authentication to connect to Amazon S3. Select one the following authentication types: - NONE. Select if you do not want to credential profile file authentication. - Credential Profile File Authentication. Select to use credential profile file authentication to access the Amazon S3 credentials from a credential file. Enter the credential profile file path and profile name to connect to Amazon S3. You can use permanent IAM credentials or temporary session tokens when you configure the credential profile file authentication. Default is NONE.
Credential Profile File Path	The credential profile file path. If you don't enter the credential profile path, the Secure Agent uses the credential profile file available in the following default location in your home directory: ~/.aws/credentials Note: Database Ingestion and Replication has not been certified with the Credential Profile File Path and Profile Name connection properties. Database Ingestion and Replication finds AWS credentials by using the default credential provider chain that is implemented by the DefaultAWSCredentialsProviderChain class, which includes the credential profile file.
Profile Name	Name of the profile in the credential profile file used to get credentials to access Amazon S3 resources. If you don't enter the profile name, the credentials from the default profile in the credential profile file are used.

Federated single sign-on authentication

Federated user single sign-on authentication requires the user name and password of the federated user, IdP SSO URL, ARN of the SAML identity provider, and ARN of the IAM role assumed by the federated user. You can only use ADFS 3.0 (IDP) for SSO.

The following table describes the basic connection properties for federated single sign-on authentication:

Property	Description
Folder Path	Amazon S3 bucket name or the folder path within the Amazon S3 bucket where the Amazon S3 objects are stored. For example, <bucket name>/<my folder name> For application ingestion and replication and database ingestion and replication tasks, add a trailing slash. For example: <bucket name>/<my folder name>/.
Region Name	The AWS region of the bucket that you want to access. Select one of the following regions: - Africa(Cape Town) - Asia Pacific(Mumbai) - Asis Pacific(Jakarta) - Asia Pacific (Osaka) - Asia Pacific(Seoul) - Asia Pacific(Singapore) - Asia Pacific(Sydney) - Asia Pacific(Tokyo) - Asia Pacific(Hong Kong) - AWS GovCloud(US) - AWS GovCloud(US-East) - Canada(Central) - China(Bejing) - China(Ningxia) - EU(Ireland) - EU(Frankfurt) - EU(London) - EU(Milan) - EU(Paris) - EU(Stockholm) - South America(Sao Paulo) - Middle East(Bahrain) - Middle East(UAE) - US East(N. Virginia) - US East(Ohio) - US ISO East - US ISOB East(Ohio) - US ISO West - US West(N. California) - US West(Oregon) Default is US East(N. Virginia).
Federated SSO IdP	SAML 2.0-enabled identity provider for the federated user single sign-on to use with the AWS account. You can only use ADFS 3.0 (IDP) for SSO. Select None if you don't want to use federated user single sign-on. Note: Federated user single sign-on is not applicable to application ingestion and replication tasks,database ingestion and replication tasks, and streaming ingestion and replication tasks.
Federated User Name	User name of the federated user to access the AWS account through the identity provider.
Federated User Password	Password for the federated user to access the AWS account through the identity provider.
IdP SSO URL	Single sign-on URL of the identity provider for AWS. Doesn't apply to astreaming ingestion and replication task.
SAML Identity Provider ARN	ARN of the SAML identity provider that the AWS administrator created to register the identity provider as a trusted provider.
Role ARN	ARN of the IAM role assumed by the federated user.

Advanced settings

The following table describes the advanced connection properties:

Property	Description
S3 Account Type	The type of the Amazon S3 account. Select from the following options: - Amazon S3 Storage. Enables you to use the Amazon S3 services. - S3 Compatible Storage. Enables you to use the endpoint for a third-party storage provider such as Scality RING or MinIO. Default is Amazon S3 storage.
REST Endpoint	The S3 storage endpoint required for S3 compatible storage. Enter the S3 storage endpoint in HTTP or HTTPs format. For example, http://s3.isv.scality.com.
S3 VPC Endpoint Type	The type of Amazon Virtual Private Cloud endpoint for Amazon S3. You can use a VPC endpoint to enable private communication with Amazon S3. Select one of the following options: - None. Select if you do not want to use a VPC endpoint. - Gateway Endpoint. Select to establish private communication with Amazon S3 through an interface endpoint. A gateway endpoint is a target for a route in your route table that is used to forward S3 traffic to the S3 gateway endpoint. - Interface Endpoint. Select to establish private communication with Amazon S3 through an interface endpoint which uses a private IP address from the IP address range of your subnet. It serves as an entry point for traffic destined to an AWS service. Default is None. Doesn't apply to anapplication ingestion and replication task or database ingestion and replication task.
Endpoint DNS Name for Amazon S3	The DNS name for the Amazon S3 interface endpoint. Enter the DNS name in the following format: bucket.<DNS name of the interface endpoint> Doesn't apply to anapplication ingestion and replication task or database ingestion and replication task.
STS VPC Endpoint Type	The type of Amazon Virtual Private Cloud endpoint for AWS Security Token Service. This option applies when you select the S3 VPC interface endpoint and when use AssumeRole via IAM user or EC2 role authentication or Federated SSO IdP authentication. Doesn't apply to anapplication ingestion and replication task, streaming ingestion and replication task, ordatabase ingestion and replication task.
Endpoint DNS Name for AWS STS	The DNS name for the AWS STS interface endpoint. Doesn't apply to an application ingestion and replication task or database ingestion and replication task.
KMS VPC Endpoint Type	The type of Amazon Virtual Private Cloud endpoint for AWS Key Management Service. This option applies when you select the S3 VPC interface endpoint and required when you specify the customer master key ID. Doesn't apply to an application ingestion and replication task or database ingestion and replication task.
Endpoint DNS Name for AWS KMS	The DNS name for the AWS KMS interface endpoint. Doesn't apply to an application ingestion and replication task or database ingestion and replication task.
Master Symmetric Key	A 256-bit AES encryption key in the Base64 format when you use client-side encryption. You can generate a key using a third-party tool. Doesn't apply to anapplication ingestion and replication task, database ingestion and replication task, or streaming ingestion and replication task.
Customer Master Key ID	The customer master key ID or alias name generated by AWS Key Management Service (AWS KMS) or the Amazon Resource Name (ARN) of your custom key for cross-account access. You must generate the customer master key for the same region where the Amazon S3 bucket resides. You can specify the following master keys: - Customer generated customer master key. Enables client-side or server-side encryption. - Default customer master key. Enables client-side or server-side encryption. Only the administrator user of the account can use the default customer master key ID to enable client-side encryption. Doesn't apply to anapplication ingestion and replication task, database ingestion and replication task, or streaming ingestion and replication task.

Private communication with Amazon S3

You can enable private communication with Amazon S3 by configuring a gateway endpoint or interface endpoint on AWS console and in the Amazon S3 V2 connection.

You can configure Amazon S3 V2 Connector to establish private communication with Amazon S3 without exposing your traffic to the public internet. To access Amazon S3, ensure that the Secure Agent is a part of the subnet in the AWS Virtual Private Cloud (VPC). AWS S3 VPC endpoint enables an S3 request to be routed to the Amazon S3 service, without having to connect a subnet to an internet gateway. You can create an interface endpoint or a gateway endpoint.

For more information, see Configuring private communication with Amazon S3 using the Amazon S3 V2 Connector.

Server-side encryption with KMS

To use the customer master key managed by AWS Key Management Service (AWS KMS) and enable the encryption with KMS, you need to create a KMS policy.

You can perform the following operations to use the temporary security credentials and enable the encryption with KMS:

•GenerateDataKey
•DescribeKey
•Encrypt
•Decrypt
•ReEncrypt

See the following sample KMS policy for reference:

{
"Version":"2012-10-17", "Statement":[{ "Effect":"Allow", "Action":[ "kms:GenerateDataKey", "kms:DescribeKey", "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*" ], "Resource": ["arn:aws:kms:region:account:key/<KMS_key>"]
}
]
}

When you configure KMS and access an Amazon S3 endpoint in the China region, use the following sample policy:

Proxy server settings

If your organization uses an outgoing proxy server to connect to the Internet, the Secure Agent connects to Informatica Intelligent Cloud Services through the proxy server.

You can configure the Secure Agent to use the proxy server on Windows and Linux.

You can use only an unauthenticated proxy server to connect to Informatica Intelligent Cloud Services.

To configure the proxy settings for the Secure Agent, perform the following tasks:

•Configure the Secure Agent through the Secure Agent Manager on Windows or shell command on Linux.

For instructions, see Configure the proxy settings on Windows or Configure the proxy settings on Linux.

•Configure the JVM options for the DTM in the Secure Agent properties. For instructions, see the Proxy server settings Knowledge Base article.
•Configure the proxy server properties in the proxy.ini file.

Note: If you enable both HTTP and SOCKS proxies, SOCKS proxy is used by default. If you want to use HTTP proxy instead of SOCKS proxy, set the value of the DisableSocksProxy property to true in the System property.

Bypass proxy server

If your organization uses an outgoing proxy server to connect to the internet, the Secure Agent connects to Informatica Intelligent Cloud Services through the proxy server.

However, if you want to exclude certain IP addresses and host names from the proxy, you can bypass the proxy. Set the InfaAgent.NonProxyHost property in the proxy.ini file and the -Dhttp.nonProxyHosts property in the JVM options of the Secure Agent properties and include the IP addresses and host names that you want to exclude.

The following table shows the proxy setting that you can configure through the proxy.ini file or the JVM options:

Proxy configuration	Proxy Flag Setting
Proxy.ini	InfaAgent.NonProxyHost=localhost\|<your_bucket_name>.s3.\|127.\|[\:\:1] For example, to bypass a single S3 Bucket iam.qa.bucket, use the following proxy setting: InfaAgent.NonProxyHost=localhost\|iam.qa.bucket.s3.\|127.\|[\:\:1] To bypass all S3 buckets, use the following proxy setting: InfaAgent.NonProxyHost=localhost\|*.s3.\|127.\|[\:\:1]
JVM option	-Dhttp.nonProxyHosts=localhost\|<your_bucket_name>.s3.\|127.* \|[\:\:1] For example, to bypass a single S3 Bucket, iam.qa.bucket, use the following proxy setting: -Dhttp.nonProxyHosts=localhost\|iam.qa.bucket.s3.\|127.\|[\:\:1] To bypass all S3 buckets, use the following proxy setting: -Dhttp.nonProxyHosts=localhost\|.s3.\|127.\|[\:\:1]

Bypass the proxy server in advanced mode

To bypass the proxy server, you must update the NonProxyHost value in the proxy.ini file. You can set the property in the agent core path to configure the NonProxyHost in the advanced cluster configuration.

To bypass the proxy at the Amazon S3 endpoint, perform the following steps:

1Edit the proxy.ini file and set the property in the NonProxyHost with the cluster region.
2Enter the appropriate region name in the property in the following format:

InfaAgent.NonProxyHost=localhost|127.*|[\:\:1]|169.254.169.254|.<REGION_NAME>.elb.amazonaws.com|*.<REGION_NAME>.elb.amazonaws.com

The following example shows how you can update the NonProxyHost for the US West region in the proxy.ini file:

3After you edit the proxy.ini file, you must set the property ccs.enable.storage.proxy.settings to false in the runtime properties of the advanced cluster.

Perform the following steps to set the property:

aGo to Administrator.
bIn the Advanced Clusters page, select the name of the configuration that you want to edit from the list of advanced configurations.
cSet the property ccs.enable.storage.proxy.settings to false and save the cluster configuration in the Runtime Properties for the particular cluster.

The following image shows the configured cluster runtime properties:

Rules and guidelines for AssumeRole via IAM user authentication

Consider the following guidelines for Assume Role via IAM user authentication:

•The IAM user or IAM role that requests for the temporary security credentials must not have access to any AWS resources.
•Only authenticated IAM users or IAM roles can request for the temporary security credentials from the AWS Security Token Service (AWS STS).
•Before you run a task, ensure that you have enough time to use the temporary security credentials for running the task. You cannot extend the time duration of the temporary security credentials for an ongoing task.

For example, when you read from and write to Amazon S3 and if the temporary security credentials expire, you cannot extend the time duration of the temporary security credentials which causes the task to fail.

•After the temporary security credentials expire, AWS does not authorize the IAM users or IAM roles to access the resources using the credentials. You must request for new temporary security credentials before the previous temporary security credentials expire in a mapping.
•Do not use the root user credentials of an AWS account to use the temporary security credentials. You must use the credentials of an IAM user to use the temporary security credentials.
•If both the source and target in a mapping point to the same Amazon S3 bucket, use the same Amazon S3 connection in the Source and Target transformations. If you use two different Amazon S3 connections, configure the same values in the connection properties for both the connections.
•If the source and target in a mapping point to different Amazon S3 buckets, you can use two different Amazon S3 connections.

You can configure different values in the connection properties for both the connections. However, you must select the Use EC2 Role to Assume Role check box in the connection property. You must also specify the same value for the Temporary Credential Duration field in the source and target properties.

•In a mapping, if you configure two or more Amazon S3 data sources from the same Amazon S3 bucket with different IAM roles, each IAM role must be able to access the data source of the other IAM role.
•In a mapping with two data sources, if you set up one Amazon S3 data source to use user credentials and another to use an IAM role, consider the following rules:

- The IAM user for the first data source must also be able to assume the IAM role of the second Amazon S3 data source.
- The IAM role that you configured for the second data source must also have access to the first Amazon S3 data source.

Rules and guidelines for AWS regions

Consider the following rules and guidelines when you configure the region name of the bucket in the connection properties:

•When you change the runtime environment of an existing connection, the region is changed to the default region US East (N. Virginia). Select the region manually to change the default region.
•When you edit an existing connection, you see duplicate entries for regions. Use the regions that contain spaces because these regions are populated from AWS SDK. For example, use US West (Oregon) instead of US West(Oregon).

Rules and guidelines for S3 compatible storage

Consider the following rules and guidelines when you configure S3 compatible storage in an Amazon S3 V2 connection:

•You can only configure basic authentication when you use S3 compatible storage.
•You cannot configure SSE-KMS encryption for the Scality RING S3 compatible storage. You cannot configure SSE and SSE-KMS encryption for MinIO S3 compatible storage.
•You cannot configure SQL ELT optimization to load data from Amazon S3 sources to Amazon Redshift.