Serverless runtime environment setup on AWS

Configure your environment

VPC

A VPC contains the data to process in the serverless runtime environment.

Create a VPC in your AWS account. Enable DNS hostnames and DNS resolution for the VPC.

Also, ensure that at least one of the following scenarios apply to you:

• - Your VPC's DHCP option is set with AmazonProvidedDNS.
• - If you have custom DNS servers in your DHCP option set, ensure that AmazonProvidedDNs is part of the option set or that the DNS servers can resolve EC2 internal hostnames. To ensure that the DNS servers can resolve EC2 internal hostnames, internally redirect the DNS query to AmazonProvidedDNS.

Security group

A security group controls the traffic flow from the serverless runtime environment.

Create a security group in the VPC. The security group is associated with all ENIs that the serverless runtime environment creates. You specify this security group in the serverless runtime environment properties.

Leave the inbound rules empty to restrict all incoming traffic. The outbound rules can either allow all traffic or limit traffic to all Amazon S3 resources and all source and target systems that the serverless runtime environment accesses.

Private subnet to host the ENI

A private subnet hosts the ENI that the serverless runtime environment uses to connect to your VPC.

Create a private subnet and configure a CIDR range to determine the maximum number of IP addresses and therefore, the scalability, of the serverless runtime environment. Configure the CIDR range to have at least 25 IP addresses per serverless runtime environment so that the serverless runtime environment can scale effectively when developers run concurrent workloads.

After your organization administrator creates a serverless runtime environment in Administrator, the serverless runtime environment creates a ENI in your private subnet.

Public subnet for internet access

A public subnet provides internet access through a NAT gateway.

Create a public subnet using any availability zone in the region where you created the VPC. The CIDR range must be within the VPC CIDR range. Choose a range based on the number of IP addresses that you want to have within the subnet.

VPC to VPC connectivity

VPC to VPC connectivity is used to access data in a different VPC than the VPC that connects to the serverless runtime environment. For example, a mapping might read data from an Amazon Redshift cluster in a VPC and write data to a different Amazon Redshift cluster in another VPC.

If you process data across VPCs, configure VPC to VPC connectivity. AWS provides several ways to configure VPC to VPC connectivity, such as VPC peering or AWS Transit Gateway. Use AWS PrivateLink wherever it's applicable. For more information, refer to the AWS documentation.

NAT gateway for internet access from the private subnet

A NAT gateway allows outbound traffic to the internet from private instances. All compute instances in the serverless runtime environment that are associated with the ENI are private.

Create a NAT gateway to route outbound traffic from the private subnet to the internet. AWS provides several ways to configure subnet routing rules, such as route tables and NACL. For more information, refer to the AWS documentation.

IAM role

An IAM role defines a minimal policy that the serverless runtime environment and advanced cluster worker nodes use to create, attach, detach, and delete the ENI that's associated with the private subnet in your VPC.

The IAM role must be able to access the S3 location for supplementary files as well as the sources and targets you use in mappings. You can use the following template:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"ec2:DetachNetworkInterface",
"ec2:DeleteTags",
"ec2:DescribeTags",
"ec2:CreateTags",
"ec2:DeleteNetworkInterface",
"ec2:DescribeSecurityGroups",
"ec2:CreateNetworkInterface",
"ec2:DeleteNetworkInterfacePermission",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeAvailabilityZones",
"ec2:CreateNetworkInterfacePermission",
"ec2:AttachNetworkInterface",
"ec2:DescribeNetworkInterfacePermissions",
"ec2:DescribeSubnets",
"ec2:DescribeNetworkAcls"
"ec2:DescribeVpcs"
"ec2:DescribeRegions"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketAcl"
],
"Resource": "arn:aws:s3:::<S3 bucket name>"
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::<Supplementary file location>/*"
]
}
]
}

In the trust relationship, specify the Informatica account number as a trusted entity and create an external ID. To find the Informatica account number, create a serverless runtime environment in Administrator and check the environment properties. You can use the following template:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<Informatica account>:root"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "<External ID>"
}
}
}
]
}

VPC configuration tasks

• •Add trusted IP addresses. If your organization filters based on IP addresses, add the safe Informatica addresses so that they won't get blocked by the firewall. For more information, see Adding trusted Informatica IP addresses.
• •Configure a system disk. A system disk can help improve mapping performance. For guidelines on setting up a system disk, see Configuring a system disk. For more information on the system disk properties, see Configuring the System Disk properties.
• •Additional configuration if you want to use existing EFS or NFS directories as data disks in a serverless runtime environment. For more information, see Using EFS or NFS directories as data disks.
• •Create a data disk in your serverless runtime environment if you have files in EFS or NFS directories that you want to use in the environment. For more information, see Configuring a data disk.
• •Create a supplementary file location. If your mappings use JAR files and external libraries, set up a location on Amazon S3 to store the files. For more information, see Creating the supplementary file location.
• •Configure TLS to authenticate REST APIs. If you use a REST V3 Connector, you can configure TLS to authenticate REST APIs. For more information, see Configuring TLS to authenticate REST APIs.

Adding trusted Informatica IP addresses

US

APJ

Canada

EMEA

UK

Configuring a system disk

Rules and guidelines for the EFS file system

• •Set the file system to the ID of the EFS file system.
• •Allow the subnet in the serverless runtime environment to access to the Amazon EFS file system.
• •Configure the EFS security group to allow inbound access from the security group configured in the serverless runtime environment.
• •Configure the IAM role in the serverless environment with full access to the EFS file system. You can grant full access in the file system policy or in the IAM role. For example, the following file system policy allows root access to ServerlessRole (SREIICS) for file system fs-12345 and allows SecureTransport only:
"Version": "2012-10-17",
"Id": "efs-policy-wizard-<efs policy wizard ID>",
"Statement": [
{
"Sid": "efs-statement-<efs statement ID>",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<arn ID>:role/SREIICS"
},
"Action": [
"elasticfilesystem:ClientWrite",
"elasticfilesystem:ClientMount",
"elasticfilesystem:ClientRootAccess"
],
"Resource": "arn:aws:elasticfilesystem:us-west-2: <arn ID>:file-system/fs-12345",
"Condition": {
"Bool": {
"elasticfilesystem:AccessedViaMountTarget": "true"
}
}
},
{
"Sid": "efs-statement-<efs statement ID>",
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
"Action": "*",
"Resource": "arn:aws:elasticfilesystem:us-west-2: 123456789:file-system/fs-12345",
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
}
]
}
The following table describes the actions in the sample policy:

Action	Description
elasticfilesystem:ClientMount	Provides read-only access to a file system.
elasticfilesystem:ClientWrite	Provides write permissions on a file system.
elasticfilesystem:ClientRootAccess	Provides use of the root user when accessing a file system.

• •Create any folder required by an access point before creating the access point itself. For example, if the access point refers to the folder /my-company/dev, then define this folder first before you set up the access point.
• •Configure the IAM role to restrict access to specific access points on the file system. For more information, see https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html

Rules and guidelines for the NFS file system

• •Set the file system to the DNS of the NFS server.
• •Configure the subnet in the serverless runtime environment to allow access to the NFS file server.
• •Configure the file server security group to allow inbound access from the security group configured in the serverless runtime environment.

Using EFS or NFS directories as data disks

1Mount the EFS or NFS directories in an EC2 instance that you have.
2Log in to the EC2 instance.
3Locate a user with ID=501. If one doesn't exist, create a new user with this ID.
User ID 501 is the user cldagnt, which the serverless runtime environment uses to access mounted EFS or NFS directories.
4Assign read and write permissions to the mounted directories for user 501.
For more information, see the following topic in the AWS documentation: Working with users, groups, and permissions at the Network File System Level.

Configuring a data disk

• •Flat file support. You can use flat files from the mounted EFS or NFS locations in your mappings.
• •Parameter file support. You can use parameter files stored in the mounted EFS or NFS locations. This simplifies migrating jobs from a Secure Agent group to a serverless runtime environment, since you do not need to modify your mappings.

Creating the supplementary file location

1. 1Create the following file structure on Amazon S3:
<Supplementary file location>
├── ext
├── odbc
│ └── lib
└── serverless_agent_config
├── jars
├── SSL
├── j_depends
└── py_depends
2. 2Create a serverlessUserAgentConfig.yml file. For a template, see Populating the serverlessUserAgentConfig.yml File.
3. 3Add the serverlessUserAgentConfig.yml file directly under the serverless_agent_config directory.

Configuring TLS to authenticate REST APIs

1Navigate to the supplementary file location on Amazon S3.
2In the serverless_agent_config folder, create a subfolder called SSL.
3Add the truststore and keystore certificates to the SSL folder.
For one-way secure communication, add the truststore certificates. For two-way secure communication, add both the truststore and keystore certificates.
4 Copy the following code snippet to a text editor and add the relative path to each certificate in the supplementary file location:
version: 1
agent:
agentAutoApply:
general:
sslStore:
- fileCopy:
sourcePath: SSL/<REST V3 truststore certificate name>.jks
- fileCopy:
sourcePath: SSL/<REST V3 keystore certificate name>.jks
5In the serverless_agent_config folder, open the serverlessUserAgentConfig.yml file.
6Add the code snippet to the serverlessUserAgentConfig.yml file and save the file.
The serverless runtime environment will copy the certificates from the supplementary file location to its own reference directory so that it can use the certificates at run time.
7In the REST V3 connection properties, use the following format to specify each truststore and keystore file path in the serverless runtime environment: /home/cldagnt/SystemAgent/serverless/configurations/ssl_store/<certificate name>.jks

Configure the serverlessUserAgentConfig.yml file

• •To create the serverlessUserAgentConfig.yml file, use the template listed in Populating the serverlessUserAgentConfig.yml File and adjust accordingly.
To configure which files to copy from the supplementary file location to the serverless runtime environment, specify the file paths within the serverlessUserAgentConfig.yml file.
• •To copy files for the Elastic Server, use the code snippet provided in Copying files for the Elastic Server.
• •To copy JAR files for the JDBC V2 Connector, use the code snippet provided in Copying JDBC V2 Connector JAR files.
• •To copy JAR files for the Java transformation, use the code snippet provided in Copying Java transformation JAR files.
• •To copy resource files for the Python transformation, use the code snippet provided in Copying Python transformation resource files
• •To add files to the supplementary file location while the serverless runtime environment is running, see Adding files while the environment is running.

Populating the serverlessUserAgentConfig.yml File

Copying files for the Elastic Server

• •JDBC V2 Connector JAR files
• •JAR files for the Java transformation
• •Installation and resource files for the Python transformation

• •<Supplementary file location>/serverless_agent_config/jdbc_v2_jars/common/
• •<Supplementary file location>/serverless_agent_config/jdbc_v2_jars/spark/

Copying JDBC V2 Connector JAR files

Copying Java transformation JAR files

Copying Python transformation resource files

Adding files while the environment is running

1. 1Add the file to the appropriate location in <Supplementary file location>/serverless_agent_config/.
2. 2Specify the file in the serverlessUserAgentConfig.yml file. For information about the serverlessUserAgentConfig.yml file, see Creating the supplementary file location or the help for the appropriate connector.

• •Update an existing file.
To update an existing file while the serverless runtime environment is running, you must add the file to the supplementary file location and to the serverlessUserAgentConfig.yml file using a different name.
• •Add other file types, such as ODBC shared libraries.
• •Add a new folder or directory, such as a Python installation directory for the Python transformation.
• •Remove files from the serverless runtime environment.

Proxy servers in a serverless runtime environment

Configuring the proxy in the serverlessUserAgentConfig.yml file

Configuring the proxy in the JVM options

1On the Serverless Environments page, click the name of the serverless runtime environment.
2Click Edit.
3In the Runtime Configuration Properties section, select the Service as Data Integration Server and the Type as DTM.
4Edit any of the JVMOption fields and specify appropriate values for each parameter based on whether you use an HTTPS or HTTP proxy server.
The following table describes the parameters:

Parameter	Description
-Dhttp.proxySet=	Determines if the serverless runtime environment must use the proxy settings when the outgoing proxy server is HTTP. Select -Dhttp.proxySet=True to use the proxy.
-Dhttps.proxySet=	Determines if the serverless runtime environment must use the proxy settings when the outgoing proxy server is HTTPS. Select -Dhttps.proxySet=True to use the proxy.
-Dhttp.proxyHost=	Host name of the outgoing HTTP proxy server.
-Dhttp.proxyPort=	Port number of the outgoing HTTP proxy server.
-Dhttp.proxyUser=	Authenticated user name for the HTTP proxy server.
-Dhttp.proxyPassword=	Password for the authenticated user.
-Dhttps.proxyHost=	Host name of the outgoing HTTPS proxy server.
-Dhttps.proxyPort=	Port number of the outgoing HTTPS proxy server.
-Dhttps.proxyUser=	Authenticated user name for the HTTPS proxy server.
-Dhttps.proxyPassword=	Password for the authenticated user.

5Click Save.

Allowing domains in the proxy server

Serverless runtime environment in Administrator

1. 1Option to create a serverless runtime environment
2. 2Refresh icon
3. 3Actions menu

Configuring the Basic Configuration properties

Configuring the Platform Configuration properties

Configuring the Runtime Configuration properties

Setting variables for default directories

Reducing the number of simultaneous tasks

Configuring the System Disk properties

Configuring the Data Disk properties

Serverless runtime validation

• •ec2:DescribeRegions
• •ec2:DescribeAvailabilityZones
• •ec2:DescribeVpcs
• •ec2:DescribeSubnets
• •ec2:DescribeSecurityGroups

• •Create a new serverless runtime environment.
• •Edit a failed serverless runtime environment and save the updates.
• •Clone a serverless runtime environment and save the configurations.
• •Redeploy a failed serverless runtime environment.

Serverless runtime environment management

Editing the serverless runtime environment

• •Up and Running. You can only update the following fields: Max Compute Units Per Task and Task Timeout. The updated values take effect for subsequent task runs.
• •Failed. You can update all the properties. The updated properties take effect once you use the Redeploy action.

Redeploying the serverless runtime environment

• •You change your organization's licenses.
• •The serverless runtime environment shuts down because the organization ran out of serverless compute units. You can add more compute units to your organization and redeploy the serverless runtime environment.
• •You update the configuration in your cloud environment. For example, you update files in the supplementary file location, or you update the policy that is attached to the IAM role.

Cloning the serverless runtime environment

Deleting the serverless runtime environment

• •Use Monitor to make sure that the environment is not running any jobs.
• •Use the Show Dependencies action to see if the environment is being used by any tasks, mappings, or connections. If dependencies exist, remove them before deleting the environment.

Using metering serverless compute units

Configuring disaster recovery

Disaster recovery procedure

1. 1Create a temporary serverless runtime environment in a stable region or availability zone.
2. 2Make sure that the connections used in jobs are available in the stable region or availability zone.
3. 3Clean up data related to incomplete job runs. If data was partially loaded to a target, manually delete the data or update the mapping to truncate the target before writing new rows.
4. 4Redirect jobs to the temporary environment.

Restoring the primary environment

1. 1Clean up the ENIs that were created in your AWS account for the primary environment.
2. 2Redeploy the primary environment.
3. 3Redirect jobs to the primary environment.
4. 4Delete the temporary environment.