Runtime Environments > Elastic runtime environments > Set up the AWS environment
  

Set up the AWS environment

Set up your AWS environment to create and configure the required AWS resources before deploying an elastic runtime environment.
To set up the AWS environment, complete the following tasks:
  1. 1Create and configure AWS resources that the elastic runtime environment can use to start a Secure Agent and to run jobs.
  2. 2Create a cluster installer policy that contains the permissions that allow the cluster to interact with an elastic runtime environment.
  3. 3Create an IAM role for the cluster installer that allows the cluster installer to deploy an elastic runtime environment.
  4. 4Create a worker policy that contains the permissions that allow worker nodes to process data in an elastic runtime environment.
  5. 5Create an IAM role for the worker nodes that allows the worker nodes to process data in the elastic runtime environment.
  6. 6Launch an EC2 instance for the jump host.
  7. The jump host is an EC2 instance in the public subnet that you use to SSH into nodes in the elastic runtime environment in the private subnet. The elastic runtime environment must run in a private subnet and remote access requires a jump host.
  8. 7Launch an EC2 instance for the master node.

Step 1. Create AWS resources

Create and configure AWS resources that the elastic runtime environment can use to start a Secure Agent and run jobs.
Create and configure the following AWS resources:
VPC
A VPC contains all your AWS resources, including the Kubernetes cluster that hosts the Secure Agent.
Create a VPC in your AWS account. Use IPv4 CIDR manual input and enter the CIDR block for the VPC to use.
Public subnet
A public subnet provides internet access through a NAT gateway. To create the public subnet, use the following guidelines:
Private subnet
A private subnet hosts IDMC servers and resources. To create the private subnet, use the following guidelines:
NAT gateway
A NAT gateway allows outbound traffic to the internet from nodes in the private subnet. The NAT gateway ensures that private nodes are isolated from the public internet.
To create the NAT gateway, use the following guidelines:
Internet gateway
An internet gateway is used for internet access. The public subnet and the internet gateway allow the jump host to receive SSH connections from the public internet.
Create an internet gateway in AWS and then attach it to the VPC.
For information about the jump host, see Step 6. Create the jump host.
Public route table
A public route table routes traffic in the public subnet. To create the public route table, use the following guidelines:
Private route table
A private route table routes traffic in the private subnet. To create the private route table, use the following guidelines:
Security group for the elastic runtime environment
A security group allows SSH access to the runtime environment. To create the security group, use the following guidelines:
You need to add the name of the security group to the config.txt file that you will configure when you deploy the elastic runtime environment. If you don’t create a security group for the elastic runtime environment as part of this step, the cluster installer will create one and populate the inbound rules. For more information, see Deploy an elastic runtime environment.
Security group for the jump host
A security group allows SSH access to the jump host from your local machine.
To create the security group, add an inbound rule that allows SSH traffic on port 22 from source <local machine IP address>/32.
For information about creating the jump host, see Step 6. Create the jump host.
EFS file systems for system storage (required) and data storage (optional)
An elastic runtime environment uses EFS file systems for system storage and data storage. System storage is required for Secure Agent operations, and data storage is used to store flat files that you used as data sources in tasks, such as mapping tasks.
Create an EFS file system for system storage and then create an access point for it. Optionally, you can create another EFS file system for data storage and create an access point for it as well.
To create each file system, use the following guidelines:
To create each access point, use the following guidelines:
The following image shows an example of an EFS file system in the AWS Management Console:
An EFS file system in the AWS Management Console has an access point configured with a POSIX user UID of 1200, a GID of 1200, and permissions set to 0755.

Step 2. Create a cluster installer policy

Create a cluster installer policy that contains the permissions that allow the cluster to interact with an elastic runtime environment.
To create the cluster installer policy, download the Permissions_cluster_installer_policy.json file from the Elastic runtime environment IAM policy permissions JSON files Knowledge article. Replace the placeholders and use the content to create a policy in AWS.
The following table describes each placeholder:
Placeholder
Description
{{account-id}}
AWS account ID.
{{VPC_ID}}
VPC ID.
{{cluster-installer-role}}
Cluster installer role name, such as cluster_installer_role. The name must follow the AWS naming convention for roles.
Note: You can decide on the role name in this step, and then use the same role name when you create the role. For information about creating the cluster installer role, see Step 3. Create a cluster installer role.
{{worker-node-role}}
Worker role name, such as worker_node_role. The name must follow the AWS naming convention for roles.
Note: You can decide on the role name in this step, and then use the same role name when you create the role. For information about creating the cluster installer role, see Step 5. Create a worker role.
{{region}}
AWS region, such as us-west-2.
For more information about the permissions in the cluster installer policy, see Cluster installer policy statements.

Step 3. Create a cluster installer role

Create an IAM role for the cluster installer that allows the cluster installer to deploy an elastic runtime environment.
To create the cluster installer role, use the following guidelines:

Step 4. Create a worker policy

Create a worker policy that contains the permissions that allow worker nodes to process data in an elastic runtime environment.
To create the worker policy, download the file Permissions_worker_node_policy.json from the Elastic runtime environment IAM policy permissions JSON files Knowledge article. Use the content to create a policy in AWS.
For more information about the permissions in the worker policy, see Worker policy statements.

Step 5. Create a worker role

Create an IAM role for the worker nodes that allows the worker nodes to process data in the elastic runtime environment.
To create the worker role, use the same worker role name that you specified in the cluster installer policy. If AWS doesn't automatically create an instance profile for the role, create the instance profile manually

Step 6. Create the jump host

Launch an EC2 instance for the jump host.
The jump host is an EC2 instance in the public subnet that you use to SSH into nodes in the elastic runtime environment in the private subnet. Because the elastic runtime environment is designed to run in a private subnet, a jump host is required to remotely access the environment.
Using a jump host is a best practice for secure and scalable infrastructure so that the elastic runtime environment isn't exposed to the internet. The jump host greatly reduces the attack surface and prevents unauthorized access to the elastic runtime environment, and access is controlled using SSH authentication and security group rules. If the jump host is compromised, nodes in the elastic runtime environment are protected by a firewall and security groups.
Create a jump host using the launch instance wizard in AWS. Use the following guidelines:
Then, launch the instance.

Step 7. Create the master node

Launch an EC2 instance for the master node.
Create the master node using the launch instance wizard in AWS. Use the following guidelines:
Then, launch the instance.