Use a CLAIRE-powered configuration to create an advanced cluster that stays within budget. CLAIRE, Informatica's AI engine, selects cluster infrastructure, reports on the estimated cloud infrastructure savings, visualizes the infrastructure costs over time, and generates insights and recommendations. It provides transparency into the costs that an advanced cluster incurs and enables FinOps capabilities in your organization.
You can use a CLAIRE-powered configuration to create an advanced cluster in an AWS environment if CLAIRE recommendations are enabled in your organization.
When you use a CLAIRE-powered configuration, you can optimize the cluster for cost or for performance. Then, you specify the target average cost per hour and maximum cost per hour, and CLAIRE configures the cluster to stay within your budget.
For a cluster that's optimized for cost, infrastructure costs are less likely to exceed the average cost per hour, but medium- and low-priority jobs might take longer to run. For a cluster that's optimized for performance, infrastructure costs are more likely to exceed the average cost per hour, especially for large workloads. However, jobs typically run faster and can meet lower target durations.
CLAIRE can create an advanced cluster as long as the maximum cost per hour is greater than $1.00 USD. To adjust the target average and maximum costs per hour, use Monitor to view the infrastructure costs that the cluster incurs. If the actual infrastructure costs are close to the maximum cost per hour or you want to decrease the time to run workloads, you can increase the maximum cost. If you set a maximum cost per hour that's much higher than the actual infrastructure costs, CLAIRE will use only the cloud resources that the cluster needs.
The infrastructure costs that CLAIRE manages include compute instance, storage, and elastic load balancer costs. To keep infrastructure costs within budget, CLAIRE performs the following tasks:
•Selects Spot Instances over On-Demand Instances where appropriate. CLAIRE selects Spot Instances only in clusters that are optimized for cost.
•Selects instance types based on the cluster optimization preference and the resources that the cluster uses to run a typical workload.
•Scales the cluster and local storage based on the expected workload.
•Schedules jobs to efficiently use cluster resources.
•Shuts down the cluster when the cluster is expected to be idle.
CLAIRE doesn't manage data transfer costs, disk operation costs, and IPU costs. These costs depend on the workload that you run on the cluster. CLAIRE generates recommendations to reduce workload-dependent costs and improve cluster performance in the Recommendations panel.
Cluster budget estimates
CLAIRE uses the target average cost per hour to determine how to create a cluster and control cluster infrastructure costs. It uses the maximum cost per hour to optimize the maximum reachable number of worker nodes based on the workload that the cluster runs.
When CLAIRE isn't familiar with the workload that the cluster runs, it uses a default minimum and maximum number of worker nodes to start the cluster for the first time. As CLAIRE collects and processes cluster metadata to learn about the typical cluster workload, it updates the worker nodes up to a maximum reachable number of worker nodes based on the cluster budget. If your workload is smaller than expected, the cluster might have fewer worker nodes than it started with.
You can use the following tables to estimate how many worker nodes your cluster might have based on your budget. Currency values are in USD.
Optimized for cost
The following table lists the minimum and maximum number of worker nodes on a cluster that's optimized for cost based on a given budget:
Target average cost per hour
Maximum cost per hour
Default number of worker nodes
Maximum reachable number of worker nodes
$1
$2
Min: 1
Max: 3
Min: 1
Max: 5
$1
$5
Min: 1
Max: 5
Min: 1
Max: 13
$4
$10
Min: 1
Max: 14
Min: 1
Max: 28
$4
$15
Min: 1
Max: 17
Min: 1
Max: 42
If Spot Instances are enabled, the cluster always uses a minimum of one worker node so that CLAIRE can add Spot Instances to the cluster as needed. If you disable Spot Instances, the minimum number of worker nodes might change.
Optimized for performance
The following table lists the minimum and maximum number of worker nodes on a cluster that's optimized for performance based on a given budget:
Target average cost per hour
Maximum cost per hour
Default number of worker nodes
Maximum reachable number of worker nodes
$1
$5
Min: 1
Max: 7
Min: 1
Max: 9
$5
$10
Min: 5
Max: 15
Min: 9
Max: 18
$5
$15
Min: 5
Max: 22
Min: 9
Max: 28
$10
$20
Min: 9
Max: 31
Min: 18
Max: 38
CLAIRE recommendations
View CLAIRE recommendations to improve cluster performance and reduce infrastructure costs in the Recommendations panel. The Recommendations panel is available in CLAIRE-powered configurations.
The following image shows the Recommendations panel:
1. Actions menu
Use the Actions menu to mark recommendations as complete or incomplete, or to opt in and opt out of recommendations.
CLAIRE automatically applies some recommendations, such as using Spot Instances. You can use the Actions menu to opt out of the recommendation or opt back in.
Other recommendations require manual action, such as changing the cluster region. These recommendations appear as to-do items. You can use the Actions menu to mark to-do items as complete, or mark them as incomplete. You can also opt out of the recommendation or opt back in.
2. Filter recommendations
Use the Filter menu to filter recommendations. You can use the following filters:
- All shows all recommendations.
- To Do shows recommendations that require manual action.
- Applied shows recommendations that are applied automatically and recommendations that are marked complete.
- Opted-Out shows recommendations that are opted out of.
3. Refresh recommendations
Refresh the recommendations to update the recommendations in the Recommendations panel, or use Monitor to check if any new recommendations are available for the advanced clusters in your organization.
Note: CLAIRE clears the recommendations in the Recommendations panel if you change the runtime environment that's associated with the CLAIRE-powered configuration.
