Autoscaling ClickHouse® cluster resources

Autoscaling allows your Managed ClickHouse® cluster resources to automatically adjust depending on the cluster usage. You can configure the cluster to automatically increase the storage when it nears capacity and adjust the cluster resources to match the workload.

Autoscaling also helps you control costs because your cluster only uses the resources it needs. When creating a cluster, you don't need to allocate storage for future potential database growth from the very start. Instead, storage capacity incrementally grows together with the database size, so you only pay for what you use. Similarly, you don't need to allocate resources for potential load spikes. You can specify the amount of CPU and RAM for your current needs and then have the cluster automatically scale up and down to handle varying workloads.

How autoscaling works

Storage autoscaling

When storage autoscaling is enabled, DoubleCloud monitors how much storage is available in the cluster. When disk utilization on any host reaches 90%, it adds storage on all hosts towards the next capacity available in a given resource preset. DoubleCloud sends notification emails before storage increases.

The interval between storage increases is at least six hours. All storage increases are permanent, and the cost depends on the actual storage size.

Instance autoscaling

When instance autoscaling is enabled, DoubleCloud monitors the CPU utilization in the cluster. If the average CPU utilization on all hosts in the cluster exceeds 60% for 5 minutes, the cluster scales up towards the next available resource preset. The cluster can incrementally scale up until it reaches the maximum preset specified in the cluster configuration.

When the period of increased CPU utilization is over, the cluster returns to the base resource preset. For the cluster to scale down, the CPU usage should be under 50% of the next lower resource preset for an hour.

The minimal interval between instance autoscaling events is 15 minutes, but the actual time may be different depending on the load.

Configure autoscaling

You can configure autoscaling when creating a ClickHouse® cluster or enable it later in the cluster settings. Autoscaling is available for SSD storage and instances for both ClickHouse® and Keeper hosts.

resource "doublecloud_clickhouse_cluster" "example-clickhouse" {
  ...
  resources {
    clickhouse {
      min_resource_preset_id = "s2-c2-m8"
      max_resource_preset_id = "s2-c16-m64"
      disk_size          = 34359738368     # 32 GB in bytes
      max_disk_size      = 103079215104    # 96 GB in bytes
      ...
    }
    dedicated_keeper {
      resource_preset_id = "s2-c2-m4"
      disk_size          = 34359738368     # 32 GB in bytes
      max_disk_size      = 51539607552     # 48 GB in bytes
      ...
    }
  }
  ...
}