Create a Managed ClickHouse® cluster

This tutorial guides you through creating a ClickHouse® cluster in DoubleCloud using a method of your choice. Learn how to create and configure a cluster in the DoubleCloud console or using the API, SDK, or Terraform.

Console

Terraform

API

Python

Step 1. Configure resources

Go to the Clusters page in the console and click Create cluster at the top right.
Select ClickHouse.
Choose a provider and region. You can create Managed ClickHouse® clusters on AWS or Google Cloud in any of the available regions. By default, DoubleCloud preselects the region nearest to you.
In the Resources section, select between the x86 or ARM machine type and then select a resource preset that’s suitable for your expected workload.

For testing or development, you can select any preset. In production, ClickHouse® developers recommend having at least 16 GB of RAM.
ClickHouse® resource preset reference
A resource preset has the following structure:
```
<cpu-platform>-c<number-of-cpu-cores>-m<gigabytes-of-ram>
```
There are three available CPU platforms:
- g: ARM Graviton
- i: Intel (x86)
- s: AMD (x86)
For example, the i2-c2-m8 preset corresponds to an Intel-family machine with 2 virtual cores and 8 GB of RAM.

Learn more about the availability of CPU platforms in areas and regions
Select the number of shards, number of replicas, and amount of RAM for your workload. For a high-availability configuration, make sure to select at least three replicas.

About shards and replicas
Replicas are ClickHouse® hosts that hold a copy of the data in the cluster or shard. Storing the same data in several replicas enables higher data availability and redundancy. Learn more

Shards refer to subsets of data. Sharding data across multiple servers allows you to distribute and divide the load so that you don’t exceed the capacity of a single server. Learn more
Under Basic settings, enter a cluster name, such as clickhouse-dev.
In Version, select the ClickHouse® version for the cluster. Unless you need a specific version, select the latest one.

Step 2. Configure advanced settings

Under Maintenance settings, select between the arbitrary and scheduled maintenance:

About maintenance settings
If you select Arbitrary, DoubleCloud selects the maintenance window automatically. Usually, maintenance takes place at the earliest available time slot.

Warning

If your cluster has only one host, arbitrary maintenance can make it unavailable at a random time.

To perform maintenance on a specific date and time, select By schedule and specify the day and time (UTC) when you want the cluster maintenance to be performed.
In Autoscaling, select whether you want the cluster resources to automatically scale and specify the maximum limits they can increase to.

If autoscaling is enabled, DoubleCloud regularly checks the resource utilization and automatically adjusts them depending on the cluster usage. Learn more
In ClickHouse Keeper hosts, select the deployment mode.

About ClickHouse Keeper hosts
In the embedded mode, ClickHouse Keeper shares hosts with ClickHouse®. In the dedicated mode, ClickHouse Keeper uses three separate hosts, which improves performance, availability, and fault tolerance. Learn more

For high-load production clusters, make sure to use dedicated ClickHouse Keeper hosts for better performance — when running on separate hosts, ClickHouse® and ClickHouse Keeper don’t compete for CPU and RAM.

Warning

You can’t change the ClickHouse Keeper deployment mode after creating the cluster.

For dedicated ClickHouse Keeper hosts, select the appropriate resource preset. It applies to all three hosts and is billed accordingly.
Under Networking → VPC, select the network where you want to create the cluster.

If you don’t need to place the cluster in a specific network, leave the preselected default option.
Specify or adjust your cluster’s DBMS settings. For more information, refer to Managed ClickHouse® settings reference.
In the Summary block on the right, review the resources to be created and their price.
Click Submit.

Creating a cluster usually takes five to seven minutes depending on the cloud provider and region. When the cluster is ready, its status changes from Creating to Alive.

To view the cluster details, select it from the cluster list:

Screenshot of the ClickHouse® cluster page

You can create a Managed ClickHouse® cluster using the DoubleCloud Terraform provider .

Tip

If you haven't used Terraform before, refer to Create DoubleCloud resources with Terraform for more detailed instructions.

Example provider and resource configuration:

# main.tf

terraform {
  required_providers {
    doublecloud = {
      source    = "registry.terraform.io/doublecloud/doublecloud"
    }
  }
}

provider "doublecloud" {
  authorized_key = file("authorized_key.json")
}

data "doublecloud_network" "default" {
  name       = NETWORK_NAME              # Replace with the name of the network you want to use
  project_id = DOUBLECLOUD_PROJECT_ID    # Replace with your project ID
}

resource "doublecloud_clickhouse_cluster" "example-clickhouse" {
  project_id = DOUBLECLOUD_PROJECT_ID    # Replace with your project ID
  name       = "example-clickhouse"
  region_id  = "eu-central-1"
  cloud_type = "aws"
  network_id = data.doublecloud_network.default.id

  resources {
    clickhouse {
      resource_preset_id = "s2-c2-m4"
      disk_size          = 34359738368
      replica_count      = 1
    }
  }
    
  config {
    log_level       = "LOG_LEVEL_TRACE"
    max_connections = 120
  }

  access {
    data_services    = ["transfer"]
    ipv4_cidr_blocks = [
      {
        value       = "10.0.0.0/24"
        description = "Office in Berlin"
      }
    ]
  }
}

To learn how to get the authorized_key.json file, refer to Create an API key. You can find the DoubleCloud project ID on the project settings page.

Tip

This example contains a minimum set of parameters required to create a functional example cluster. When you create your production cluster, make sure to use the configuration that is suitable for your needs. For a full list of available parameters, refer to the DoubleCloud ClickHouse cluster resource schema .

To create a ClickHouse® cluster, use the ClusterService create method. The required parameters to create a functional cluster:

project_id: ID of your project. You can get the ID on your project’s information page.
cloud_type: aws or gcp.
region_id: For the list of available regions and their region codes, refer to Areas and regions for Managed Service for ClickHouse®.
name: Name of your cluster. It must be unique within the project.
resources: Specify the following settings from the doublecloud.clickhouse.v1.ClusterResources model:
- resource_preset_id: Name of the resource preset for your cluster. For the list of available presets for ClickHouse® clusters, refer to DoubleCloud hardware instances.
- disk_size: Storage size for your cluster in bytes.
- replica_count: Number of replicas between 1 and 3.
- shard_count: Number of shards.

View this example on GitHub

import json
import logging

from google.protobuf.wrappers_pb2 import Int64Value

import doublecloud

from doublecloud.clickhouse.v1.cluster_pb2 import ClusterResources
from doublecloud.clickhouse.v1.cluster_service_pb2 import CreateClusterRequest
from doublecloud.clickhouse.v1.cluster_service_pb2_grpc import ClusterServiceStub

def create_cluster(sdk, project_id, cloud_type, region_id, name, network_id):
    cluster_service = sdk.client(ClusterServiceStub)
    operation = cluster_service.Create(
        CreateClusterRequest(
            project_id=project_id,
            cloud_type=cloud_type,
            region_id=region_id,
            name=name,
            resources=ClusterResources(
                clickhouse=ClusterResources.Clickhouse(
                    resource_preset_id="s2-c2-m4",
                    disk_size=Int64Value(value=<storage_size_in_bytes>),
                    replica_count=Int64Value(value=<number_of_replicas>),
                )
            ),
            network_id=network_id,
        )
    )
    logging.info("Creating initiated")
    return operation

package main

import (
   "context"
   "flag"
   "fmt"
   "log"

   "github.com/doublecloud/go-genproto/doublecloud/clickhouse/v1"
   dc "github.com/doublecloud/go-sdk"
   "google.golang.org/protobuf/types/known/wrapperspb"

   "github.com/doublecloud/go-sdk/iamkey"
   "github.com/doublecloud/go-sdk/operation"
)

func createCluster(ctx context.Context, dc *dc.SDK, flags *cmdFlags) (*operation.Operation, error) {
   x, err := dc.ClickHouse().Cluster().Create(ctx, &clickhouse.CreateClusterRequest{
      ProjectId: *flags.projectID,
      CloudType: "aws",
      RegionId:  *flags.region,
      Name:      *flags.name,
      Resources: &clickhouse.ClusterResources{
         Clickhouse: &clickhouse.ClusterResources_Clickhouse{
            ResourcePresetId: "s2-c2-m4",
            DiskSize:         wrapperspb.Int64(32 * 2 << 30),
            ReplicaCount:     wrapperspb.Int64(1),
         },
      },
      NetworkId: *flags.networkID,
   })
   if err != nil {
      return nil, err
   }
   log.Println("Creating clickhouse cluster ...")
   log.Println("https://app.double.cloud/clickhouse/" + x.ResourceId + "/operations")
   op, err := dc.WrapOperation(x, err)
   if err != nil {
      panic(err)
   }
   err = op.Wait(ctx)
   return op, err
}

For more in-depth examples, check out the DoubleCloud API Go SDK repository .

What’s next

Connect to the cluster using WebSQL
Replicate your data to the cluster with Transfer
Create and manage ClickHouse® users and roles
Manage the cluster allowlist

All instructions

Manage a cluster