Create a Managed ClickHouse® cluster

In this tutorial, you learn how to get started with Managed ClickHouse® in DoubleCloud by creating a new cluster with a method of your choice.

  1. Go to the Clusters overview page in the console.

  2. Click Create cluster in the upper-right corner of the page.

  3. Select ClickHouse®.

  4. Choose a provider and a region.

  5. Under Resources:

    1. Select the s1-c2-m4 preset for CPU, RAM capacity, and storage space to create a cluster with minimal configuration.

      Understand your ClickHouse® resource preset

      A resource preset has the following structure:

      <CPU platform> - C<number of CPU cores> - M<number of gigabytes of RAM>

      There are three available CPU platforms:

      • g - ARM Graviton

      • i - Intel (x86)

      • s - AMD (x86)

      For example, the i1-c2-m8 preset means that it's an Intel platform 2-core CPU with 8 gigabytes of RAM.

      You can see the availability of CPU platforms across our Managed Service for ClickHouse® areas and regions.

    2. Choose the number of replicas. Let's keep it as is with a single replica.

    3. Select the number of shards. Keep a single shard.

      Understand shards and replicas

      Shards refer to the servers that contain different parts of the data (to read all the data, you must access all the shards). Replicas are duplicating servers (to read all the data, you can access the data on any of the replicas).

  6. Under Basic settings:

    1. Enter the cluster Name, in this scenario - tutorial-cluster.

    2. From the Version drop-down list, select the ClickHouse® version the Managed ClickHouse® cluster will use. For most clusters, we recommend using the latest version.

  7. Under Advanced:

    1. Under Maintenance settings, select the scheduling type:

      • Arbitrary to delegate maintenance window selection to DoubleCloud. Usually, your cluster will perform maintenance procedure at the earliest available time slot.

        Warning

        We suggest not to use this scheduling type with single-host clusters, as it can lead to your cluster becoming unavailable at random.

      • By schedule to set the weekday and time (UTC) when DoubleCloud may perform maintenance on your cluster.

    2. Under Autoscaling, select whether you want the cluster resources to automatically increase and specify the maximum limits they can increase to.

      If autoscaling is enabled, DoubleCloud regularly checks the resource utilization and incrementally adds additional resources when the utilization is nearing capaticy.

    3. Under Networking → VPC, specify in which DoubleCloud VPC to locate your cluster. Use the default value in the previously selected region if you don't need to create this cluster in a specific network.

    4. Select the allocation for the ClickHouse Keeper service - embedded or dedicated.

      We recommend using dedicated hosts for high-load production clusters. Dedicated ClickHouse Keeper hosts ensure that your production cluster's performance remains unaffected under heavy loads - they don't use its CPU or memory.

      ClickHouse Keeper host location is irreversible

      After creating the cluster, you won't be able to change the ClickHouse Keeper deployment type.

      For dedicated ClickHouse Keeper hosts, select the appropriate resource preset. Please note that this resource preset will apply to all three hosts and will be billed accordingly.

    5. Specify or adjust your cluster's DBMS settings. For more information, see the Settings reference.

  8. Click Submit.

Your cluster will appear with the Creating status on the Clusters page in the console. Setting everything up may take some time. When the cluster is ready, it changes its state to Alive.

Click on the cluster to open its information page:

cluster-created

Tip

The DoubleCloud service creates the superuser admin and its password automatically. You can find both the User and the Password in the Overview tab on the cluster information page.

To create users for other roles, see Manage ClickHouse® users

You can create a Managed ClickHouse® cluster using the DoubleCloud Terraform provider .

Tip

If you haven't used Terraform before, refer to Create DoubleCloud resources with Terraform for more detailed instructions.

Example provider and resource configuration:

# main.tf

terraform {
  required_providers {
    doublecloud = {
      source    = "registry.terraform.io/doublecloud/doublecloud"
    }
  }
}

provider "doublecloud" {
  authorized_key = file("authorized_key.json")
}

data "doublecloud_network" "default" {
  name       = NETWORK_NAME              # Replace with the name of the network you want to use
  project_id = DOUBLECLOUD_PROJECT_ID    # Replace with your project ID
}

resource "doublecloud_clickhouse_cluster" "example-clickhouse" {
  project_id = DOUBLECLOUD_PROJECT_ID    # Replace with your project ID
  name       = "example-clickhouse"
  region_id  = "eu-central-1"
  cloud_type = "aws"
  network_id = data.doublecloud_network.default.id

  resources {
    clickhouse {
      resource_preset_id = "s1-c2-m4"
      disk_size          = 34359738368
      replica_count      = 1
    }
  }
    
  config {
    log_level       = "LOG_LEVEL_TRACE"
    max_connections = 120
  }

  access {
    data_services    = ["transfer"]
    ipv4_cidr_blocks = [
      {
        value       = "10.0.0.0/24"
        description = "Office in Berlin"
      }
    ]
  }
}

To learn how to get the authorized_key.json file, refer to Create an API key. You can find the DoubleCloud project ID on the project settings page.

Tip

This example contains a minimum set of parameters required to create a functional example cluster. When you create your production cluster, make sure to use the configuration that is suitable for your needs. For a full list of available parameters, refer to the DoubleCloud ClickHouse cluster resource schema .

To create a ClickHouse® cluster, use the ClusterService create method. The required parameters to create a functional cluster:

  • project_id - the ID of your project. You can get this value on your project's information page.

  • cloud_type - aws or gcp.

  • region_id - for the list of available regions and their region codes, see Areas and regions for Managed Service for ClickHouse®.

  • name - your cluster's name. It must be unique within the project.

  • resources - specify the following from the doublecloud.clickhouse.v1.ClusterResources model:

    • resource_preset_id - specify the name of the hardware resource preset for your cluster. For the list of available presets for ClickHouse® clusters, see DoubleCloud hardware instances.

    • disk_size - the storage size for your cluster in bytes. We recommend allocating no less than 34359738368 bytes (32 GB).

    • replica_count - specify the number of replicas between 1 and 3.

    • shard_count - specify the number of shards.

github-mark-white

View this example on GitHub 

import json
import logging

from google.protobuf.wrappers_pb2 import Int64Value

import doublecloud

from doublecloud.clickhouse.v1.cluster_pb2 import ClusterResources
from doublecloud.clickhouse.v1.cluster_service_pb2 import CreateClusterRequest
from doublecloud.clickhouse.v1.cluster_service_pb2_grpc import ClusterServiceStub

def create_cluster(sdk, project_id, cloud_type, region_id, name, network_id):
    cluster_service = sdk.client(ClusterServiceStub)
    operation = cluster_service.Create(
        CreateClusterRequest(
            project_id=project_id,
            cloud_type=cloud_type,
            region_id=region_id,
            name=name,
            resources=ClusterResources(
                clickhouse=ClusterResources.Clickhouse(
                    resource_preset_id="s2-c2-m4",
                    disk_size=Int64Value(value=<storage_size_in_bytes>),
                    replica_count=Int64Value(value=<number_of_replicas>),
                )
            ),
            network_id=network_id,
        )
    )
    logging.info("Creating initiated")
    return operation

package main

import (
   "context"
   "flag"
   "fmt"
   "log"

   "github.com/doublecloud/go-genproto/doublecloud/clickhouse/v1"
   dc "github.com/doublecloud/go-sdk"
   "google.golang.org/protobuf/types/known/wrapperspb"

   "github.com/doublecloud/go-sdk/iamkey"
   "github.com/doublecloud/go-sdk/operation"
)

func createCluster(ctx context.Context, dc *dc.SDK, flags *cmdFlags) (*operation.Operation, error) {
   x, err := dc.ClickHouse().Cluster().Create(ctx, &clickhouse.CreateClusterRequest{
      ProjectId: *flags.projectID,
      CloudType: "aws",
      RegionId:  *flags.region,
      Name:      *flags.name,
      Resources: &clickhouse.ClusterResources{
         Clickhouse: &clickhouse.ClusterResources_Clickhouse{
            ResourcePresetId: "s2-c2-m4",
            DiskSize:         wrapperspb.Int64(32 * 2 << 30),
            ReplicaCount:     wrapperspb.Int64(1),
         },
      },
      NetworkId: *flags.networkID,
   })
   if err != nil {
      return nil, err
   }
   log.Println("Creating clickhouse cluster ...")
   log.Println("https://app.double.cloud/clickhouse/" + x.ResourceId + "/operations")
   op, err := dc.WrapOperation(x, err)
   if err != nil {
      panic(err)
   }
   err = op.Wait(ctx)
   return op, err
}

For more in-depth examples, check out DoubleCloud API Go SDK repository .

See also

Previous
All instructions
Next
Manage a cluster