ClickHouse Keeper hosts

ClickHouse Keeper is a coordination system analogous to Zookeeper with a compatible interface. It's a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.

Resource allocation

ClickHouse Keeper operates on the basis of majority quorums forming an ensemble. This approach requires setting up hardware infrastructure with the following requirements:

• Number of hosts:

  ClickHouse Keeper requires a minimum of three servers to form a majority quorum and ensure fault tolerance. It's generally recommended to use an odd number of servers.

• CPU and Memory:

  ClickHouse Keeper resource requirements are relatively modest. However, the CPU and memory capacity should be sufficient to handle the expected workload and concurrent client requests. The specific CPU and memory requirements depend on factors such as the number of clients, the complexity of the operations, and the data size being managed.

• Storage:

  ClickHouse Keeper primary data storage is in-memory, which makes it optimal for read-heavy workloads. However, it also writes transaction logs to disk for durability. The disk storage capacity should be able to accommodate the transaction logs and any additional data, such as snapshots, if enabled. The required disk I/O performance depends on the write load and the rate of changes in the data.

DoubleCloud offers two ClickHouse Keeper deployment types:

High-availability configuration

High availability can be achieved for clusters with at least 3 ClickHouse Keeper hosts.

The following table describes the minimal configurations to achieve high availability when using dedicated and embedded ClickHouse Keeper hosts:

This is how dedicated ClickHouse Keeper hosts contribute to high availability of your Managed Service for ClickHouse® cluster:

• Leader election

  When a ClickHouse Keeper loses its leader (the replica responsible for handling write requests), it elects a new one among the available replicas. This process ensures that data can still be written to the cluster, even in the event of host failures.

• Data Consistency

  ClickHouse Keeper helps maintain consistency across replicas. All replicas store metadata of data parts there and use it for data replication and consistent merges.

The embedded ClickHouse Keeper solution requires at least 3 hosts running at any time, so you'll need to add one extra shard for enhanced fault-tolerance, and, since the shard configurations must be mirrored, you end up with a more expensive setup.