Connect Prometheus to your Managed ClickHouse® cluster

DoubleCloud Managed Service for ClickHouse® allows you to seamlessly integrate Prometheus to monitor ClickHouse® metrics and enable alerting. You'll only need to access the 9363 port of your cluster and to specify the /metrics endpoint in your Prometheus configuration file. This feature is enabled for all Managed ClickHouse® clusters by default.

This scenario illustrates how to scrape several metrics with a local Prometheus instance.

Create a Managed ClickHouse® cluster

  1. Go to the Clusters overview page.

  2. Click Create cluster in the upper-right corner of the page.

    1. Select ClickHouse®.

    2. Choose a provider and a region closest to your geographical location.

    3. Under Resources:

      • Select a preset for CPU, RAM capacity, and storage space. The first s1-c2-m8 preset will be more than enough for our tutorial.

      • Select 1 replica and 1 shard.

    4. Under Basic settings:

      • Enter the cluster Name: clickhouse-for-prometheus.

      • Keep the Version as is - this is the latest stable version of ClickHouse®.

    5. Under NetworkingVPC, specify in which DoubleCloud VPC to locate your cluster. Use the default value in the previously selected region if you don't need to create this cluster in a specific network.

    6. Click Submit.

Your cluster will appear with the Creating status on the Clusters page in the console. Setting everything up may take some time. When the cluster is ready, it changes its state to Alive.

Install and configure Prometheus

  1. Open your Linux terminal and install the software:

    1. (Optional) Start Docker if needed:

      service docker start
      
    2. Install Prometheus from Docker :

      docker pull prom/prometheus
      
    1. Run the following command to install Prometheus:

      sudo apt install prometheus
      
  2. Create a configuration file with the following structure. A common location for this file is the /etc/prometheus/ folder. Your configuration file should contain your cluster's FQDN as a target. You can find the FQDN on your cluster's information page:

    prometheus.yml

    global:
      scrape_interval: 30s
      external_labels:
        monitor: 'codelab-monitor'
    scrape_configs:
      - job_name: 'clickhouse'
        scrape_interval: 1m
        scrape_timeout: 1m
        metrics_path: /metrics
        static_configs:
          - targets: ['<FQDN of your cluster>:9363']
    
  3. Perform the following query to see if the cluster is alive and the metrics endpoint is opened. The response for this query returns all the available metrics:

    curl <FQDN or IP address of your cluster>:9363/metrics
    

    Consider the following metrics to get started with monitoring. You can add them as rules or expressions in the web interface:

    • ClickHouseProfileEvents_InsertQuery: the number of INSERT queries to be interpreted and potentially executed.

    • ClickHouseProfileEvents_SelectQuery: The number of SELECT queries to be interpreted and potentially executed.

    • ClickHouseAsyncMetrics_MaxPartCountForPartition: The maximum number of parts for partition. If this number is low, it might lead to problems with the execution of INSERT queries.

  4. Run Prometheus. Note that you have to bind your configuration file using the absolute path:

    docker run -p 9090:9090 
    -v /etc/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml \
    prom/prometheus
    
    sudo systemctl start prometheus
    

See also: