ClickHouse Keeper

ClickHouse Keeper is a full-featured replacement for Apache ZooKeeper, developed specifically to coordinate and manage ClickHouse^® clusters. It ensures full compatibility with the ZooKeeper client protocol and uses the same data model, while offering major improvements in performance and functionality.

Simplified installation and operation

• Implemented in C++ instead of Java for more efficient resource utilization.
• Can operate either embedded into ClickHouse^® or as a standalone service.
• Uses less memory than ZooKeeper for the same data volume.

Optimized data storage

• Snapshots and logs take up much less disk space due to improved compression.
• No default packet size or node data size limits, versus ZooKeeper’s 1 MB limit.
• No ZXID overflow problem that requires ZooKeeper to restart every 2 billion transactions.

Enhanced performance and reliability

• Faster post-network segmentation recovery thanks to an improved distributed consensus protocol.
• Extra consistency guarantees. Ensures the same consistency guarantees as ZooKeeper: linearizable writes and strict operation order within a single session.
• Optional linearizable reads via the quorum_reads setting.

Core coordination features

ClickHouse Keeper performs critical functions in the ClickHouse^® ecosystem:

• Provides a coordination system for data replication in self-managed ClickHouse^® clusters.
• Automatically deduplicates inserts for replicated MergeTree-family tables using block hash checksums.
• Provides consensus with regard to part names, assignment of part merges and mutations to particular cluster nodes.
• Enables the KeeperMap table engine, which allows using Keeper as a consistent key-value store.
• Tracks processed files in the S3Queue table engine.
• Stores all Replicated Database metadata.
• Coordinates the creation of backups via the ON CLUSTER clause.
• Serves as a repository for user-defined functions and access control data.
• Operates as the central storage of all ClickHouse Cloud’s metadata.

ClickHouse Keeper is a robust and efficient coordination solution for distributed systems, purpose-built and optimized for ClickHouse^®.

1. Create a cloud network and a subnet to host the virtual machine (VM).

2. Create a security group for the VM named vm-keeper in your new cloud network and configure the rules as follows:

   Traffic direction	Description	Port range	Protocol	Source / Destination	CIDR blocks
   Ingress	CH Keeper	9181	TCP	CIDR	0.0.0.0/0
   Ingress	SSH	22	TCP	CIDR	0.0.0.0/0
   Egress	Any Egress	0-65535	Any	CIDR	0.0.0.0/0

3. Create a security group for the VM named vm-test in the same network and configure the rules as follows:

   Traffic direction	Description	Port range	Protocol	Source / Destination	CIDR blocks
   Ingress	HTTP	80	TCP	CIDR	0.0.0.0/0
   Ingress	SSH	22	TCP	CIDR	0.0.0.0/0
   Egress	Any Egress	0-65535	Any	CIDR	0.0.0.0/0

4. Get an SSH key pair for connection to the VM.

5. Install and configure the application.

   1. In our Marketplace, find ClickHouse Keeper and click Create VM.

      1. Under Network settings, specify the earlier-created cloud network, subnet, and security group for vm-keeper.

      2. Under Access, specify the username and public SSH key for connection to the VM.

      3. Under General information, specify the VM name: vm-keeper.

      4. Complete setting up your VM and click Create VM.

         Wait for the VM to be created and the application to be installed: this may take up to 10 minutes.

   2. Learn the public IP address of vm-keeper.

   3. Connect to vm-keeper over SSH. Use the username you set when creating the VM and the private SSH key you created earlier.

   4. Add the following to /etc/clickhouse-keeper/keeper_config.xml right after the <clickhouse> line:

      <listen_host>0.0.0.0</listen_host>
      

      The 0.0.0.0 address is not recommended in production environments.

   5. Check that ClickHouse Keeper is running:

      sudo systemctl status clickhouse-keeper
      

      Result:

      ● clickhouse-keeper.service - ClickHouse Keeper - zookeeper compatible distributed coordination server
          Loaded: loaded (/usr/lib/systemd/system/clickhouse-keeper.service; enabled; preset: enabled)
          Active: active (running) since Wed 2026-03-04 19:15:51 UTC; 17min ago
      ...
      

      If your ClickHouse Keeper is not running, start it manually using this command:

      sudo systemctl start clickhouse-keeper
      

6. Create and configure a VM to test ClickHouse Keeper operation.

   1. When creating the VM, specify the following:

      1. Under Boot disk image on the Marketplace tab, select an Ubuntu or Debian image, e.g., Ubuntu 24.04.
      2. Under Network settings, specify the earlier created cloud network, subnet, and security group for vm-test.
      3. Under Access, specify the username and public SSH key for connection to the VM.
      4. Under General information, specify the VM name: vm-test.

   2. Learn the public IP address of vm-test.

   3. Connect to vm-test over SSH. Use the username you set when creating the VM and the private SSH key you created earlier.

   4. Install ClickHouse Server and ClickHouse Client as detailed in Install ClickHouse on Debian/Ubuntu.

   5. Create a file named /etc/clickhouse-server/config.d/zookeeper.xml and paste the following code into it:

      <clickhouse>
          <zookeeper>
              <node>
                  <host>VM_public_IP_address</host>
                  <port>9181</port>
              </node>
          </zookeeper>
      </clickhouse>
      

      In the host field, specify vm-keeper's public IP address.

   6. Start a server by running this command:

      sudo systemctl start clickhouse-server
      

   7. Enable the SQL query execute mode by running this command:

      clickhouse-client
      

   8. Run this query:

      SELECT
          hostname() AS clickhouse_server,
          zookeeperSessionUptime() AS keeper_session_seconds
      

      Result:

      Query id: 040b6642-faf0-4a2b-9c79-50bb********
         ┌─clickhouse_server─┬─keeper_session_seconds─┐
      1. │ vm-test           │                    158 │
         └───────────────────┴────────────────────────┘
      1 row in set. Elapsed: 0.002 sec.
      

7. Check your connection.

   1. Connect to vm-keeper over SSH.

   2. Run this command:

      sudo ss -anp | grep :9181
      

      Result:

      tcp   LISTEN 0   64   *:9181   *:*   users:(("clickhouse-keep",pid=806,fd=42))
      tcp   ESTAB  0   0    [::ffff:10.0.0.30]:9181   [::ffff:62.**.***.***]:58646 users:(("clickhouse-keep",pid=806,fd=37))
      

      If the command is successful, your ClickHouse Server on vm-test is connected to ClickHouse Keeper on vm-keeper.

• Coordinating data replication in ClickHouse^® clusters.
• Automatic deduplication of inserts for replicated MergeTree tables.
• Implementing task queues using the KeeperMap table engine.
• Exactly-once delivery in Kafka Connect Sink.
• Tracking processed files in S3Queue.
• Storing Replicated Database metadata.
• Coordinating the creation of cluster backups.
• Storing user-defined functions and access control settings.
• Central metadata storage for ClickHouse Cloud.

Yandex Cloud technical support is available 24/7. The types of requests you can submit and the relevant response times depend on your pricing plan. You can switch to the paid support plan in the management console. You can learn more about the technical support terms and conditions here.