Contact UsGet started

Managing backups in Managed Service for ClickHouse®

Written by
Updated at November 18, 2024

You can create backups and restore clusters from existing backups.

Managed Service for ClickHouse® also creates automatic daily backups. You can set the backup start time.

Creating a backup

Warning

Backups are created based on a random replica host. If there is no cluster host data consistency, restoring clusters from backups does not guarantee complete data recovery. For more information, see Backups.

  1. Go to the folder page and select Managed Service for ClickHouse.
  2. Click the cluster name and open the Backups tab.
  3. Click Create backup.

The service will start creating a backup without an additional confirmation.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To create a cluster backup:

  1. View a description of the CLI create ClickHouse® backup command:

    yc managed-clickhouse cluster backup --help

  2. Request the creation of a backup specifying the cluster name or ID:

    yc managed-clickhouse cluster backup <cluster_name_or_ID>

    You can get the cluster name and ID with a list of clusters in the folder.

To create a backup, use the backup REST API method for the Cluster resource or the ClusterService/Backup gRPC API call and provide the cluster ID in the clusterId parameter of your request.

You can get the cluster ID with a list of clusters in the folder.

Warning

While you are creating your backup, the cluster performance might degrade.

Restoring clusters from backups

When you restore a cluster from a backup, you create a new cluster with the backup data. If the cloud does not have sufficient resources to create such a cluster, you will not be able to restore your data from a backup. The average backup recovery speed is 10 MBps per database core.

You can restore an individual shard or the whole cluster. You can restore the whole cluster only by using the CLI or API.

Before you begin, make sure your account in Yandex Cloud is assigned the iam.serviceAccounts.user role or higher. Restoring a cluster from a backup with a link to a service account.

Important

When restoring a backup to a cluster without any ZooKeeper hosts, all the tables in the ReplicatedMergeTree family engine are converted to simple MergeTree tables. Data in the converted tables is saved. For more information, see the ClickHouse® documentation.

To restore an existing cluster from a backup:

  1. Go to the folder page and select Managed Service for ClickHouse.
  2. Click the cluster name and open the Backups tab.
  3. Click for the backup you need and click Restore cluster.
  4. Change the settings of the new cluster if required. You can select a folder for the new cluster from the Folder list.
  5. Click Restore cluster.

To restore a previously deleted cluster from a backup:

  1. Go to the folder page and select Managed Service for ClickHouse.
  2. In the left-hand panel, select Backups.
  3. Find the backup you need using the backup creation time and cluster ID. The ID column contains IDs formatted as <cluster_ID>:<backup_IP>.
  4. Click for the backup you need and click Restore cluster.
  5. Change the settings of the new cluster if required. You can select a folder for the new cluster from the Folder list.
  6. Click Restore cluster.

Managed Service for ClickHouse® will launch the operation to create a cluster from the backup.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To restore a cluster from a backup:

  1. View a description of the CLI restore ClickHouse® cluster command:

    yc managed-clickhouse cluster restore --help

  2. Getting a list of available ClickHouse® cluster backups:

    yc managed-clickhouse backup list

    +--------------------------+---------------------+----------------------+---------------------+-------------+-------+-----------+
|            ID            |     CREATED AT      |  SOURCE CLUSTER ID   |     STARTED AT      | SHARD NAMES | SIZE  |   TYPE    |
+--------------------------+---------------------+----------------------+---------------------+-------------+-------+-----------+
| c9qud5etkq19********:... | 2023-12-08 00:09:17 | c9qud5etkq19******** | 2023-12-08 00:08:06 | shard1      | 30 KB | AUTOMATED |
| ...                      |                     |                      |                     |             |       |           |
+--------------------------+---------------------+----------------------+---------------------+-------------+-------+-----------+

  3. To restore an individual shard, pass the ID of a single backup:

    yc managed-clickhouse cluster restore \
   --backup-id=<backup_ID> \
   --name=<cluster_name> \
   --environment=<environment> \
   --network-name=<network_name> \
   --host type=<host_type>,`
         `zone-id=<availability_zone>,`
         `subnet-id=<subnet_ID> \
   --clickhouse-disk-size=<storage_size_in_GB> \
   --clickhouse-disk-type=<disk_type> \
   --clickhouse-resource-preset=<host_class>

    Where:

    • --backup-id: Backup ID.

    • --name: Cluster name.

    • --environment: Environment:

      • PRODUCTION: For stable versions of your apps.
      • PRESTABLE: For testing purposes. The prestable environment is similar to the production environment and likewise covered by the SLA, but it is the first to get new functionalities, improvements, and bug fixes. In the prestable environment, you can test compatibility of new versions with your application.

    • --network-name: Network name.

    • --host: Host parameters:

      • type: Host type: clickhouse or zookeeper.
      • zone-id: Availability zone.
      • subnet-id: Subnet ID. Specify if two or more subnets are created in the selected availability zone.

    • --resource-preset: Host class.

    • --disk-size: Storage size in GB.

    • --disk-type: Disk type:

      • network-hdd

      • network-ssd

      • local-ssd

      • network-ssd-nonreplicated

  4. To restore the entire cluster, provide backup IDs for all cluster shards:

    yc managed-clickhouse cluster restore \
   --backup-id=<list_of_backup_IDs_for_all_shards> \
   ...

To restore a cluster from a backup, use the restore REST API method for the Cluster resource or the ClusterService/Restore gRPC API call and provide the following in the request:

  • The shard's backup ID in the backupId parameter. To find out the ID, get a list of cluster backups.
  • Name of the new cluster that will contain the data recovered from the backup, in the name parameter. It must be unique within the folder.
  • Cluster environment in the environment parameter.
  • Cluster configuration in the configSpec parameter.
  • Configuration of the cluster hosts in one or more hostSpecs parameters.
  • Network ID in the networkId parameter.

To restore the whole cluster, provide backup IDs for the remaining cluster shards in the additionalBackupIds parameter.

Getting a list of backups

To get a list of cluster backups:

  1. Go to the folder page and select Managed Service for ClickHouse.
  2. Click the cluster name and open the Backups tab.

To get a list of all backups in a folder:

  1. Go to the folder page and select Managed Service for ClickHouse.
  2. In the left-hand panel, select Backups.

These lists contain the following information:

  • Backup name.
  • Source shard.
  • Backup size.
  • Backup type: Automatic (Automated) or manual (Manual).
  • Start time of backup creation in UTC (Coordinated Universal Time).
  • End time of backup creation in UTC.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To get a list of ClickHouse® cluster backups available in the default folder, run the command:

yc managed-clickhouse backup list

+--------------------------+---------------------+----------------------+---------------------+-------------+-------+-----------+
|            ID            |     CREATED AT      |  SOURCE CLUSTER ID   |     STARTED AT      | SHARD NAMES | SIZE  |   TYPE    |
+--------------------------+---------------------+----------------------+---------------------+-------------+-------+-----------+
| c9qud5etkq19********:... | 2023-12-08 00:09:17 | c9qud5etkq19******** | 2023-12-08 00:08:06 | shard1      | 30 KB | AUTOMATED |
| c9qud5etkq19********:... | 2023-12-07 08:17:04 | c9qud5etkq19******** | 2023-12-07 08:15:54 | shard1      | 30 KB | MANUAL    |
+--------------------------+---------------------+----------------------+---------------------+-------------+-------+-----------+

The resulting table contains the following information:

  • ID of the backup.
  • End time of backup creation in UTC (Coordinated Universal Time).
  • ID of the cluster that the backup was created for.
  • Start time of backup creation in UTC.
  • Source shard name.
  • Backup size.
  • Backup type: Automatic (AUTOMATED) or manual (MANUAL).

To get a list of cluster backups, use the listBackups REST API method for the Cluster resource or the ClusterService/ListBackups gRPC API call and provide the cluster ID in the clusterId parameter of your request.

To get a list of backups for all the Managed Service for ClickHouse® clusters in the folder, use the list REST API method for the Backup resource or the BackupService/List gRPC API call and provide the folder ID in the folderId parameter of your request.

You can get the cluster ID with a list of clusters in the folder.

Getting information about backups

To get information about the backup of an existing cluster:

  1. Go to the folder page and select Managed Service for ClickHouse.
  2. Click the cluster name and open the Backups tab.

To get information about the backup of a previously deleted cluster:

  1. Go to the folder page and select Managed Service for ClickHouse.
  2. In the left-hand panel, select Backups.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To get information about a ClickHouse® cluster backup, run the command:

yc managed-clickhouse backup get <backup_ID>

You can retrieve the backup ID with a list of backups.

To get information about a backup, use the get REST API method for the Backup resource or the BackupService/Get gRPC API call and provide the backup ID in the backupId parameter of your request.

To find out the ID, retrieve a list of backups.

Setting the backup start time

In the management console, you can set the backup start time when creating or updating a cluster.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To set the backup start time, provide the required value in HH:MM:SS format in the --backup-window-start argument of the cluster update command:

yc managed-clickhouse cluster update <cluster_name_or_ID> \
   --backup-window-start=<backup_start_time>

You can request the cluster ID and name with a list of clusters in the folder.

To set the backup start time, use the update REST API method for the Cluster resource or the ClusterService/Update gRPC API call and provide the following in the request:

  • Cluster ID in the clusterId parameter. You can get it with a list of clusters in the folder.
  • New backup start time in the configSpec.backupWindowStart parameter.
  • List of updatable cluster configuration fields in the updateMask parameter (in this case, configSpec.backupWindowStart).

Warning

The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.

ClickHouse® is a registered trademark of ClickHouse, Inc.

Previous
Migrating hosts to a different availability zone
Next
Deleting a cluster