Yandex Cloud
Search
Discuss with expertTry it for free
  • Customer Stories
  • Documentation
  • Blog
  • All Services
    • Cloud Interconnect
    • Cloud Backup
    • Cloud Registry
    • Yandex AI Studio
    • Compute Cloud
    • Object Storage
    • Managed Service for Kubernetes®
    • Yandex BareMetal
    • Smart Web Security
    • Security Deck
    • Managed Service for PostgreSQL
    • Managed Service for ClickHouse®
    • Monium
    • Cloud CDN
    • Network Load Balancer
    • Virtual Private Cloud
    • Cloud DNS
    • Application Load Balancer
    • Yandex Cloud Video
    • Stackland
    • Yandex Cloud Router
    • Yandex Managed Service for Trino
    • Managed Service for MySQL®
    • Managed Service for Valkey™
    • Managed Service for Apache Spark™
    • Yandex StoreDoc
    • Managed Service for OpenSearch
    • Managed Service for Apache Kafka®
    • Data Transfer
    • Yandex MPP Analytics Engine for PostgreSQL
    • Yandex Managed Service for Apache Airflow®
    • Data Processing
    • Yandex MetaData Hub
    • Managed Service for YDB
    • Managed Service for Sharded PostgreSQL
    • Managed Service for YTsaurus
    • Yandex WebSQL
    • DataLens
    • Yandex Search API
    • SpeechSense
    • SpeechKit
    • DataSphere
    • Vision OCR
    • Translate
    • Yandex Identity Hub
    • Key Management Service
    • Certificate Manager
    • Yandex Lockbox
    • Audit Trails
    • SmartCaptcha
    • Yandex SIEM
    • SourceCraft Code Assistant
    • Container Registry
    • Managed Service for GitLab
    • Managed Service for Prometheus®
    • Cloud Functions
    • API Gateway
    • Yandex Cloud Postbox
    • Message Queue
    • Data Streams
    • Serverless Containers
    • Cloud Notification Service
    • Identity and Access Management
    • Yandex Cloud Console
    • Resource Manager
    • Yandex Cloud Billing
    • Yandex Cloud Quota Manager
    • Cloud Apps
  • Marketplace
    • Featured
    • Infrastructure & Network
    • Data Platform
    • AI for business
    • Security
    • DevOps tools
    • Serverless
    • Monitoring & Resources
  • All Solutions
    • By industry
    • By use case
    • Economics and Pricing
    • Security
    • Technical Support
    • Start testing with double trial credits
    • Cloud credits to scale your IT product
    • Gateway to Russia
    • Cloud for Startups
    • Center for Technologies and Society
    • Yandex Cloud Partner program
    • Price calculator
    • Pricing plans
  • Customer Stories
  • Documentation
  • Blog
© 2026 Direct Cursus Technology L.L.C.
Yandex Managed Service for ClickHouse®
  • Getting started
    • All guides
      • Managing extensions
      • Managing a custom geobase
      • Connecting external dictionaries
      • Managing format schemas
      • Managing machine learning models
      • Configuring access to Object Storage
      • Working with hybrid storage
  • Access management
  • Inspections and recommendations
  • Pricing policy
  • Yandex Monitoring metrics
  • Audit Trails events
  • Public materials
  • Release notes

In this article:

  • Enabling hybrid storage for a cluster
  • Enabling hybrid storage when creating a cluster
  • Enabling hybrid storage for an existing cluster
  • Changing the hybrid storage settings
  • Setting data storage policies
  • Getting information about policies
  • Setting a policy when creating a table
  • Setting a policy for an existing a table
  • Monitoring space taken up by data in Object Storage
  • Use cases
  1. Step-by-step guides
  2. Storing and processing data
  3. Working with hybrid storage

Working with hybrid storage

Written by
Yandex Cloud
Updated at November 16, 2026
View in Markdown
  • Enabling hybrid storage for a cluster
    • Enabling hybrid storage when creating a cluster
    • Enabling hybrid storage for an existing cluster
  • Changing the hybrid storage settings
  • Setting data storage policies
    • Getting information about policies
    • Setting a policy when creating a table
    • Setting a policy for an existing a table
  • Monitoring space taken up by data in Object Storage
  • Use cases

Hybrid storage enables you to store frequently used data in cluster storage (on the Managed Service for ClickHouse® cluster's network drives) while keeping rarely used data in Yandex Object Storage.

Enabling hybrid storage for a clusterEnabling hybrid storage for a cluster

Warning

Once hybrid storage is enabled, you cannot disable it.

Enabling hybrid storage when creating a clusterEnabling hybrid storage when creating a cluster

Management console
CLI
Terraform
REST API
gRPC API
  1. In the management console, select the folder where you want to create a ClickHouse® cluster.

  2. Navigate to Managed Service for ClickHouse.

  3. Click Create cluster and set the cluster parameters. To learn more about parameters, see the guide on creating a cluster.

  4. Enable Hybrid storage under DBMS settings and, optionally, set the hybrid storage settings:

    • Data caching: Enables caching data accessed from the object storage in the cluster storage. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing.

      By default, data caching is enabled.

    • Cache size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

    • Min. share of free space on cluster disks: Sets the minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

      Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

  5. Click Create cluster.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id.

If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

To create a cluster with hybrid storage enabled and, optionally, configure its settings:

  1. View the description of the CLI command for creating a cluster:

    yc managed-clickhouse cluster create --help
    
  2. Run this command, specifying hybrid storage settings:

    yc managed-clickhouse cluster create \
       ...
       --cloud-storage=true \
       --cloud-storage-data-cache=<cache_data> \
       --cloud-storage-data-cache-max-size=<maximum_memory_in_bytes> \
       --cloud-storage-move-factor=<share_of_free_space> \
       --cloud-storage-prefer-not-to-merge=<disable_merging_data_parts>
      ...
    

    Where:

    • --cloud-storage: Enables hybrid storage.
    • --cloud-storage-data-cache: Enables caching data accessed from the object storage in the cluster storage. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

    • --cloud-storage-data-cache-max-size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

    • --cloud-storage-move-factor: Sets the minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to Yandex Object Storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

      Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

    • --cloud-storage-prefer-not-to-merge: Disables merging of data parts in cluster and object storages. To disable merges, set the parameter to true or provide it with no value. To keep merges enabled, set the parameter to false or do not provide it in the CLI command when creating a cluster.

      Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

    To learn more about the yc managed-clickhouse cluster create command's other parameters, see the guide on creating a cluster.

  1. Create a Terraform configuration file describing your infrastructure.

  2. Add the cloud_storage section to the configuration file:

    resource "yandex_mdb_clickhouse_cluster_v2" "<cluster_name>" {
      ...
      cloud_storage = {
        enabled             = true
        move_factor         = <share_of_free_space>
        data_cache_enabled  = <cache_data>
        data_cache_max_size = <maximum_memory_in_bytes>
        prefer_not_to_merge = <disable_merging_data_parts>
      }
    }
    

    Where:

    • enabled: Enables hybrid storage.
    • move_factor: Minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

      Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

    • data_cache_enabled: Allows caching data accessed from the object storage in the cluster storage, true or false. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

    • data_cache_max_size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

    • prefer_not_to_merge: Disable merging of data parts in the cluster and object storage, true or false. To disable merging, set to true. To leave merging enabled, set to false.

      Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

  3. Make sure the Terraform configuration file is correct:

    1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

    2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

  4. Create a cluster:

    1. Run this command to view the intended changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

    2. If everything looks correct, apply the changes:

      1. Run this command:

        terraform apply
        
      2. Confirm updating the resources.

      3. Wait for the operation to complete.

Timeouts

The Terraform provider sets the following timeouts for Managed Service for ClickHouse® cluster operations:

  • Creating a cluster, including by restoring from a backup: 60 minutes.
  • Updating a cluster: 90 minutes.
  • Deleting a cluster: 30 minutes.

Operations exceeding the timeout are aborted.

How to change these limits

Add the timeouts section to your cluster description, such as the following:

resource "yandex_mdb_clickhouse_cluster_v2" "<cluster_name>" {
  ...
  timeouts = {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}
  1. Get an IAM token for API authentication and put it into an environment variable:

    export IAM_TOKEN="<IAM_token>"
    
  2. Call the Cluster.Create method, e.g., via the following cURL request:

    1. Create a file named body.json and paste the following code into it:

      {
        <cluster_parameters>
        ...
        "configSpec": {
          "cloudStorage": {
            "enabled": true,
            "moveFactor": "<share_of_free_space>",
            "dataCacheEnabled": <cache_data>,
            "dataCacheMaxSize": "<maximum_memory_in_bytes>",
            "preferNotToMerge": <disable_merging_data_parts>
          },
          ...
        }
      }
      

      Where:

      • enabled: Enables hybrid storage.
      • moveFactor: Minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

        Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

      • dataCacheEnabled: Allows caching data accessed from the object storage in the cluster storage, true or false. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

      • dataCacheMaxSize: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

      • preferNotToMerge: Disable merging of data parts in the cluster and object storage, true or false. To disable merging, set to true. To leave merging enabled, set to false.

        Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

      To learn more about other parameters, see the guide on creating a cluster.

    2. Run this request:

      curl \
        --request POST \
        --header "Authorization: Bearer $IAM_TOKEN" \
        --header "Content-Type: application/json" \
        --url 'https://mdb.api.cloud.yandex.net/managed-clickhouse/v1/clusters' \
        --data '@body.json'
      
  3. View the server response to make sure your request was successful.

  1. Get an IAM token for API authentication and put it into an environment variable:

    export IAM_TOKEN="<IAM_token>"
    
  2. Clone the cloudapi repository:

    cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
    

    Below, we assume that the repository contents reside in the ~/cloudapi/ directory.

  3. Call the ClusterService.Create method, e.g., via the following gRPCurl request:

    1. Create a file named body.json and paste the following code into it:

      {
        <cluster_parameters>
        ...
        "config_spec": {
          "cloud_storage": {
            "enabled": true,
            "move_factor": "<share_of_free_space>",
            "data_cache_enabled": <cache_data>,
            "data_cache_max_size": "<maximum_memory_in_bytes>",
            "prefer_not_to_merge": <disable_merging_data_parts>
          },
          ...
        }
      }
      

      Where:

      • enabled: Enables hybrid storage.
      • move_factor: Minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

        Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

      • data_cache_enabled: Allows caching data accessed from the object storage in the cluster storage, true or false. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

      • data_cache_max_size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

      • prefer_not_to_merge: Disable merging of data parts in the cluster and object storage, true or false. To disable merging, set to true. To leave merging enabled, set to false.

        Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

      To learn more about other parameters, see the guide on creating a cluster.

    2. Run this request:

      grpcurl \
        -format json \
        -import-path ~/cloudapi/ \
        -import-path ~/cloudapi/third_party/googleapis/ \
        -proto ~/cloudapi/yandex/cloud/mdb/clickhouse/v1/cluster_service.proto \
        -rpc-header "Authorization: Bearer $IAM_TOKEN" \
        -d @ \
        mdb.api.cloud.yandex.net:443 \
        yandex.cloud.mdb.clickhouse.v1.ClusterService.Create \
        < body.json
      
  4. Check the server response to make sure your request was successful.

Enabling hybrid storage for an existing clusterEnabling hybrid storage for an existing cluster

Management console
CLI
Terraform
REST API
gRPC API
  1. In the management console, select the folder containing the cluster.

  2. Navigate to Managed Service for ClickHouse.

  3. Select your cluster and click Edit in the top panel.

  4. Enable Hybrid storage under DBMS settings and, optionally, set the hybrid storage settings:

    • Data caching: Enables caching data accessed from the object storage in the cluster storage. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing.

      By default, data caching is enabled.

    • Cache size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

    • Min. share of free space on cluster disks: Sets the minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

      Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

  5. Click Save changes.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id.

If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

To enable hybrid storage in a cluster and, optionally, configure its settings, run this command:

yc managed-clickhouse cluster update <cluster_name_or_ID> \
   --cloud-storage=true \
   --cloud-storage-data-cache=<cache_data> \
   --cloud-storage-data-cache-max-size=<maximum_memory_in_bytes> \
   --cloud-storage-move-factor=<share_of_free_space> \
   --cloud-storage-prefer-not-to-merge=<disable_merging_data_parts>

Where:

  • <cluster_name_or_ID>: Cluster name or ID which you can get with the list of clusters in the folder.
  • --cloud-storage: Enables hybrid storage.
  • --cloud-storage-data-cache: Enables caching data accessed from the object storage in the cluster storage. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

  • --cloud-storage-data-cache-max-size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

  • --cloud-storage-move-factor: Sets the minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to Yandex Object Storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

    Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

  • --cloud-storage-prefer-not-to-merge: Disables merging of data parts in cluster and object storages. To disable merges, set the parameter to true or provide it with no value. To keep merges enabled, set the parameter to false or do not provide it in the CLI command when creating a cluster.

    Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

To enable hybrid storage in a cluster and, optionally, configure its settings:

  1. Open the current Terraform configuration file with the infrastructure plan.

    To learn how to create this file, see Creating a cluster.

    For the complete list of configurable Managed Service for ClickHouse® cluster fields, see this Terraform provider guide.

  2. Add the cloud_storage section to the Managed Service for ClickHouse® cluster description:

    resource "yandex_mdb_clickhouse_cluster_v2" "<cluster_name>" {
      ...
      cloud_storage = {
        enabled             = true
        move_factor         = <share_of_free_space>
        data_cache_enabled  = <cache_data>
        data_cache_max_size = <maximum_memory_in_bytes>
        prefer_not_to_merge = <disable_merging_data_parts>
      }
    }
    

    Where:

    • enabled: Enables hybrid storage.
    • move_factor: Minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

      Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

    • data_cache_enabled: Allows caching data accessed from the object storage in the cluster storage, true or false. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

    • data_cache_max_size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

    • prefer_not_to_merge: Disable merging of data parts in the cluster and object storage, true or false. To disable merging, set to true. To leave merging enabled, set to false.

      Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

  3. Make sure the Terraform configuration file is correct:

    1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

    2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

  4. Confirm resource changes:

    1. Run this command to view the intended changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

    2. If everything looks correct, apply the changes:

      1. Run this command:

        terraform apply
        
      2. Confirm updating the resources.

      3. Wait for the operation to complete.

Timeouts

The Terraform provider sets the following timeouts for Managed Service for ClickHouse® cluster operations:

  • Creating a cluster, including by restoring from a backup: 60 minutes.
  • Updating a cluster: 90 minutes.
  • Deleting a cluster: 30 minutes.

Operations exceeding the timeout are aborted.

How to change these limits

Add the timeouts section to your cluster description, such as the following:

resource "yandex_mdb_clickhouse_cluster_v2" "<cluster_name>" {
  ...
  timeouts = {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}
  1. Get an IAM token for API authentication and put it into an environment variable:

    export IAM_TOKEN="<IAM_token>"
    
  2. Call the Cluster.Update method, e.g., via the following cURL request:

    Warning

    The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.

    curl \
        --request PATCH \
        --header "Authorization: Bearer $IAM_TOKEN" \
        --header "Content-Type: application/json" \
        --url 'https://mdb.api.cloud.yandex.net/managed-clickhouse/v1/clusters/<cluster_ID>' \
        --data '{
                  "updateMask": "<list_of_settings_to_update>",
                  "configSpec": {
                    "cloudStorage": {
                      "enabled": true,
                      "moveFactor": "<share_of_free_space>",
                      "dataCacheEnabled": <cache_data>,
                      "dataCacheMaxSize": "<maximum_memory_in_bytes>",
                      "preferNotToMerge": <disable_merging_data_parts>
                    }
                  }
                }'
    

    Where:

    • <cluster_ID>: Cluster ID which you can get with the list of clusters in the folder.

    • updateMask: Comma-separated string of settings to update.

    • configSpec.cloudStorage: Hybrid storage settings:

      • enabled: Enables hybrid storage.
      • moveFactor: Minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

        Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

      • dataCacheEnabled: Allows caching data accessed from the object storage in the cluster storage, true or false. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

      • dataCacheMaxSize: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

      • preferNotToMerge: Disable merging of data parts in the cluster and object storage, true or false. To disable merging, set to true. To leave merging enabled, set to false.

        Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

  3. Check the server response to make sure your request was successful.

  1. Get an IAM token for API authentication and put it into an environment variable:

    export IAM_TOKEN="<IAM_token>"
    
  2. Clone the cloudapi repository:

    cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
    

    Below, we assume that the repository contents reside in the ~/cloudapi/ directory.

  3. Call the ClusterService.Update method, e.g., via the following gRPCurl request:

    Warning

    The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the update_mask parameter as an array of paths[] strings.

    Format for listing settings
    "update_mask": {
        "paths": [
            "<setting_1>",
            "<setting_2>",
            ...
            "<setting_N>"
        ]
    }
    
    grpcurl \
        -format json \
        -import-path ~/cloudapi/ \
        -import-path ~/cloudapi/third_party/googleapis/ \
        -proto ~/cloudapi/yandex/cloud/mdb/clickhouse/v1/cluster_service.proto \
        -rpc-header "Authorization: Bearer $IAM_TOKEN" \
        -d '{
              "cluster_id": "<cluster_ID>",
              "update_mask": {
                "paths": [
                  <list_of_settings_to_update>
                ]
              },
              "config_spec": {
                "cloud_storage": {
                  "enabled": true,
                  "move_factor": "<share_of_free_space>",
                  "data_cache_enabled": <cache_data>,
                  "data_cache_max_size": "<maximum_memory_in_bytes>",
                  "prefer_not_to_merge": <disable_merging_data_parts>
                }
              }
            }' \
        mdb.api.cloud.yandex.net:443 \
        yandex.cloud.mdb.clickhouse.v1.ClusterService.Update
    

    Where:

    • <cluster_ID>: Cluster ID which you can get with the list of clusters in the folder.

    • update_mask: List of settings to update as an array of strings (paths[]).

    • config_spec.cloud_storage: Hybrid storage settings:

      • enabled: Enables hybrid storage.
      • move_factor: Minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

        Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

      • data_cache_enabled: Allows caching data accessed from the object storage in the cluster storage, true or false. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

      • data_cache_max_size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

      • prefer_not_to_merge: Disable merging of data parts in the cluster and object storage, true or false. To disable merging, set to true. To leave merging enabled, set to false.

        Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

    You can get the cluster ID with the list of clusters in the folder.

  4. Check the server response to make sure your request was successful.

Changing the hybrid storage settingsChanging the hybrid storage settings

If a Managed Service for ClickHouse® cluster has hybrid storage enabled, you can change its settings.

Management console
CLI
Terraform
REST API
gRPC API
  1. In the management console, select the folder containing the cluster.

  2. Navigate to Managed Service for ClickHouse.

  3. Select your cluster and click Edit in the top panel.

  4. Under DBMS settings, edit the hybrid storage settings:

    • Data caching: Enables caching data accessed from the object storage in the cluster storage. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing.

      By default, data caching is enabled.

    • Cache size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

    • Min. share of free space on cluster disks: Sets the minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

      Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

  5. Click Save changes.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id.

If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

To edit hybrid storage settings, run this command:

yc managed-clickhouse cluster update <cluster_name_or_ID> \
    --cloud-storage-data-cache=<cache_data> \
    --cloud-storage-data-cache-max-size=<maximum_memory_in_bytes> \
    --cloud-storage-move-factor=<share_of_free_space> \
    --cloud-storage-prefer-not-to-merge=<disable_merging_data_parts>

Where:

  • <cluster_name_or_ID>: Cluster name or ID which you can get with the list of clusters in the folder.
  • --cloud-storage-data-cache: Enables caching data accessed from the object storage in the cluster storage. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

  • --cloud-storage-data-cache-max-size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

  • --cloud-storage-move-factor: Sets the minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to Yandex Object Storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

    Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

  • --cloud-storage-prefer-not-to-merge: Disables merging of data parts in cluster and object storages. To disable merges, set the parameter to true or provide it with no value. To keep merges enabled, set the parameter to false or do not provide it in the CLI command when creating a cluster.

    Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

  1. Open the current Terraform configuration file with the infrastructure plan.

    To learn how to create this file, see Creating a cluster.

    For the complete list of configurable Managed Service for ClickHouse® cluster fields, see this Terraform provider guide.

  2. In the Managed Service for ClickHouse® cluster description, edit the cloud_storage section attributes:

    resource "yandex_mdb_clickhouse_cluster_v2" "<cluster_name>" {
      ...
      cloud_storage = {
        ...
        move_factor         = <share_of_free_space>
        data_cache_enabled  = <cache_data>
        data_cache_max_size = <maximum_memory_in_bytes>
        prefer_not_to_merge = <disable_merging_data_parts>
      }
    }
    

    Where:

    • move_factor: Minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

      Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

    • data_cache_enabled: Allows caching data accessed from the object storage in the cluster storage, true or false. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

    • data_cache_max_size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

    • prefer_not_to_merge: Disable merging of data parts in the cluster and object storage, true or false. To disable merging, set to true. To leave merging enabled, set to false.

      Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

  3. Make sure the Terraform configuration file is correct:

    1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

    2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

  4. Confirm resource changes:

    1. Run this command to view the intended changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

    2. If everything looks correct, apply the changes:

      1. Run this command:

        terraform apply
        
      2. Confirm updating the resources.

      3. Wait for the operation to complete.

Timeouts

The Terraform provider sets the following timeouts for Managed Service for ClickHouse® cluster operations:

  • Creating a cluster, including by restoring from a backup: 60 minutes.
  • Updating a cluster: 90 minutes.
  • Deleting a cluster: 30 minutes.

Operations exceeding the timeout are aborted.

How to change these limits

Add the timeouts section to your cluster description, such as the following:

resource "yandex_mdb_clickhouse_cluster_v2" "<cluster_name>" {
  ...
  timeouts = {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}
  1. Get an IAM token for API authentication and put it into an environment variable:

    export IAM_TOKEN="<IAM_token>"
    
  2. Call the Cluster.Update method, e.g., via the following cURL request:

    Warning

    The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.

    curl \
        --request PATCH \
        --header "Authorization: Bearer $IAM_TOKEN" \
        --header "Content-Type: application/json" \
        --url 'https://mdb.api.cloud.yandex.net/managed-clickhouse/v1/clusters/<cluster_ID>' \
        --data '{
                  "updateMask": "<list_of_settings_to_update>",
                  "configSpec": {
                    "cloudStorage": {
                      "moveFactor": "<share_of_free_space>",
                      "dataCacheEnabled": <cache_data>,
                      "dataCacheMaxSize": "<maximum_memory_in_bytes>",
                      "preferNotToMerge": <disable_merging_data_parts>
                    }
                  }
                }'
    

    Where:

    • <cluster_ID>: Cluster ID which you can get with the list of clusters in the folder.

    • updateMask: Comma-separated string of settings to update.

    • configSpec.cloudStorage: Hybrid storage settings:

      • moveFactor: Minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

        Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

      • dataCacheEnabled: Allows caching data accessed from the object storage in the cluster storage, true or false. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

      • dataCacheMaxSize: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

      • preferNotToMerge: Disable merging of data parts in the cluster and object storage, true or false. To disable merging, set to true. To leave merging enabled, set to false.

        Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

  3. Check the server response to make sure your request was successful.

  1. Get an IAM token for API authentication and put it into an environment variable:

    export IAM_TOKEN="<IAM_token>"
    
  2. Clone the cloudapi repository:

    cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
    

    Below, we assume that the repository contents reside in the ~/cloudapi/ directory.

  3. Call the ClusterService.Update method, e.g., via the following gRPCurl request:

    Warning

    The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the update_mask parameter as an array of paths[] strings.

    Format for listing settings
    "update_mask": {
        "paths": [
            "<setting_1>",
            "<setting_2>",
            ...
            "<setting_N>"
        ]
    }
    
    grpcurl \
        -format json \
        -import-path ~/cloudapi/ \
        -import-path ~/cloudapi/third_party/googleapis/ \
        -proto ~/cloudapi/yandex/cloud/mdb/clickhouse/v1/cluster_service.proto \
        -rpc-header "Authorization: Bearer $IAM_TOKEN" \
        -d '{
              "cluster_id": "<cluster_ID>",
              "update_mask": {
                "paths": [
                  <list_of_settings_to_update>
                ]
              },
              "config_spec": {
                "cloud_storage": {
                  "move_factor": "<share_of_free_space>",
                  "data_cache_enabled": <cache_data>,
                  "data_cache_max_size": "<maximum_memory_in_bytes>",
                  "prefer_not_to_merge": <disable_merging_data_parts>
                }
              }
            }' \
        mdb.api.cloud.yandex.net:443 \
        yandex.cloud.mdb.clickhouse.v1.ClusterService.Update
    

    Where:

    • <cluster_ID>: Cluster ID which you can get with the list of clusters in the folder.

    • update_mask: List of settings to update as an array of strings (paths[]).

    • config_spec.cloud_storage: Hybrid storage settings:

      • move_factor: Minimum percentage of free space in the cluster storage. If your free space percentage is below this value, the data will be moved to object storage. The minimum value is 0, the maximum value is 1, and the default value is 0.01.

        Data parts are queued up in descending order by size. The number of data parts that will be moved is determined by the condition regarding the specified minimum share of free space in cluster storage.

      • data_cache_enabled: Allows caching data accessed from the object storage in the cluster storage, true or false. As a result, the cold data accessed from the object storage is moved to high-speed drives for faster processing. The default value is true (allowed).

      • data_cache_max_size: Sets the maximum cache size, in bytes, allocated in the cluster storage for data accessed from the object storage. If no value is set, the maximum cache size defaults to half the size of the cluster storage.

      • prefer_not_to_merge: Disable merging of data parts in the cluster and object storage, true or false. To disable merging, set to true. To leave merging enabled, set to false.

        Once inserted into the table, the data is saved as a data part and sorted by its primary key. Then, ClickHouse® runs a background merge of data parts belonging to the same partition into a larger data part within 10 to 15 minutes after the insertion. You can use the system.parts system table to view the merged data parts and partitions.

  4. Check the server response to make sure your request was successful.

Setting data storage policiesSetting data storage policies

In a Managed Service for ClickHouse® cluster with hybrid storage enabled, three storage policies are available: default, local, and object_storage. For DB tables, you can specify one of them. You cannot create new storage policies or update the existing ones.

Getting information about policiesGetting information about policies

To view policy settings, run this query:

SQL
SELECT *
FROM system.storage_policies;

Setting a policy when creating a tableSetting a policy when creating a table

To set a policy when creating a table, run this query:

SQL
CREATE TABLE <table_name> (
    <table_schema>
) ENGINE = MergeTree
...
SETTINGS storage_policy = '<storage_policy>';

Setting a policy for an existing a tableSetting a policy for an existing a table

To set or edit a policy for an existing a table, run this query:

SQL
ALTER TABLE <table_name>
MODIFY SETTING storage_policy = '<storage_policy>';

Monitoring space taken up by data in Object StorageMonitoring space taken up by data in Object Storage

To monitor the amount of space MergeTree table parts occupy in Object Storage, use the ch_s3_disk_parts_size metric in Yandex Monitoring.

Management console
  1. Open the management console.

  2. Navigate to Monitoring.

  3. Open the Metric Explorer section.

  4. Run this request:

    "ch_s3_disk_parts_size"{service="managed-clickhouse", resource_type="cluster", node="by_host", resource_id="<cluster_ID>", subcluster_name="clickhouse_subcluster"}
    

Use casesUse cases

  • Using hybrid storage in Managed Service for ClickHouse®

Was the article helpful?

Previous
Configuring access to Object Storage
Next
Managing shards
© 2026 Direct Cursus Technology L.L.C.