Updates in a v2 cluster

Written by

Updated at October 27, 2026

List of updates in v2
Complete configuration example: v1 vs. v2
Migration errors

Resource v1 (yandex_mdb_clickhouse_cluster) is now deprecated and will be deleted in one of the upcoming provider versions.

Resource v2 (yandex_mdb_clickhouse_cluster_v2) is implemented on the Terraform plugin framework and fixes a number of issues the resource v1 had:

In v1, hosts were stored as an ordered list; therefore, changing their order or any discrepancies between the plan and the current resource state might have resulted in excessive host recreation.
Users and databases within the cluster resource led to excessive changes in plan when only the cluster was changed.
Changes that Terraform showed in plan did not always match the apply result.

List of updates in v2

Nested sections are now specified as attributes.
The host section was replaced with the hosts associative array.
The coordination service's hosts were updated.
The database and user sections were deleted from the cluster resource.
The shard section was replaced with the shards associative array.
The pattern section name was changed to patterns.
The compression repeat sections were transformed into a list.
Certain fields were added, changed, or deleted.

Unchanged attributes

In both versions, the following attributes have the same names and types:

name
description
folder_id
network_id
environment
version
labels
deletion_protection
service_account_id
security_group_ids
disk_encryption_key_id
sql_user_management
sql_database_management
admin_password
embedded_keeper
copy_schema_on_new_hosts
backup_retain_period_days

In v2, the standard timeouts section is supported without changes in the syntax.

Nested sections

Nested sections are now specified as attributes. In v1, a lot of nested objects were described using the section syntax. In v2, they use the assignment syntax: = { }.

v1 (section syntax)	v2 (assignment syntax)
`clickhouse { }`	`clickhouse = { }`
`zookeeper { }`	`zookeeper = { }`
`access { }`	`access = { }`
`cloud_storage { }`	`cloud_storage = { }`
`backup_window_start { }`	`backup_window_start = { }`
`clickhouse.config.kafka { }`	`kafka = { }` (inside `config`)
`clickhouse.config.rabbitmq { }`	`rabbitmq = { }` (inside `config`)
`clickhouse.config.compression { }` (repeat section)	`compression = [{ method = "LZ4", ... }]`
`host { }` (repeat section)	`hosts = { "key" = { } }`
`shard { }` (repeat section)	`shards = { "shard1" = { } }`

The table contains an incomplete list: other nested objects inside clickhouse.config, e.g., merge_tree and query_cache, are also specified using the assignment syntax in v2.

The following sections do not require the assignment syntax and remain unchanged:

maintenance_window { }
shard_group { }
format_schema { }
ml_model { }
extension { } (only v2)

New attributes introduced in v2 that use the assignment syntax:

restore = { backup_id = "...", include_patterns = [...], exclude_patterns = [...] }.
performance_diagnostics = { enabled = true, processes_refresh_interval = "15s" }.
external_dictionary = { "dict_name" = { ... } }: Associative array, where the key is the dictionary name.

The `host`section

The host section was replaced with the hosts associative array. This is the most critical change.

v1 used repeat unnamed host { } sections, as Terraform could not map a specific section to a specific host in the API. This resulted in false changes if shard names were empty or the host order changed.

v2 uses an associative array where a key is assigned to each host. The provider uses this key to exactly identify a host in state.

You can specify any name for a key. The main principle is name comprehensibility, e.g., "h1", "shard1-rc1a", or "rc1a-fqdn.mdb.yandexcloud.net".

Example

# v1
host {
  type      = "CLICKHOUSE"
  zone      = "ru-central1-a"
  subnet_id = "..."
}
host {
  type      = "CLICKHOUSE"
  zone      = "ru-central1-b"
  subnet_id = "..."
}

# v2
hosts = {
  "h1" = {
    type      = "CLICKHOUSE"
    zone      = "ru-central1-a"
    subnet_id = "..."
  }
  "h2" = {
    type      = "CLICKHOUSE"
    zone      = "ru-central1-b"
    subnet_id = "..."
  }
}

Coordination service hosts

Your ZooKeeper hosts from v1 (type = "ZOOKEEPER") directly migrate to v2: the type does not change, and the zookeeper { } section is transformed into zookeeper = { }. The semantics does not change.

Note

v2 also supports the KEEPER value (for a separate ClickHouse® Keeper) for the type argument, while v1 did not have this value. The type name does not change the configuration structure: the zookeeper = { resources } section is still required. In v2, the filed reference only contains CLICKHOUSE and ZOOKEEPER. It is an error, because KEEPER is also an acceptable value.

The `database` and `user`sections

The database and user sections were deleted from the cluster resource. In v1, you could manage databases and users directly inside the cluster resource. In v2, they have separate resources.

The permission { }, settings { }, and quota { } nested sections migrate to the new yandex_mdb_clickhouse_user resource without any changes and keep their section syntax and fields.

Example

# v1: inline inside yandex_mdb_clickhouse_cluster
database {
  name = "mydb"
}
user {
  name     = "alice"
  password = "secret"
}

# v2: separate resources
resource "yandex_mdb_clickhouse_database" "mydb" {
  cluster_id = yandex_mdb_clickhouse_cluster_v2.main.id
  name       = "mydb"
}

resource "yandex_mdb_clickhouse_user" "alice" {
  cluster_id = yandex_mdb_clickhouse_cluster_v2.main.id
  name       = "alice"
  password   = "secret"
}

The `shard`section

The shard section was replaced with the shards associative array. Like v1, v2 fully supports overriding resources for individual shards. Only the syntax has changed: an associative array is used instead of a section, and resources inside are specified using = { }.

v2 has the disk_size_autoscaling parameter you can specify for each shard. v1 did not support this parameter.

Warning

In v1, you could concurrently specify clickhouse.resources and shard.resources, and shard-level resources took priority. In v2, it is prohibited: clickhouse.resources and shards[*].resources are mutually exclusive. If you set both parameters, the provider will return an error. Only use one-level settings.

The same rule applies to disk_size_autoscaling.

# v1
shard {
  name   = "shard1"
  weight = 100
  resources {
    resource_preset_id = "s2.micro"
    disk_size          = 32
    disk_type_id       = "network-ssd"
  }
}

# v2
shards = {
  "shard1" = {
    weight = 100
    resources = {
      resource_preset_id = "s2.micro"
      disk_size          = 32
      disk_type_id       = "network-ssd"
    }
    # optional: disk_size_autoscaling = { disk_size_limit = 100, ... }
  }
}

If you do not need to override resources for individual shards, do not specify resources:

shards = {
  "shard1" = { weight = 100 }
  "shard2" = { weight = 100 }
}

The `pattern`section

The name of the nested repeat section inside clickhouse.config.graphite_rollup has changed from pattern to patterns. The section contents have not changed.

# v1
clickhouse {
  config {
    graphite_rollup {
      name = "my_rollup"
      pattern {
        regexp    = ".*"
        function  = "any"
        retention { age = 0, precision = 60 }
      }
    }
  }
}

# v2
clickhouse = {
  config = {
    graphite_rollup = [{
      name = "my_rollup"
      patterns = [{
        regexp    = ".*"
        function  = "any"
        retention = [{ age = 0, precision = 60 }]
      }]
    }]
  }
}

The `compression`sections

The repeat compression sections were transformed into a list. In v1, compression comprised a set of repeat sections. In v2, it is ListNestedAttribute, thus you need to use the list syntax. The field set has not changed: method, min_part_size, min_part_size_ratio, and level.

# v1
clickhouse {
  config {
    compression {
      method            = "LZ4"
      min_part_size     = 1073741824
      min_part_size_ratio = 0.01
    }
    compression {
      method            = "ZSTD"
      min_part_size     = 1073741824
      min_part_size_ratio = 0.01
    }
  }
}

# v2
clickhouse = {
  config = {
    compression = [
      { method = "LZ4",  min_part_size = 1073741824, min_part_size_ratio = 0.01 },
      { method = "ZSTD", min_part_size = 1073741824, min_part_size_ratio = 0.01, level = 3 },
    ]
  }
}

Fields

Removed fields

Field v1	Notes
`database { }` (inside the cluster resource)	Replaced by the `yandex_mdb_clickhouse_database` resource.
`user { }` (inside the cluster resource)	Replaced by the `yandex_mdb_clickhouse_user` resource.
`clickhouse.config.kafka_topic { }`	v2 does not support topic-level Apache Kafka® configuration.
`clickhouse.config.mark_cache_size`	In v2, this field was removed.
`clickhouse.config.merge_tree.allow_remote_fs_zero_copy_replication`	In v2, this field was removed.
`status`	The read-only attribute was removed. If your configuration contains `output { value = ...cluster.main.status }`, delete this fragment.
`health`	The read-only attribute was removed. Same as `status`.

Renamed fields

v1	v2	Notes
`clickhouse.config.graphite_rollup[*].pattern`	`clickhouse.config.graphite_rollup[*].patterns`	The name was changed to the plural. The nested object's structure has not changed.

New fields

Warning

You cannot specify both clickhouse.resources and shards[*].resources at the same time. Use either the global clickhouse.resources or shard-level shards[*].resources, but not both at the same time.

The same rule applies to disk_size_autoscaling.

Cluster level

Field v2	Syntax	Description
`allow_host_recreation`	`allow_host_recreation = true`	It allows the provider to recreate hosts when needed, e.g., when changing the disk type. This field is optional; no default value is set.
`restore`	`restore = { backup_id = "..." }`	It allows restoring from a backup when creating a host. The fields include `backup_id` (required), `include_patterns` (optional, a list), and `exclude_patterns` (optional, a list). They are specified using the assignment syntax.
`performance_diagnostics`	`performance_diagnostics = { ... }`	The fields include `enabled` (`bool` type) and `processes_refresh_interval` (`string` type, e.g., `"15s"`). They are specified using the assignment syntax.
`external_dictionary`	`external_dictionary = { "dict_name" = { ... } }`	Associative array of external dictionaries, where the key is the dictionary name. It is specified using the assignment syntax rather than as a section. The supported data sources include `http`, `mysql`, `clickhouse`, `mongodb`, and `postgresql`.
`extension`	`extension { name = "..." version = "..." }`	ClickHouse® cluster extension. It is set as a section. It is only supported in v2; v1 did not have this field.

clickhouse level

Field v2	Notes
`disk_size_autoscaling`	It is set on the same level as the `resources` field rather than inside it. The fields include `disk_size_limit`, `planned_usage_threshold`, and `emergency_usage_threshold`.

zookeeper level

Field v2	Notes
`disk_size_autoscaling`	The structure is the same as the above one. It is set at the same level as `resources`.

shards level

Field v2	Notes
`disk_size_autoscaling`	It is set at the same level as `resources` inside the `shard` element.

clickhouse.config level

Fields v2	Notes
`custom_macros`	List of objects in the `{ name = "...", value = "..." }` format (non-associative array).
`mysql_protocol`	`bool` field that allows using the MySQL® compatibility protocol.
`access_control_improvements`	It is set via assignment: `access_control_improvements = { select_from_system_db_requires_grant = true, select_from_information_schema_requires_grant = true }`.
`async_insert_threads`, `backup_threads`, `restore_threads`	Settings for the number of threads (`Int64`).
`total_memory_tracker_sample_probability`	`Float64`.
`error_log_enabled`, `error_log_retention_size`, `error_log_retention_time`	Separate `bool` and `int64` fields, not a nested section.
`query_metric_log_enabled`, `query_metric_log_retention_size`, `query_metric_log_retention_time`	The same structure is used.
`processors_profile_log_enabled`, `processors_profile_log_retention_size`, `processors_profile_log_retention_time`	The same structure is used. Note: The official guide contains erroneous descriptions of the `processors_profile_log_retention_size` and `processors_profile_log_retention_time` fields. The fields themselves work correctly.
`kafka.batch_size`, `kafka.message_max_bytes`	New fields inside `kafka = { }`.
`merge_tree.materialize_ttl_recalculate_only`	Field inside `merge_tree = { }`.
`merge_tree.deduplicate_merge_projection_mode`, `merge_tree.lightweight_mutation_projection_mode`	`string` listing fields inside `merge_tree`.
`merge_tree.fsync_after_insert`, `merge_tree.fsync_part_directory`	`bool` fields inside `merge_tree`.
`merge_tree.min_rows_to_fsync_after_merge`, `merge_tree.min_compressed_bytes_to_fsync_after_merge`, `merge_tree.min_compressed_bytes_to_fsync_after_fetch`	`Int64` fields inside `merge_tree`.

Complete configuration example: v1 vs. v2

# v1
resource "yandex_mdb_clickhouse_cluster" "main" {
  name        = "my-cluster"
  environment = "PRODUCTION"
  network_id  = yandex_vpc_network.net.id
  version     = "25.3"

  deletion_protection      = false
  copy_schema_on_new_hosts = true
  embedded_keeper          = false
  backup_retain_period_days = 7

  labels = {
    env = "prod"
  }

  clickhouse {
    resources {
      resource_preset_id = "s2.micro"
      disk_type_id       = "network-ssd"
      disk_size          = 32
    }
    config {
      log_level               = "TRACE"
      max_connections         = 100
      max_concurrent_queries  = 50
      timezone                = "UTC"
      merge_tree {
        min_bytes_for_wide_part = 104857600
        ttl_only_drop_parts     = true
      }
    }
  }

  zookeeper {
    resources {
      resource_preset_id = "s2.micro"
      disk_type_id       = "network-ssd"
      disk_size          = 10
    }
  }

  access {
    data_lens     = true
    web_sql       = true
    data_transfer = false
  }

  cloud_storage {
    enabled             = true
    data_cache_enabled  = false
    data_cache_max_size = 0
    move_factor         = 0.01
  }

  maintenance_window {
    type = "WEEKLY"
    day  = "MON"
    hour = 3
  }

  shard {
    name   = "shard1"
    weight = 100
  }
  shard {
    name   = "shard2"
    weight = 100
  }

  host {
    type       = "CLICKHOUSE"
    zone       = "ru-central1-a"
    subnet_id  = yandex_vpc_subnet.a.id
    shard_name = "shard1"
  }
  host {
    type       = "CLICKHOUSE"
    zone       = "ru-central1-b"
    subnet_id  = yandex_vpc_subnet.b.id
    shard_name = "shard2"
  }
  host {
    type      = "ZOOKEEPER"
    zone      = "ru-central1-a"
    subnet_id = yandex_vpc_subnet.a.id
  }
  host {
    type      = "ZOOKEEPER"
    zone      = "ru-central1-b"
    subnet_id = yandex_vpc_subnet.b.id
  }
  host {
    type      = "ZOOKEEPER"
    zone      = "ru-central1-d"
    subnet_id = yandex_vpc_subnet.d.id
  }

  database { name = "mydb" }

  user {
    name     = "alice"
    password = "secret"
    permission {
      database_name = "mydb"
    }
  }

  shard_group {
    name        = "group1"
    description = "first group"
    shard_names = ["shard1", "shard2"]
  }
}

# v2
resource "yandex_mdb_clickhouse_cluster_v2" "main" {
  name        = "my-cluster"
  environment = "PRODUCTION"
  network_id  = yandex_vpc_network.net.id
  version     = "25.3"

  deletion_protection       = false
  copy_schema_on_new_hosts  = true
  embedded_keeper           = false
  backup_retain_period_days = 7

  labels = {
    env = "prod"
  }

  clickhouse = {
    resources = {
      resource_preset_id = "s2.micro"
      disk_type_id       = "network-ssd"
      disk_size          = 32
    }
    config = {
      log_level              = "TRACE"
      max_connections        = 100
      max_concurrent_queries = 50
      timezone               = "UTC"
      merge_tree = {
        min_bytes_for_wide_part = 104857600
        ttl_only_drop_parts     = true
      }
    }
  }

  zookeeper = {
    resources = {
      resource_preset_id = "s2.micro"
      disk_type_id       = "network-ssd"
      disk_size          = 10
    }
  }

  access = {
    data_lens     = true
    web_sql       = true
    data_transfer = false
  }

  cloud_storage = {
    enabled             = true
    data_cache_enabled  = false
    data_cache_max_size = 0
    move_factor         = 0.01
  }

  maintenance_window {
    type = "WEEKLY"
    day  = "MON"
    hour = 3
  }

  shards = {
    "shard1" = { weight = 100 }
    "shard2" = { weight = 100 }
  }

  # Hosts: named map instead of repeated blocks
  hosts = {
    "c1" = {
      type       = "CLICKHOUSE"
      zone       = "ru-central1-a"
      subnet_id  = yandex_vpc_subnet.a.id
      shard_name = "shard1"
    }
    "c2" = {
      type       = "CLICKHOUSE"
      zone       = "ru-central1-b"
      subnet_id  = yandex_vpc_subnet.b.id
      shard_name = "shard2"
    }
    "k1" = {
      type      = "ZOOKEEPER"
      zone      = "ru-central1-a"
      subnet_id = yandex_vpc_subnet.a.id
    }
    "k2" = {
      type      = "ZOOKEEPER"
      zone      = "ru-central1-b"
      subnet_id = yandex_vpc_subnet.b.id
    }
    "k3" = {
      type      = "ZOOKEEPER"
      zone      = "ru-central1-d"
      subnet_id = yandex_vpc_subnet.d.id
    }
  }

  # extension is v2-only, uses block syntax.
  # This section did not exist in v1: remove it for a no-op migration.
  extension {
    name    = "geodb"
    version = "2024.07.03-210328"
  }

  # shard_group stays as block syntax
  shard_group {
    name        = "group1"
    description = "first group"
    shard_names = ["shard1", "shard2"]
  }
}

# Databases: now separate resources
resource "yandex_mdb_clickhouse_database" "mydb" {
  cluster_id = yandex_mdb_clickhouse_cluster_v2.main.id
  name       = "mydb"
}

# Users: now separate resources
resource "yandex_mdb_clickhouse_user" "alice" {
  cluster_id = yandex_mdb_clickhouse_cluster_v2.main.id
  name       = "alice"
  password   = "secret"
  permission {
    database_name = "mydb"
  }
}

Migration errors

Blocks of type "X" are not expected here

│ Error: Unsupported block type
│   Blocks of type "access" are not expected here. Did you mean to define argument "access"?
│   If so, use the equals sign to assign it a value.

To fix the error, add = after access, i.e., access = { ... }.

Invalid field name.

The name is missing in clickhouse.config or was removed. For more information, see the field reference above.
Invalid type.

The field exists but has an invalid type, e.g., a string instead of an integer. You can find the required data type in the field reference.
Not supported in this ClickHouse® version.

Some fields are available starting from a certain version. You can find the version requirements in this provider guide.
Provider-side mapping error.

Sometimes the field is present in the schema but the provider does not process it. If the field name and type are valid, create a task in the provider's repository.

Useful links

Migrating a cluster from v1 to v2

Updates in a v2 cluster

List of updates in v2

Unchanged attributes

Nested sections

The `host`section

Coordination service hosts

The `database` and `user`sections

The `shard`section

The `pattern`section

The `compression`sections

Fields

Removed fields

Renamed fields

New fields

Cluster level

clickhouse level

zookeeper level

shards level

clickhouse.config level

Complete configuration example: v1 vs. v2

Migration errors

Blocks of type "X" are not expected here

Blocks of type "host" are not expected here

Error: Argument or block definition required (for shards)

Error protobuf filler: Attribute X is not mapped

Useful links

Was the article helpful?

Updates in a v2 cluster

List of updates in v2List of updates in v2

Unchanged attributesUnchanged attributes

Nested sectionsNested sections

The sectionThe hostsection

Coordination service hostsCoordination service hosts

The and sectionsThe database and usersections

The sectionThe shardsection

The sectionThe patternsection

The sectionsThe compressionsections

FieldsFields

Removed fieldsRemoved fields

Renamed fieldsRenamed fields

New fieldsNew fields

Cluster levelCluster level

clickhouse levelclickhouse level

zookeeper levelzookeeper level

shards levelshards level

clickhouse.config levelclickhouse.config level

Complete configuration example: v1 vs. v2Complete configuration example: v1 vs. v2

Migration errorsMigration errors

Blocks of type "X" are not expected hereBlocks of type "X" are not expected here

Blocks of type "host" are not expected hereBlocks of type "host" are not expected here

Error: Argument or block definition required (for shards)Error: Argument or block definition required (for shards)

Error protobuf filler: Attribute X is not mappedError protobuf filler: Attribute X is not mapped

Useful linksUseful links

Was the article helpful?

List of updates in v2

Unchanged attributes

Nested sections

The `host`section

Coordination service hosts

The `database` and `user`sections

The `shard`section

The `pattern`section

The `compression`sections

Fields

Removed fields

Renamed fields

New fields

Cluster level

clickhouse level

zookeeper level

shards level

clickhouse.config level

Complete configuration example: v1 vs. v2

Migration errors

Blocks of type "X" are not expected here

Blocks of type "host" are not expected here

Error: Argument or block definition required (for shards)

Error protobuf filler: Attribute X is not mapped

Useful links