Managing shards in a ClickHouse® cluster
You can group several shards of a ClickHouse® cluster in a shard group and save tables in this group.
Listing shard groups in a cluster
- In the management console
, go to the folder page and select Managed Service for ClickHouse. - Click the cluster name and open the Shard groups tab.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To get a list of shard groups in a cluster, run the command:
yc managed-clickhouse shard-groups list \
--cluster-name=<cluster_name>
You can request the cluster name with a list of clusters in the folder.
To get a list of shard groups in a cluster, use the listShardGroups REST API method for the Cluster resource or the ClusterService/ListShardGroups gRPC API call and provide the cluster ID in the clusterId
request parameter.
To find out the cluster ID, get a list of clusters in the folder.
Viewing detailed information about a shard group
- In the management console
, go to the folder page and select Managed Service for ClickHouse. - Click the cluster name and open the Shard groups tab.
- Select a shard group to view detailed information.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To view detailed information about a shard group in a cluster, run the command:
yc managed-clickhouse shard-groups get \
--cluster-name=<cluster_name> \
--name=<shard_group_name>
You can request the cluster name with a list of clusters in the folder.
To get detailed information about a shard group, use the getShardGroup REST API method for the Cluster resource or the ClusterService/GetShardGroup gRPC API call and provide the following in the request:
- Cluster ID, in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - Name of the shard group, in the
shardGroupName
parameter. To find out the name, get a list of shard groups in the cluster.
Creating a shard group
- In the management console
, go to the folder page and select Managed Service for ClickHouse. - Click the cluster name and open the Shard groups tab.
- Click Create shard group.
- Fill in the form fields and click Apply.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To create a shard group in a cluster, run the command:
yc managed-clickhouse shard-groups create \
--cluster-name=<cluster_name> \
--name=<shard_group_name> \
--description=<shard_group_description> \
--shards=<list_of_shard_names>
Where --shards
is a list of shard names to be included in the group.
You can request the cluster name with a list of clusters in the folder.
You can request a shard name with a list of shards in the cluster.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
Add a
shard_group
description block to the Managed Service for ClickHouse® cluster description.resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" { ... shard_group { name = "<shard_group_name>" description = "<optional_shard_group_description>" shard_names = [ # List of shards in the group "<name_of_shard_1>", ... "<name_of_shard_N>" ] } }
-
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
A Terraform provider sets the timeout for Managed Service for ClickHouse® cluster operations:
- Creating a cluster, including by restoring one from a backup: 60 minutes.
- Editing a cluster: 90 minutes.
- Deleting a cluster: 30 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To create a shard group, use the createShardGroup REST API method for the Cluster resource or the ClusterService/CreateShardGroup gRPC API call and provide the following in the request:
- ID of the cluster in which you want to create a group, in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - Name of the shard group, in the
shardGroupName
parameter. - List of shard names be included in the group, in the
shardNames
parameter. To find out the names, get a list of shards in the cluster. - Description of the shard group, in the
description
parameter, if required.
Changing a shard group
- In the management console
, go to the folder page and select Managed Service for ClickHouse. - Click the cluster name and open the Shard groups tab.
- Click
for the required shard group and select Edit.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To change a shard group in a cluster, run the command:
yc managed-clickhouse shard-groups update \
--cluster-name=<cluster_name> \
--name=<shard_group_name> \
--description=<new_shard_group_description> \
--shards=<new_list_of_shard_names>
Where --shards
is a new list of shard names be included in the group.
This command replaces the existing list of shards in the group with the new one that was provided to the command in the --shards
parameter. Before running the command, make sure that you added all the appropriate shards in the new list.
You can request the cluster name with a list of clusters in the folder.
You can request the name of the shard group with a list of shard groups in the cluster.
You can request a shard name with a list of shards in the cluster.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
In the Managed Service for ClickHouse® cluster description, change the
shard_group
block to the required shard group:resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" { ... shard_group { name = "<new_shard_group_name>" description = "<new_shard_group_description>" shard_names = [ # New list of shards in the group "<name_of_shard_1>", ... "<name_of_shard_N>" ] } }
-
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
A Terraform provider sets the timeout for Managed Service for ClickHouse® cluster operations:
- Creating a cluster, including by restoring one from a backup: 60 minutes.
- Editing a cluster: 90 minutes.
- Deleting a cluster: 30 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To update a shard group, use the updateShardGroup REST API method for the Cluster resource or the ClusterService/UpdateShardGroup gRPC API call and provide the following in the request:
- ID of the cluster in which you want to change a group, in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - Name of the shard group, in the
shardGroupName
parameter. To find out the name, get a list of shard groups in the cluster. - If necessary, a new description of the shard group, in the
description
parameter. - If necessary, a new list of shard names to be included in the group, in the
shardNames
parameter. To find out the names, get a list of shards in the cluster. This list will replace the current one, so make sure that you added all the appropriate shards in the new list. - Names of parameters to change, in the
updateMask
parameter.
Warning
This API method overrides all parameters of the object being modified that were not explicitly passed in the request to the default values. To avoid this, list the settings you want to change in the updateMask
parameter (one line separated by commas).
Deleting a group of shards
Deleting a group of shards doesn't affect the shards in the group: they are kept in the cluster.
Tables created on the deleted group are kept, but they are disabled and attempts to query them result in errors. However, you can delete these tables before or after you delete the shard group.
- In the management console
, go to the folder page and select Managed Service for ClickHouse. - Click the cluster name and open the Shard groups tab.
- Click
for the required shard group and select Delete.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To delete a shard group in a cluster, run the command:
yc managed-clickhouse shard-groups delete \
--cluster-name=<cluster_name> \
--name=<shard_group_name>
You can request the cluster name with a list of clusters in the folder.
You can request the name of the shard group with a list of shard groups in the cluster.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
Delete the
shard_group
block describing the required group from the Managed Service for ClickHouse® cluster description. -
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Type
yes
and press Enter.-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
A Terraform provider sets the timeout for Managed Service for ClickHouse® cluster operations:
- Creating a cluster, including by restoring one from a backup: 60 minutes.
- Editing a cluster: 90 minutes.
- Deleting a cluster: 30 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To delete a shard group, use the deleteShardGroup REST API method for the Cluster resource or the ClusterService/DeleteShardGroup gRPC API call and provide the following in the request:
- ID of the cluster you want to delete a group from, in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - Name of the shard group, in the
shardGroupName
parameter. To find out the name, get a list of shard groups in the cluster.
ClickHouse® is a registered trademark of ClickHouse, Inc