Managing cluster shards
You can add and remove cluster shards, request the list of shards in a selected cluster, and rebalance your cluster.
Warning
You can only manage shards in sharded clusters.
You can create a sharded cluster or enable sharding in an existing non-sharded cluster.
Getting a list of shards in a cluster
- In the management console
, select the folder containing your cluster. - Navigate to the Yandex Managed Service for Valkey™ service.
- Click the name of your cluster and select the Shards tab.
If you do not have the Yandex Cloud CLI yet, install and initialize it.
The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id options.
To get a list of cluster shards, run this command:
yc managed-redis shards list --cluster-name <cluster_name>
Result:
+--------------+
| NAME |
+--------------+
| test-shard-1 |
| test-shard-2 |
| test-shard-3 |
+--------------+
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Cluster.ListShards method, e.g., via the following cURL
request:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-redis/v1/clusters/<cluster_ID>/shards'You can get the cluster ID with the list of clusters in the folder.
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume that the repository contents reside in the
~/cloudapi/directory. -
Call the ClusterService.ListShards method, e.g., via the following gRPCurl
request:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/redis/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.redis.v1.ClusterService.ListShardsYou can get the cluster ID with the list of clusters in the folder.
-
Check the server response to make sure your request was successful.
Getting detailed information about a shard
If you do not have the Yandex Cloud CLI yet, install and initialize it.
The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id options.
To get information about a shard, run the following command:
yc managed-redis shards get <shard_name> --cluster-name <cluster_name>
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Cluster.GetShard method, e.g., via the following cURL
request:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-redis/v1/clusters/<cluster_ID>/shards/<shard_name>'You can get the cluster ID with the list of clusters in the folder, and the shard name, with the list of shards in the cluster.
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume that the repository contents reside in the
~/cloudapi/directory. -
Call the ClusterService.GetShard method, e.g., via the following gRPCurl
request:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/redis/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "shard_name": "<shard_name>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.redis.v1.ClusterService.GetShardYou can get the cluster ID with the list of clusters in the folder, and the shard name, with the list of shards in the cluster.
-
Check the server response to make sure your request was successful.
You can get the shard name with the list of shards in the cluster, and the cluster name, with the list of clusters in the folder.
Creating a shard
For clusters with the local-ssd disk type, the minimum allowed number of hosts per shard is two.
To add a shard:
- Navigate to the Yandex Managed Service for Valkey™ service.
- Click the cluster name and navigate to the Shards tab.
- Click Create shard.
- Specify Shard name.
- Under Hosts:
- Optionally, edit the host settings.
- Click Add host.
- In the Availability zone field, select the availability zone and Subnet, and enable Public access.
- Click Create shard.
If you do not have the Yandex Cloud CLI yet, install and initialize it.
The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id options.
See the description of the CLI command for adding a shard:
yc managed-redis shards add --help
To create a shard of two hosts, one with public access and the other with a host priority of 50, run this command:
yc managed-redis shards add --name=<new_shard_name> \
--cluster-name=<cluster_name> \
--host zone-id=<availability_zone>,`
`subnet-name=<subnet_name>,`
`assign-public-ip=true \
--host zone-id=<availability_zone>,`
`subnet-name=<subnet_name>,`
`replica-priority=50
To add a shard:
-
Open the current Terraform configuration file describing your infrastructure.
For more on how to create this file, see Creating a cluster.
-
In the
hostssection, add the required number of hosts to the Yandex Managed Service for Valkey™ cluster description and specify the shard name in theshard_nameargument:resource "yandex_mdb_redis_cluster_v2" "<cluster_name>" { ... hosts = { ... "<host_name>" = { zone = "<availability_zone>" subnet_id = <subnet_ID> assign_public_ip = <public_access> replica_priority = <host_priority> shard_name = "<shard_name>" } } }Where:
zone_id: Availability zone.subnet_id: ID of the subnet in the selected availability zone.assign_public_ip: Public access to the host,trueorfalse.replica_priority: Host priority for master promotion during failover.shard_name: Name of the shard the host will reside in.
The host name must be unique within the cluster.
-
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider guide.
Timeouts
The Terraform provider sets the following timeouts for Yandex Managed Service for Valkey™ cluster operations:
- Creating a cluster, including by restoring it from a backup: 15 minutes.
- Editing a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the timeout are aborted.
How do I change these limits?
Add the timeouts section to your cluster description, such as the following:
resource "yandex_mdb_redis_cluster_v2" "<cluster_name>" {
...
timeouts = {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Cluster.AddShard method, e.g., via the following cURL
request:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-redis/v1/clusters/<cluster_ID>/shards' \ --data '{ "shardName": "<shard_name>", "hostSpecs": [ { "zoneId": "<availability_zone>", "subnetId": "<subnet_ID>", "shardName": "<shard_name>", "replicaPriority": "<host_priority>", "assignPublicIp": <public_access_to_cluster_host> }, { <similar_settings_for_host_2> }, { ... }, { <similar_settings_for_host_N> } ] }'Where:
-
shardName: New shard’s name. -
hostSpecs: Host settings:zoneId: Availability zone.subnetId: Subnet ID. You must specify this setting if the selected availability zone has more than one subnet.shardName: Shard name for the host.replicaPriority: Host priority for master promotion during failover.assignPublicIp: Host accessibility from the internet via a public IP address,trueorfalse. You can only enable public access if your cluster supports TLS.
You can get the cluster ID with the list of clusters in the folder.
-
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume that the repository contents reside in the
~/cloudapi/directory. -
Call the ClusterService.AddShard method, e.g., via the following gRPCurl
request:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/redis/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "shard_name": "<shard_name>", "host_specs": [ { "zone_id": "<availability_zone>", "subnet_id": "<subnet_ID>", "shard_name": "<shard_name>", "replica_priority": "<host_priority>", "assign_public_ip": <public_access_to_cluster_host> }, { <similar_settings_for_host_2> }, { ... }, { <similar_settings_for_host_N> } ] }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.redis.v1.ClusterService.AddShardWhere:
-
shard_name: New shard’s name. -
host_specs: Host settings:zone_id: Availability zone.subnet_id: Subnet ID. You must specify this setting if the selected availability zone has more than one subnet.shard_name: Shard name for the host.replica_priority: Host priority for master promotion during failover.assign_public_ip: Host accessibility from the internet via a public IP address,trueorfalse. You can only enable public access if your cluster supports TLS.
You can get the cluster ID with the list of clusters in the folder.
-
To enable data placement on the new shard, run Yandex Managed Service for Valkey™ cluster rebalancing.
Deleting a shard
Alert
- All the shard hosts are deleted with the shard.
- If a cluster has two or three shards, use the CLI, Terraform, or API to delete a shard.
- In the management console
, go to the folder containing the cluster to delete the shard from. - Navigate to the Yandex Managed Service for Valkey™ service.
- Click the name of your cluster and select the Shards tab.
- Next to the shard in question, click
and select Delete. - In the window that opens, click Remove.
If you do not have the Yandex Cloud CLI yet, install and initialize it.
The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id options.
To delete a shard from a cluster, run this command:
yc managed-redis shards delete <shard_name> \
--cluster-name=<cluster_name>
You can get the shard name from the list of cluster shards, and the cluster name from the list of clusters in your folder.
-
Open the current Terraform configuration file describing your infrastructure.
For more on how to create this file, see Creating a cluster.
-
Delete all shard-related hosts in the
hostssection from the Yandex Managed Service for Valkey™ cluster description. -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Type
yesand press Enter.-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider guide.
Timeouts
The Terraform provider sets the following timeouts for Yandex Managed Service for Valkey™ cluster operations:
- Creating a cluster, including by restoring it from a backup: 15 minutes.
- Editing a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the timeout are aborted.
How do I change these limits?
Add the timeouts section to your cluster description, such as the following:
resource "yandex_mdb_redis_cluster_v2" "<cluster_name>" {
...
timeouts = {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Cluster.DeleteShard method, e.g., via the following cURL
request:curl \ --request DELETE \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-redis/v1/clusters/<cluster_ID>/shards/<shard_name>'You can get the cluster ID with the list of clusters in the folder, and the shard name, with the list of shards in the cluster.
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume that the repository contents reside in the
~/cloudapi/directory. -
Call the ClusterService.DeleteShard method, e.g., via the following gRPCurl
request:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/redis/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "shard_name": "<shard_name>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.redis.v1.ClusterService.DeleteShardYou can get the cluster ID with the list of clusters in the folder, and the shard name, with the list of shards in the cluster.
-
Check the server response to make sure your request was successful.
Rebalancing a cluster
New shards come with no hash slots and cannot store any data. To enable placing data on a new shard, rebalance the cluster to allocate some of its hash slots to that shard. The cluster will move the data in the reassigned hash slots to the matching shard. You can run a rebalance on a running cluster without affecting the availability or integrity of its data.
For more information, see Scalability.
To rebalance a cluster:
- In the management console
, select the folder with your cluster. - Navigate to the Yandex Managed Service for Valkey™ service.
- Click the cluster name.
- In the Overview tab, click Rebalance.
Tip
You can also rebalance a cluster by clicking Rebalance cluster on the Shards tab.
If you do not have the Yandex Cloud CLI yet, install and initialize it.
The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id options.
To rebalance a cluster, run this command:
yc managed-redis cluster rebalance \
--name=<cluster_name>
You can get the cluster name with the list of clusters in the folder.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Cluster.Rebalance method, e.g., via the following cURL
request:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-redis/v1/clusters/<cluster_ID>:rebalance'You can get the cluster ID with the list of clusters in the folder.
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume that the repository contents reside in the
~/cloudapi/directory. -
Call the ClusterService.Rebalance method, e.g., via the following gRPCurl
request:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/redis/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.redis.v1.ClusterService.RebalanceYou can get the cluster ID with the list of clusters in the folder.
-
Check the server response to make sure your request was successful.