Managing Yandex StoreDoc shards
You can create a sharded cluster or enable sharding later. After that, you can add and configure shards.
Make sure your shards consist of at least three MONGOD hosts for higher availability. We do not recommend sharding small collections: query processing is faster with a standard replica cluster.
Alert
After cluster sharding is enabled:
- There is no way to disable sharding. The cluster will always maintain a minimum number of
MONGOS,MONGOCFG, orMONGOINFRAhosts depending on the sharding type. - To access databases, use only the
MONGOSorMONGOINFRAhosts that route queries to shards. For this purpose, edit host addresses in your application code accordingly.
Enabling sharding
The Yandex StoreDoc interface lets you quickly create a Yandex StoreDoc sharding infrastructure.
To learn how to directly shard your Yandex StoreDoc database and collections, see Sharding collections.
To enable sharding, you need:
- At least 3
MONGOINFRAhosts for standard sharding. - At least 2
MONGOSand 3MONGOCFGhosts for advanced sharding.
Note
Sharding is not supported for hosts of b1.medium and b2.medium classes. If you do not see the Shards tab, upgrade the cluster host class to the supported value.
-
Navigate to the folder dashboard
and select Yandex StoreDoc. -
Click the cluster name and select the Shards tab.
-
Click Enable.
-
Select a sharding type:
-
Standard: Based on
MONGOINFRAhosts. -
Advanced: Based on
MONGOSandMONGOCFGhosts.For more information, see Sharding.
Warning
After you enable sharding, you can't change its type.
-
-
Set the parameters of the hosts that will provide access to the sharded data.
-
Click Enable sharding.
The cluster will start updating, with the requested hosts and first shard of the cluster created.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
-
To enable standard cluster sharding using
MONGOINFRAhosts, run this command (the list of supported parameters in the example is not exhaustive):yc managed-mongodb cluster enable-sharding \ --cluster-name=<cluster_name> \ --host type=mongoinfra,` `zone-id=<availability_zone>,` `subnet-name=<subnet_name> \ --host type=mongoinfra,` `zone-id=<availability_zone>,` `subnet-name=<subnet_name> \ --host type=mongoinfra,` `zone-id=<availability_zone>,` `subnet-name=<subnet_name> \ --mongoinfra resource-preset=<host_class>,` `disk-size=<storage_size_in_GB>,` `disk-type=<disk_type>Where:
--cluster-name: Cluster name you can get with a list of clusters in the folder.--host: Host settings:type: Type (MONGOINFRA).zone-id: Availability zone.subnet-name: Subnet name.
--mongoinfra:MONGOINFRAhost parameters:resource-preset: Host class.disk-size: Storage size, in GB.disk-type: Disk type.
-
To enable advanced cluster sharding using
MONGOSandMONGOCFGhosts, run this command (the list of supported parameters in the example is not exhaustive):yc managed-mongodb cluster enable-sharding \ --cluster-name=<cluster_name> \ --host type=mongos,` `zone-id=<availability_zone>,` `subnet-name=<subnet_name> \ --host type=mongos,` `zone-id=<availability_zone>,` `subnet-name=<subnet_name> \ --mongos resource-preset=<host_class>,` `disk-size=<storage_size_in_GB>,` `disk-type=<disk_type> \ --host type=mongocfg,` `zone-id=<availability_zone>,` `subnet-name=<subnet_name> \ --host type=mongocfg,` `zone-id=<availability_zone>,` `subnet-name=<subnet_name> \ --host type=mongocfg,` `zone-id=<availability_zone>,` `subnet-name=<subnet_name> \ --mongocfg resource-preset=<host_class>,` `disk-size=<storage_size_in_GB>,` `disk-type=<disk_type>Where:
--cluster-name: Cluster name you can get with a list of clusters in the folder.--host: Host settings:type: Type (MONGOSorMONGOCFG).zone-id: Availability zone.subnet-name: Subnet name.
--mongos:MONGOShost parameters:resource-preset: Host class.disk-size: Storage size, in GB.disk-type: Disk type.
--mongocfg:MONGOCFGhost parameters:resource-preset: Host class.disk-size: Storage size, in GB.disk-type: Disk type.
-
Make sure the provider version you are using is 0.90 or higher:
terraform versionResult example:
Terraform v1.4.6 on darwin_arm64 + provider registry.terraform.io/yandex-cloud/yandex v0.91.0If the version is older
Update the provider version:
-
In the folder with the
.tfconfiguration file, add the credentials to environment variables. -
In the same folder, initialize the provider with the updated version specified in the configuration file:
terraform init -upgrade -
Check the provider version:
terraform version
-
-
Open the current Terraform configuration file describing your infrastructure.
For information on how to create this file, see Creating a cluster.
-
Add additional resources to the configuration file.
For standard sharding of the cluster using
MONGOINFRAhostsresources_mongoinfra { resource_preset_id = "<host_class>" disk_type_id = "<disk_type>" disk_size = <storage_size_in_GB> } host { zone_id = "<availability_zone>" subnet_id = "<subnet_ID>" type = "mongoinfra" } host { zone_id = "<availability_zone>" subnet_id = "<subnet_ID>" type = "mongoinfra" } host { zone_id = "<availability_zone>" subnet_id = "<subnet_ID>" type = "mongoinfra" }For advanced sharding of the cluster using
MONGOSandMONGOCFGhostsresources_mongos { resource_preset_id = "<host_class>" disk_type_id = "<disk_type>" disk_size = <storage_size_in_GB> } resources_mongocfg { resource_preset_id = "<host_class>" disk_type_id = "<disk_type>" disk_size = <storage_size_in_GB> } host { zone_id = "<availability_zone>" subnet_id = "<subnet_ID>" type = "mongos" } host { zone_id = "<availability_zone>" subnet_id = "<subnet_ID>" type = "mongos" } host { zone_id = "<availability_zone>" subnet_id = "<subnet_ID>" type = "mongocfg" } host { zone_id = "<availability_zone>" subnet_id = "<subnet_ID>" type = "mongocfg" } host { zone_id = "<availability_zone>" subnet_id = "<subnet_ID>" type = "mongocfg" } -
Validate your configuration.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider guide.
-
Get an IAM token for API authentication and place it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Cluster.EnableSharding method send the following request, e.g., via cURL
:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-mongodb/v1/clusters/<cluster_ID>:enableSharding' \ --data '{ "<Yandex_StoreDoc_host_type>": { "resources": { "resourcePresetId": "<host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" } }, "hostSpecs": [ { "zoneId": "<availability_zone>", "subnetId": "<subnet_ID>", "assignPublicIp": <allow_public_access_to_host>, "type": "<host_type>", "shardName": "<shard_name>", "hidden": <hide_host>, "secondaryDelaySecs": "<lag_in_seconds>", "priority": "<host_priority_for_assignment_as_master>", "tags": "<host_labels>" }, { <similar_settings_for_host_2> }, { ... }, { <similar_configuration_for_host_N> } ] }'Where:
-
Yandex StoreDoc host type depends on the sharding type. Possible values:
mongocfg,mongos, andmongoinfra. -
hostSpecsis an array of new hosts. One array element contains settings for a single host. The number of hosts depends on the sharding type.zoneId: Availability zone.subnetId: Subnet ID.assignPublicIp: Internet access to the host via a public IP address,trueorfalse.type: Host type:MONGOINFRA,MONGOS, orMONGOCFG.shardName: Shard name.hidden: The host will either be visible (false) or hidden (true).secondaryDelaySecs: Host's lag behind the master.priority: Host priority for assignment as a master if the primary master fails.tags: Host labels.
You can get the cluster ID from the folder’s cluster list.
-
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and place it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService.EnableSharding call and send a request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/mongodb/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "<Yandex_StoreDoc_host_type>": { "resources": { "resource_preset_id": "<host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" } }, "host_specs": [ { "zone_id": "<availability_zone>", "subnet_id": "<subnet_ID>", "assign_public_ip": <allow_public_access_to_host>, "type": "<host_type>", "shard_name": "<shard_name>", "hidden": <hide_host>, "secondary_delay_secs": "<lag_in_seconds>", "priority": "<host_priority_for_assignment_as_master>", "tags": "<host_labels>" }, { <similar_settings_for_host_2> }, { ... }, { <similar_configuration_for_host_N> } ] }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.mongodb.v1.ClusterService.EnableShardingWhere:
-
Yandex StoreDoc host type depends on the sharding type. Possible values:
mongocfg,mongos, andmongoinfra. -
host_specsis an array of new hosts. One array element contains settings for a single host. The number of hosts depends on the sharding type.zone_id: Availability zone.subnet_id: Subnet ID.assign_public_ip: Internet access to the host via a public IP address,trueorfalse.type: Host type:MONGOINFRA,MONGOS, orMONGOCFG.shard_name: Shard name.hidden: The host will either be visible (false) or hidden (true).secondary_delay_secs: Host's lag behind the master.priority: Host priority for assignment as a master if the primary master fails.tags: Host labels.
You can get the cluster ID from the folder’s cluster list.
-
-
Check the server response to make sure your request was successful.
Getting a list of shards in a cluster
- Navigate to the folder dashboard
and select Yandex StoreDoc. - Click the cluster name and select the Shards tab.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To get a list of shards in a cluster, run the following command:
yc managed-mongodb shards list --cluster-name <cluster_name>
Result:
+------+
| NAME |
+------+
| rs01 |
| rs02 |
+------+
You can get the cluster name from the folder’s cluster list.
-
Get an IAM token for API authentication and place it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Cluster.ListShards method and send the following request, e.g., via cURL
:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-mongodb/v1/clusters/<cluster_ID>/shards'You can get the cluster ID from the folder’s cluster list.
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and place it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService.ListShards call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/mongodb/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.mongodb.v1.ClusterService.ListShardsYou can get the cluster ID from the folder’s cluster list.
-
Check the server response to make sure your request was successful.
Creating a shard
The number of shards in Yandex StoreDoc clusters is limited by the CPU and RAM quotas available to DB clusters in your cloud. To check the resources currently in use, open the Quotas
- Navigate to the folder dashboard
and select Yandex StoreDoc. - Click the cluster name and select the Shards tab.
- Click Create shard.
- Enter a name for the shard and add the number of hosts you need.
- Click Create shard.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To create a shard, run the command below (our example does not use all available parameters):
yc managed-mongodb shards add <new_shard_name> \
--cluster-name=<cluster_name> \
--host zone-id=<availability_zone>,`
`subnet-name=<subnet_name>
Where:
--cluster-name: Cluster name you can get with a list of clusters in the folder.--host: Host settings:zone-id: Availability zone.subnet-name: Subnet name.
-
Make sure the provider version you are using is 0.90 or higher:
terraform versionResult example:
Terraform v1.4.6 on darwin_arm64 + provider registry.terraform.io/yandex-cloud/yandex v0.91.0If the version is older
Update the provider version:
-
In the folder with the
.tfconfiguration file, add the credentials to environment variables. -
In the same folder, initialize the provider with the updated version specified in the configuration file:
terraform init -upgrade -
Check the provider version:
terraform version
-
-
Open the current Terraform configuration file describing your infrastructure.
For information on how to create this file, see Creating a cluster.
-
Add to the Yandex StoreDoc cluster description the required number of
hostblocks withMONGODin thetypeparameter and shard name in theshard_nameparameter:resource "yandex_mdb_mongodb_cluster" "<cluster_name>" { ... host { zone_id = "<availability_zone>" subnet_id = "<subnet_ID>" type = "mongod" shard_name = "<shard_name>" } } -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider guide.
-
Get an IAM token for API authentication and place it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Cluster.AddShard method and make a request, e.g., via cURL
:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-mongodb/v1/clusters/<cluster_ID>/shards' \ --data '{ "shardName": "<shard_name>", "hostSpecs": [ { "zoneId": "<availability_zone>", "subnetId": "<subnet_ID>", "assignPublicIp": <allow_public_access_to_host>, "type": "<host_type>", "shardName": "<shard_name>", "hidden": <hide_host>, "secondaryDelaySecs": "<time_in_seconds>", "priority": "<host_priority_for_assignment_as_master>", "tags": "<labels>" }, { <similar_settings_for_host_2> }, { ... }, { <similar_configuration_for_host_N> } ] }'Where:
-
shardName: Name of the shard being created. -
hostSpecs: Host settings:zoneId: Availability zone.subnetId: Subnet ID.assignPublicIp: Internet access to the host via a public IP address,trueorfalse.type: Host type. SpecifyMONGOD.shardName: Shard name.hidden: The host will either be visible (false) or hidden (true).secondaryDelaySecs: Host's lag behind the master.priority: Host priority for assignment as a master if the primary master fails.tags: Host labels.
You can get the cluster ID from the folder’s cluster list.
-
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and place it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService.AddShard call and make a request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/mongodb/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "shard_name": "<shard_name>", "host_specs": [ { "zone_id": "<availability_zone>", "subnet_id": "<subnet_ID>", "assign_public_ip": <allow_public_access_to_host>, "type": "<host_type>", "shard_name": "<shard_name>", "hidden": <hide_host>, "secondary_delay_secs": "<time_in_seconds>", "priority": "<host_priority_for_assignment_as_master>", "tags": "<labels>" }, { <similar_settings_for_host_2> }, { ... }, { <similar_configuration_for_host_N> } ] }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.redis.v1.ClusterService.AddShardWhere:
-
shard_name: Name of the shard being created. -
host_specs: Host settings:zone_id: Availability zone.subnet_id: Subnet ID.assign_public_ip: Internet access to the host via a public IP address,trueorfalse.type: Host type. SpecifyMONGOD.shard_name: Shard name.hidden: The host will either be visible (false) or hidden (true).secondary_delay_secs: Host's lag behind the master.priority: Host priority for assignment as a master if the primary master fails.tags: Host labels.
You can get the cluster ID from the folder’s cluster list.
-
-
Check the server response to make sure your request was successful.
Deleting a shard
You can delete a shard from a Yandex StoreDoc cluster if it is not the only shard there. To replace the only shard in a cluster, first create a new shard and then remove the old one.
Note
The removeShard operation will be called for the shard being deleted and will safely transfer data to the remaining shards.
- Navigate to the folder dashboard
and select Yandex StoreDoc. - Click the cluster name and select the Shards tab.
- Click the
icon in the shard row and select Delete. - In the window that opens, click Remove.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To delete a shard from a cluster, run this command:
yc managed-mongodb shards delete <shard_name> \
--cluster-name=<cluster_name>
You can get the shard name with the list of shards in the cluster, and the cluster name, with the list of clusters in the folder.
-
Open the current Terraform configuration file describing your infrastructure.
For information on how to create this file, see Creating a cluster.
-
Delete all shard-related
hostblocks from the Yandex StoreDoc cluster description. -
Validate your configuration.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Type
yesand press Enter.-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider guide.
-
Get an IAM token for API authentication and place it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Cluster.DeleteShard method and send the following request, e.g., via cURL
:curl \ --request DELETE \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-mongodb/v1/clusters/<cluster_ID>/shards/<shard_name>'You can request the cluster ID with the list of clusters in the folder, and the shard name, with the list of shards in the cluster.
-
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and place it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService.DeleteShard call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/mongodb/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "shard_name": "<shard_name>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.mongodb.v1.ClusterService.DeleteShardYou can request the cluster ID with the list of clusters in the folder and the shard name, with the list of shards in the cluster.
-
View the server response to make sure your request was successful.