Creating an instance group connected to a file storage
One of the ways to handle stateful workloads is to store the application state in a file storage independent of the instance group.
To create an instance group that will automatically connect a common file storage to each of its instances:
-
By default, all operations in Instance Groups are performed on behalf of a service account. If you don't have a service account, create one.
-
If you do not have a file storage, create one.
-
To be able to create, update, and delete VMs in the group, assign the compute.editor role to the service account.
-
Create an instance group:
CLITerraformAPIIf you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the
yc config set folder-id <folder_ID>command. You can also set a different folder for any specific command using the--folder-nameor--folder-idparameter.-
See the description of the CLI command for creating an instance group:
yc compute instance-group create --help -
Check whether the folder contains any networks:
yc vpc network listIf there are no networks, create one.
-
Select one of the Yandex Cloud Marketplace public images, e.g., Ubuntu 22.04 LTS.
To get a list of available images using the CLI, run this command:
yc compute image list --folder-id standard-imagesResult:
+----------------------+-------------------------------------+--------------------------+----------------------+--------+ | ID | NAME | FAMILY | PRODUCT IDS | STATUS | +----------------------+-------------------------------------+--------------------------+----------------------+--------+ ... | fdvk34al8k5n******** | centos-7-1549279494 | centos-7 | dqni65lfhvv2******** | READY | | fdv7ooobjfl3******** | windows-2016-gvlk-1548913814 | windows-2016-gvlk | dqnnc72gj2is******** | READY | | fdv4f5kv5cvf******** | ubuntu-1604-lts-1549457823 | ubuntu-1604-lts | dqnnb6dc7640******** | READY | ... +----------------------+-------------------------------------+--------------------------+----------------------+--------+Where:
-
ID: Image ID. -
NAME: Image name. -
FAMILY: ID of the image family the image belongs to. -
PRODUCT IDS: IDs of Yandex Cloud Marketplace products associated with the image. -
STATUS: Current status of the image. It may take one of the following values:STATUS_UNSPECIFIED: Image status is not defined.CREATING: Image is being created.READY: Image is ready to use.ERROR: You cannot use the image due to an issue.DELETING: Image is being deleted.
-
-
Prepare a file with the YAML specification of the instance group and give it a name, e.g.,
specification.yaml.To connect a file storage to instances in the instance group, add the following to the specification:
-
In the
instance_templatefield, a nestedfilesystem_specsfield containing the description of the file storage:instance_template: ... filesystem_specs: - mode: READ_WRITE device_name: <instance_device_name> filesystem_id: <file_storage_ID>Where:
mode: File storage access mode,READ_WRITE(read and write).device_name: Device name for connecting the file storage to the VM instance, e.g.,sample-fs. It can only contain lowercase Latin letters, numbers, and hyphens. The first character must be a letter. The last character cannot be a hyphen. The name can be up to 63 characters long.filesystem_id: File storage ID. You can view the ID in the management console or using theyc compute filesystem listCLI command.
-
In the
#cloud-configsection of theinstance_template.metadata.user-datafield, commands for mounting the file storage to the VM:instance_template: ... metadata: user-data: |- #cloud-config ... runcmd: - mkdir <instance_mount_point> - mount -t virtiofs <instance_device_name> <instance_mount_point> - echo "test-fs <instance_mount_point> virtiofs rw 0 0" | tee -a /etc/fstabWhere:
<instance_mount_point>: Instance directory to mount the connected file storage to, e.g.,/mnt/vfs0.<instance_device_name>: Device name for connecting the file storage to the VM instance. The value must match the one specified earlier in theinstance_template.filesystem_specs.device_namefield.
Here is a YAML specification example:
name: my-vm-group-with-fs service_account_id: <service_account_ID> description: "This instance group was created using a YAML configuration file." instance_template: platform_id: standard-v3 resources_spec: memory: 2g cores: 2 boot_disk_spec: mode: READ_WRITE disk_spec: image_id: fd8dlvgiatiqd8tt2qke type_id: network-hdd size: 32g network_interface_specs: - network_id: enp9mji1m7b3******** primary_v4_address_spec: { one_to_one_nat_spec: { ip_version: IPV4 } } security_group_ids: - enpuatgvejtn******** filesystem_specs: - mode: READ_WRITE device_name: sample-fs filesystem_id: epdccsrlalon******** metadata: user-data: |- #cloud-config datasource: Ec2: strict_id: false ssh_pwauth: no users: - name: my-user sudo: ALL=(ALL) NOPASSWD:ALL shell: /bin/bash ssh_authorized_keys: - <public_SSH_key> runcmd: - mkdir /mnt/vfs0 - mount -t virtiofs sample-fs /mnt/vfs0 - echo "sample-fs /mnt/vfs0 virtiofs rw 0 0" | tee -a /etc/fstab deploy_policy: max_unavailable: 1 max_expansion: 0 scale_policy: fixed_scale: size: 2 allocation_policy: zones: - zone_id: ru-central1-a instance_tags_pool: - first - secondThis example shows a specification for creating a fixed-size instance group with a file storage connected to the instances.
For more information about the instance group specification parameters, see Specification of an instance group in YAML format.
-
-
Create an instance group in the default folder:
yc compute instance-group create --file specification.yamlThis command will create a group of two same-type instances with the following configuration:
- Name:
my-vm-group-with-fs. - OS:
Ubuntu 22.04 LTS. - Availability zone:
ru-central1-a. - vCPU: 2, RAM: 2 GB.
- Network HDD: 32 GB.
- Connected to a file storage. The file storage will be mounted to the
/mnt/vfs0directory of the group instances. -
Each VM of the group will have а public IP address assigned. This way, you can easily connect to the group VM over SSH when checking the result.
If you create an instance group without public IP addresses, you can still connect to a group VM over SSH by specifying its internal IP address or FQDN instead of the public IP address. However, you can only make such a connection from another virtual machine that has a public IP address and is located in the same Yandex Cloud cloud network as the group VM.
- Name:
If you do not have Terraform yet, install it and configure the Yandex Cloud provider.
-
In the configuration file, define the parameters of the resources you want to create:
resource "yandex_iam_service_account" "ig-sa" { name = "ig-sa" description = "Service account for managing the instance group" } resource "yandex_resourcemanager_folder_iam_member" "compute_editor" { folder_id = "<folder_ID>" role = "compute.editor" member = "serviceAccount:${yandex_iam_service_account.ig-sa.id}" depends_on = [ yandex_iam_service_account.ig-sa, ] } resource "yandex_compute_instance_group" "ig-1" { name = "fixed-ig" folder_id = "<folder_ID>" service_account_id = "${yandex_iam_service_account.ig-sa.id}" deletion_protection = "<deletion_protection>" depends_on = [yandex_resourcemanager_folder_iam_member.compute_editor] instance_template { platform_id = "standard-v3" resources { memory = <RAM_in_GB> cores = <number_of_vCPUs> } boot_disk { mode = "READ_WRITE" initialize_params { image_id = "<image_ID>" } } filesystem { mode = "READ_WRITE" device_name = "<instance_device_name>" filesystem_id = "<file_storage_ID>" } network_interface { network_id = "${yandex_vpc_network.network-1.id}" subnet_ids = ["${yandex_vpc_subnet.subnet-1.id}"] security_group_ids = ["<list_of_security_group_IDs>"] nat = true } metadata = { user-data = "#cloud-config\n datasource:\n Ec2:\n strict_id: false\n ssh_pwauth: no\n users:\n - name: <instance_username>\n sudo: ALL=(ALL) NOPASSWD:ALL\n shell: /bin/bash\n ssh_authorized_keys:\n - <public_SSH_key>\n runcmd:\n - mkdir <instance_mount_point>\n - mount -t virtiofs <instance_device_name> <instance_mount_point>\n - echo \"sample-fs <instance_mount_point> virtiofs rw 0 0\" | tee -a /etc/fstab" } } scale_policy { fixed_scale { size = <number_of_instances_in_group> } } allocation_policy { zones = ["ru-central1-a"] } deploy_policy { max_unavailable = 1 max_expansion = 0 } } resource "yandex_vpc_network" "network-1" { name = "network1" } resource "yandex_vpc_subnet" "subnet-1" { name = "subnet1" zone = "ru-central1-a" network_id = "${yandex_vpc_network.network-1.id}" v4_cidr_blocks = ["192.168.10.0/24"] }Where:
-
yandex_iam_service_account: Service account description. All operations in Instance Groups are performed on behalf of the service account.You cannot delete a service account while it is linked to an instance group.
-
yandex_resourcemanager_folder_iam_member: Description of access permissions for the folder the service account belongs to. To be able to create, update, and delete VM instances in the instance group, assign thecompute.editorrole to the service account. -
yandex_compute_instance_group: Instance group description:- General information about the instance group:
name: Instance group name.folder_id: Folder ID.service_account_id: Service account ID.deletion_protection: Instance group protection against deletion,trueorfalse. You cannot delete an instance group with this option enabled. The default value isfalse.
- Instance template:
-
platform_id: Platform. -
resources: Number of vCPUs and amount of RAM available to the VM instance. The values must match the selected platform. -
boot_disk: Boot disk settings.mode: Disk access mode,READ_ONLYorREAD_WRITE.image_id: ID of the selected image. You can get the image ID from the list of public images.
-
filesystem: File storage settings.mode: File storage access mode,READ_WRITE(read and write).device_name: Device name for connecting the file storage to the VM instance, e.g.,sample-fs. It can only contain lowercase Latin letters, numbers, and hyphens. The first character must be a letter. The last character cannot be a hyphen. The name can be up to 63 characters long.filesystem_id: File storage ID. You can view the ID in the management console or using theyc compute filesystem listCLI command.
-
network_interface: Network settings. Specify the IDs of your network, subnet, and security groups. -
metadata: In metadata, provide the following:- Instance username and public key to enable this user to access the instance via SSH.
- Instance mount point for the file storage, i.e., instance directory to mount the connected file storage to, e.g.,
/mnt/vfs0. - Instance device name, i.e., device name for connecting the file storage to the VM instance. The value must match the one specified earlier in the
device_namefield of thefilesystemsection.
For more information, see VM metadata.
-
- Policies:
deploy_policy: Instance deployment policy for the group.scale_policy: Instance scaling policy for the group.allocation_policy: Policy for allocating instances across availability zones and regions.
- General information about the instance group:
-
yandex_vpc_network: Cloud network description. -
yandex_vpc_subnet: Description of the subnet to connect the instance group to.Note
If you already have suitable resources, such as a service account, cloud network, and subnet, you do not need to redefine them. Specify their names and IDs in the appropriate parameters.
For more information about the resources you can create with Terraform, see the relevant provider documentation.
-
-
Create the resources:
-
In the terminal, go to the directory where you edited the configuration file.
-
Make sure the configuration file is correct using this command:
terraform validateIf the configuration is correct, you will get this message:
Success! The configuration is valid. -
Run this command:
terraform planYou will see a detailed list of resources. No changes will be made at this step. If the configuration contains any errors, Terraform will show them.
-
Apply the changes:
terraform apply -
Type
yesand press Enter to confirm the changes.
All the resources you need will then be created in the specified folder. You can check the new resources and their settings using the management console
.Each VM of the group will have а public IP address assigned. This way, you can easily connect to the group VM over SSH when checking the result.
If you create an instance group without public IP addresses, you can still connect to a group VM over SSH by specifying its internal IP address or FQDN instead of the public IP address. However, you can only make such a connection from another virtual machine that has a public IP address and is located in the same Yandex Cloud cloud network as the group VM.
-
Use the create REST API method for the InstanceGroup resource or the InstanceGroupService/Create gRPC API call.
-
Make sure the file storage is connected to the group instances. For this, connect to the instances via SSH and navigate to the directory you specified as the mount point.