Creating an instance group connected to Yandex Object Storage

Written by

Updated at February 12, 2025

One of the ways to handle stateful workloads is to store the application state in an Object Storage bucket independent of the instance group.

To create an instance group that will automatically connect a common Object Storage bucket to each of its instances:

By default, all operations in Instance Groups are performed on behalf of a service account. If you don't have a service account, create one.
To be able to create, update, and delete VMs in the group, assign the compute.editor role to the service account.
If you do not have an Object Storage bucket, create one.
Bucket operations are performed under the service account created within the same folder as the bucket. If there is no such service account, create one. To work with the bucket, assign the storage.editor role to the service account.

You can use either a single service account or two separate service accounts for working with the instance group and the bucket.
Create an instance group:
CLI

Terraform

API
If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder through the --folder-name or --folder-id parameter.
1. See the description of the CLI command for creating an instance group:
  yc compute instance-group create --help
2. Check whether the folder contains any networks:
  yc vpc network list
  If there are no networks, create one.
3. Select one of the Yandex Cloud Marketplace public images, e.g., Ubuntu 22.04 LTS.
  
  To get a list of available images using the CLI, run this command:
  yc compute image list --folder-id standard-images
  Result:
  +----------------------+-------------------------------------+--------------------------+----------------------+--------+ | ID | NAME | FAMILY | PRODUCT IDS | STATUS | +----------------------+-------------------------------------+--------------------------+----------------------+--------+ ... | fdvk34al8k5n******** | centos-7-1549279494 | centos-7 | dqni65lfhvv2******** | READY | | fdv7ooobjfl3******** | windows-2016-gvlk-1548913814 | windows-2016-gvlk | dqnnc72gj2is******** | READY | | fdv4f5kv5cvf******** | ubuntu-1604-lts-1549457823 | ubuntu-1604-lts | dqnnb6dc7640******** | READY | ... +----------------------+-------------------------------------+--------------------------+----------------------+--------+
  Where:
  - ID: Image ID.
  - NAME: Image name.
  - FAMILY: ID of the image family the image belongs to.
  - PRODUCT IDS: IDs of Yandex Cloud Marketplace products associated with the image.
  - STATUS: Current status of the image. It may take one of the following values:
    
    STATUS_UNSPECIFIED: Image status is not defined.
    
    CREATING: Image is being created.
    
    READY: Image is ready to use.
    
    ERROR: You cannot use the image due to an issue.
    
    DELETING: Image is being deleted.
4. Prepare a file with the YAML specification of the instance group and give it a name, e.g., specification.yaml.
  
  To connect a bucket to instances in the instance group, add the following to the specification:
  - In the instance_template field, a nested service_account_id field containing the ID of the service account with the storage.editor role assigned to it:
    
    instance_template: ... service_account_id: <service_account_ID>
    
    For more granular access management, link different service accounts with different permissions to the instance group and individual instances within that group.
  - In the #cloud-config section of the instance_template.metadata.user-data field, commands for mounting the bucket to the instance:
    
    instance_template: ... metadata: user-data: |- #cloud-config ... runcmd: - apt-get install fuse - wget https://github.com/yandex-cloud/geesefs/releases/latest/download/geesefs-linux-amd64 - chmod a+x geesefs-linux-amd64 - cp geesefs-linux-amd64 /usr/bin/geesefs - mkdir <instance_mount_point> - echo "user_allow_other" | tee -a /etc/fuse.conf - echo "<bucket_name> <instance_mount_point> fuse.geesefs _netdev,allow_other,--iam 0 0" | tee -a /etc/fstab - mount -a
    
    Where:
    
    - apt-get install fuse: Command for installing FUSE. This command can run on Ubuntu and Debian. For operating systems based on Red Hat (such as CentOS and Fedora), use the - yum install fuse command, for OpenSUSE, - zypper install fuse, etc.
    
    <instance_mount_point>: Instance directory to mount the connected bucket to, e.g., /mnt/gfs0.
    
    <bucket_name>: Name of the bucket to connect to the instance.
  Here is a YAML specification example:
  name: vm-group-with-object-storage service_account_id: <service_account_ID> // service account with the compute.editor role. description: "This instance group was created using a YAML configuration file." instance_template: platform_id: standard-v3 resources_spec: memory: 2g cores: 2 boot_disk_spec: mode: READ_WRITE disk_spec: image_id: fd8dlvgiatiqd8tt2qke type_id: network-hdd size: 32g network_interface_specs: - network_id: enp9mji1m7b3******** primary_v4_address_spec: { one_to_one_nat_spec: { ip_version: IPV4 } } security_group_ids: - enpuatgvejtn******** service_account_id: <service_account_ID> // service account with the storage.editor role. metadata: user-data: |- #cloud-config datasource: Ec2: strict_id: false ssh_pwauth: no users: - name: my-user sudo: ALL=(ALL) NOPASSWD:ALL shell: /bin/bash ssh_authorized_keys: - <public_SSH_key> runcmd: - apt-get install fuse - wget https://github.com/yandex-cloud/geesefs/releases/latest/download/geesefs-linux-amd64 - chmod a+x geesefs-linux-amd64 - cp geesefs-linux-amd64 /usr/bin/geesefs - mkdir /mnt/gfs0 - echo "user_allow_other" | tee -a /etc/fuse.conf - echo "my-bucket-for-vm-group /mnt/gfs0 fuse.geesefs _netdev,allow_other,--iam 0 0" | tee -a /etc/fstab - mount -a deploy_policy: max_unavailable: 1 max_expansion: 0 scale_policy: fixed_scale: size: 2 allocation_policy: zones: - zone_id: ru-central1-a instance_tags_pool: - first - second
  This example shows a specification for creating a fixed-size instance group with an Object Storage bucket connected to the instances.
  
  For more information about the instance group specification parameters, see Specification of an instance group in YAML format.
5. Create an instance group in the default folder:
  yc compute instance-group create --file specification.yaml
  This command will create a group of two same-type instances with the following configuration:
  - Name: my-vm-group-with-object-storage.
  - OS: Ubuntu 22.04 LTS.
  - Availability zone: ru-central1-a.
  - vCPU: 2, RAM: 2 GB.
  - Network HDD: 32 GB.
  - Connected to an Object Storage bucket. The bucket will be mounted to the /mnt/gfs0 directory of the group instances.
  - Each VM of the group will have а public IP address assigned. This way, you can easily connect to the group VM over SSH when checking the result.
    
    If you create an instance group without public IP addresses, you can still connect to a group VM over SSH by specifying its internal IP address or FQDN instead of the public IP address. However, you can only make such a connection from another virtual machine that has a public IP address and is located in the same Yandex Cloud cloud network as the group VM.
If you don't have Terraform, install it and configure the Yandex Cloud provider.
1. In the configuration file, define the parameters of the resources you want to create:
  resource "yandex_iam_service_account" "ig-sa" { name = "ig-sa" description = "Service account for managing the instance group." } resource "yandex_iam_service_account" "storage-sa" { name = "storage-sa" description = "Service account for managing the bucket." } resource "yandex_resourcemanager_folder_iam_member" "compute_editor" { folder_id = "<folder_ID>" role = "compute.editor" member = "serviceAccount:${yandex_iam_service_account.ig-sa.id}" depends_on = [ yandex_iam_service_account.ig-sa, ] } resource "yandex_resourcemanager_folder_iam_member" "storage_editor" { folder_id = "<folder_ID>" role = "storage.editor" member = "serviceAccount:${yandex_iam_service_account.storage-sa.id}" depends_on = [ yandex_iam_service_account.storage-sa, ] } resource "yandex_compute_instance_group" "ig-1" { name = "fixed-ig" folder_id = "<folder_ID>" service_account_id = "${yandex_iam_service_account.ig-sa.id}" deletion_protection = "<deletion_protection>" depends_on = [yandex_resourcemanager_folder_iam_member.compute_editor] instance_template { platform_id = "standard-v3" resources { memory = <RAM_in_GB> cores = <number_of_vCPUs> } boot_disk { mode = "READ_WRITE" initialize_params { image_id = "<image_ID>" } } service_account_id = "${yandex_iam_service_account.storage-sa.id}" network_interface { network_id = "${yandex_vpc_network.network-1.id}" subnet_ids = ["${yandex_vpc_subnet.subnet-1.id}"] security_group_ids = ["<list_of_security_group_IDs>"] nat = true } metadata = { user-data = "#cloud-config\n datasource:\n Ec2:\n strict_id: false\n ssh_pwauth: no\n users:\n - name: <instance_username>\n sudo: ALL=(ALL) NOPASSWD:ALL\n shell: /bin/bash\n ssh_authorized_keys:\n - <public_SSH_key>\n runcmd:\n - apt-get install fuse\n - wget https://github.com/yandex-cloud/geesefs/releases/latest/download/geesefs-linux-amd64\n - chmod a+x geesefs-linux-amd64\n - cp geesefs-linux-amd64 /usr/bin/geesefs\n - mkdir <instance_mount_point>\n - echo \"user_allow_other\" | tee -a /etc/fuse.conf\n - echo \"<bucket_name> <instance_mount_point> fuse.geesefs _netdev,allow_other,--iam 0 0\" | tee -a /etc/fstab\n - mount -a" } } scale_policy { fixed_scale { size = <number_of_instances_in_group> } } allocation_policy { zones = ["ru-central1-a"] } deploy_policy { max_unavailable = 1 max_expansion = 0 } } resource "yandex_vpc_network" "network-1" { name = "network1" } resource "yandex_vpc_subnet" "subnet-1" { name = "subnet1" zone = "ru-central1-a" network_id = "${yandex_vpc_network.network-1.id}" v4_cidr_blocks = ["192.168.10.0/24"] }
  Where:
  - yandex_iam_service_account: Service account description. All operations in Instance Groups are performed on behalf of the service account. For more granular access management, link different service accounts with different permissions to the instance group and individual instances within that group.
    
    You cannot delete a service account while it is linked to an instance group.
  - yandex_resourcemanager_folder_iam_member: Description of access permissions for the folder the service account belongs to. To be able to create, update, and delete VM instances in the instance group, assign the compute.editor role to the service account.
  - yandex_compute_instance_group: Instance group description:
    
    General information about the instance group:
    
    name: Instance group name.
    
    folder_id: Folder ID.
    
    service_account_id: ID of the service account for the instance group. This service account must have the compute.editor role.
    
    deletion_protection: Instance group protection against deletion, true or false. You cannot delete an instance group with this option enabled. The default value is false.
    
    Instance template:
    
    platform_id: Platform.
    
    resources: Number of vCPUs and amount of RAM available to the VM instance. The values must match the selected platform.
    
    boot_disk: Boot disk settings.
    
    mode: Disk access mode, READ_ONLY or READ_WRITE.
    
    image_id: ID of the selected image. You can get the image ID from the list of public images.
    
    service_account_id: ID of the service account for the bucket. This service account must have the storage.editor role.
    
    network_interface: Network settings. Specify the IDs of your network, subnet, and security groups.
    
    metadata: In metadata, provide the following:
    
    Instance username and public key to enable this user to access the instance via SSH.
    
    - apt-get install fuse: Command for installing FUSE. This command can run on Ubuntu and Debian. For operating systems based on Red Hat (such as CentOS and Fedora), use the - yum install fuse command, for OpenSUSE, - zypper install fuse, etc.
    
    <instance_mount_point>: Instance directory to mount the connected bucket to, e.g., /mnt/gfs0.
    
    <bucket_name>: Name of the bucket to connect to the instance.
    
    For more information, see VM metadata.
    
    Policies:
    
    deploy_policy: Instance deployment policy for the group.
    
    scale_policy: Instance scaling policy for the group.
    
    allocation_policy: Policy for allocating instances across availability zones and regions.
  - yandex_vpc_network: Cloud network description.
  - yandex_vpc_subnet: Description of the subnet to connect the instance group to.
    
    Note
    
    If you already have suitable resources, such as service accounts, a cloud network, and a subnet, you do not need to redefine them. Specify their names and IDs in the appropriate parameters.
  For more information about the resources you can create with Terraform, see the relevant provider documentation.
2. Create the resources:
  1. In the terminal, change to the folder where you edited the configuration file.
  2. Make sure the configuration file is correct using the command:
    
    terraform validate
    
    If the configuration is correct, the following message is returned:
    
    Success! The configuration is valid.
  3. Run the command:
    
    terraform plan
    
    The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.
  4. Apply the configuration changes:
    
    terraform apply
  5. Confirm the changes: type yes in the terminal and press Enter.
  All the resources you need will then be created in the specified folder. You can check the new resources and their settings using the management console.
  
  Each VM of the group will have а public IP address assigned. This way, you can easily connect to the group VM over SSH when checking the result.
  
  If you create an instance group without public IP addresses, you can still connect to a group VM over SSH by specifying its internal IP address or FQDN instead of the public IP address. However, you can only make such a connection from another virtual machine that has a public IP address and is located in the same Yandex Cloud cloud network as the group VM.
Use the create REST API method for the InstanceGroup resource or the InstanceGroupService/Create gRPC API call.

Make sure the bucket is connected to the group instances. For this, connect to the instances via SSH and navigate to the directory you specified as the mount point.

Creating an instance group connected to Yandex Object Storage

Was the article helpful?