Contact UsGet started

Managing OpenSearch host groups

Written by
Updated at October 17, 2024

In a Managed Service for OpenSearch cluster, you can manage host groups:

You can also get a list of cluster hosts.

For information about migrating host groups in a Managed Service for OpenSearch cluster to a different availability zone, see this guide.

Getting a list of host groups in a cluster

  1. In the management console, go to the folder page and select Managed Service for OpenSearch.
  2. Click the name of the cluster you need and select the  Node groups tab.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To get a list of host groups in a cluster, request information about the OpenSearch cluster:

yc managed-opensearch cluster get <cluster_name_or_ID>

You can find the list of host groups in the config.opensearch.node_groups and config.dashboards.node_groups parameters.

You can request the cluster name and ID with a list of clusters in the folder.

To get a list of host groups in a cluster, use the get REST API method for the Cluster resource or the ClusterService/Get gRPC API call and provide the cluster ID in the clusterId request parameter.

You can get the cluster ID with a list of clusters in the folder.

Creating a host group

The following limitations apply when creating host groups:

  • A Managed Service for OpenSearch cluster may contain only one Dashboards host group.
  • If you are adding a group of OpenSearch hosts and assign the MANAGER role to the hosts, the minimum number of hosts with such a role will be three.

To create a host group:

  1. In the management console, go to the folder page and select Managed Service for OpenSearch.

  2. Click the name of the cluster you need and select the  Node groups tab.

  3. Click Create node group.

  4. Specify the group parameters:

    • Group type: OpenSearch or Dashboards.

    • Name which must be unique within the cluster.

    • For a OpenSearch host group, select a host role.

    • Platform, host type, and host class.

      The host class defines the technical characteristics of virtual machines that OpenSearch nodes are deployed on. All available options are listed under Host classes.

    • Disk type and data storage size.

      The selected type determines the increments in which you can change your disk size:

      • Network HDD and SSD storage: In 1 GB increments.
      • Local SSD storage:
        • For Intel Cascade Lake: In increments of 100 GB.
        • For Intel Ice Lake: In 368 GB increments.
      • Non-replicated SSD storage: In increments of 93 GB.

    • Host distribution across availability zones and subnets.

    • Number of hosts to create.

    • Enable Public access if you want to allow connecting to hosts over the internet.

  5. Click Create node group.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To create a host group, run this command:

yc managed-opensearch node-group add --cluster-name <cluster_name> \
   --opensearch-node-group name=<name_of_OpenSearch_host_group>,`
                          `resource-preset-id=<host_class>,`
                          `disk-size=<disk_size_in_bytes>,`
                          `disk-type-id=<disk_type>,`
                          `hosts-count=<number_of_hosts_per_group>,`
                          `zone-ids=<availability_zones>,`
                          `subnet-names=<subnet_names>,`
                          `assign-public-ip=<assign_public_IP:_true_or_false>,`
                          `roles=<host_roles> \
   --dashboards-node-group name=<name_of_Dashboards_host_group>,`
                          `resource-preset-id=<host_class>,`
                          `disk-size=<disk_size_in_bytes>,`
                          `disk-type-id=<disk_type>,`
                          `hosts-count=<number_of_hosts_per_group>,`
                          `zone-ids=<availability_zones>,`
                          `subnet-names=<subnet_names>,`
                          `assign-public-ip=<assign-public-ip=<assign_public_IP:_true_or_false>

Specify the required parameters in the command depending on what type of host group you want to create:

  • --opensearch-node-group: OpenSearch host group configuration, where:

    • --resource-preset-id: Host class that defines the configuration of virtual machines the OpenSearch nodes will be deployed on. All available options are listed under Host classes.

    • disk-size: Disk size in bytes. Minimum and maximum values depend on the selected host class.

    • disk-type-id: Disk type.

    • roles: Host roles. The possible values include:

      • data: Assigns the DATA role only.
      • manager: Assigns the MANAGER role only.
      • data+manager or manager+data: Assigns both roles.

      Tip

      We do not recommend enabling public access to hosts with the MANAGER role, as this is not secure.

  • --dashboards-node-group: Dashboards host group configuration. It is configured in the same way as the OpenSearch host group, except for the host roles. You do not need to configure any roles for the Dashboards group.

  1. Open the current Terraform configuration file with an infrastructure plan.

    For more information about how to create this file, see Creating clusters.

    For a complete list of available Managed Service for OpenSearch cluster configuration fields, see the Terraform provider documentation.

  2. To create an OpenSearch host group, add the node_groups section to opensearch:

    resource "yandex_mdb_opensearch_cluster" "<cluster_name>" {
  ...
  config {
    opensearch {
      ...
      node_groups {
        name             = "<virtual_host_group_name>"
        assign_public_ip = <public_access>
        hosts_count      = <number_of_hosts>
        zone_ids         = ["<list_of_availability_zones>"]
        subnet_ids       = ["<list_of_subnet_IDs>"]
        roles            = ["<list_of_roles>"]
        resources {
          resource_preset_id = "<host_class>"
          disk_size          = <storage_size_in_bytes>
          disk_type_id       = "<disk_type>"
        }
      }
    }
    ...
  }
}

    Where:

    • assign_public_ip: Public access to the host, true or false.
    • roles: DATA and MANAGER host roles.

  3. To create a Dashboards host group, add the dashboards section to config:

    resource "yandex_mdb_opensearch_cluster" "<cluster_name>" {
  ...
  config {
    ...
    dashboards {
      node_groups {
       name             = "<virtual_host_group_name>"
        assign_public_ip = <public_access>
        hosts_count      = <number_of_hosts>
        zone_ids         = ["<list_of_availability_zones>"]
        subnet_ids       = ["<list_of_subnet_IDs>"]
        resources {
          resource_preset_id = "<host_class>"
          disk_size          = <storage_size_in_bytes>
          disk_type_id       = "<disk_type>"
        }
      }
    }
  }
}

    Where assign_public_ip is public access to the host, true or false.

    A cluster may contain only one Dashboards host group.

  4. Make sure the settings are correct.

    1. Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.

    2. Run the command:

      terraform validate

      If there are errors in the configuration files, Terraform will point to them.

  5. Confirm updating the resources.

    1. Run the command to view planned changes:

      terraform plan

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

    2. If you are happy with the planned changes, apply them:

      1. Run the command:

        terraform apply

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

    Timeouts

    The Terraform provider sets the following timeouts for Managed Service for OpenSearch cluster operations:

    • Creating a cluster, including restoring from a backup: 30 minutes.
    • Editing a cluster: 60 minutes.
    • Deleting a cluster: 15 minutes.

    Operations exceeding the set timeout are interrupted.

    How do I change these limits?

    Add the timeouts block to the cluster description, for example:

    resource "yandex_mdb_opensearch_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

To create an OpenSearch host group, use the addOpenSearchNodeGroup REST API method for the Cluster resource or the ClusterService/AddOpenSearchNodeGroup gRPC API call.

To create a Dashboards host group, use the addDashboardsNodeGroup REST API method for the Cluster resource or the ClusterService/AddDashboardsNodeGroup gRPC API call.

Provide the group configuration under nodeGroupSpec in the request:

  • Host group name in the name parameter.

  • Host class in the resources.resourcePresetId parameter.

  • Disk type in the resources.diskTypeId parameter.

  • Data storage size in the resources.diskSize parameter.

  • Number of group hosts in the hostsCount parameter.

  • List of availability zones in the zoneIds parameter.

  • List of subnets in the subnetIds parameter.

  • Public access settings in the assignPublicIp parameter.

  • List of host roles in the roles parameter (for OpenSearch host groups only).

Updating a host group configuration

  1. In the management console, go to the folder page and select Managed Service for OpenSearch.

  2. Click the name of the cluster you need and select the  Node groups tab.

  3. Click in the line with the appropriate group and select Edit.

  4. Change host group settings:

    • Host roles (for OpenSearch host groups only).

    • Platform, host type, and host class.

      The host class defines the technical characteristics of virtual machines that OpenSearch nodes are deployed on. All available options are listed under Host classes.

    • Disk size.

      Disk resizing increments depend on disk type:

      • Network HDD and SSD storage: In increments of 1 GB.

      • Local SSD storage:

        • For Intel Cascade Lake: In increments of 100 GB.
        • For Intel Ice Lake: In increments of 368 GB.

      • Non-replicated SSD storage: In increments of 93 GB.

    • Host distribution across availability zones and subnets.

    • Number of hosts.

    • Public access to hosts.

  5. Click Save.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To change the configuration of a host group, run the command:

yc managed-opensearch node-group update --cluster-name <cluster_name> \
   --node-group-name <host_group_name> \
   --resource-preset-id <host_class> \
   --disk-size <disk_size_in_bytes> \
   --hosts-count <number_of_hosts_per_group> \
   --roles <host_roles>

Specify the required parameters in the command depending on what type of host group configuration you need:

  • --node-group-name: Name of the host group you want to update.

  • --resource-preset-id: New host class that defines the configuration of virtual machines the OpenSearch nodes will be deployed on. All available options are listed under Host classes.

  • --disk-size: New disk size in bytes. Minimum and maximum values depend on the selected host class.

  • --hosts-count: New number of hosts in the group.

  • --roles: New host roles. The possible values include:

    • data: Assigns the DATA role only.
    • manager: Assigns the MANAGER role only.
    • data+manager or manager+data: Assigns both roles.

  1. Open the current Terraform configuration file with an infrastructure plan.

    For more information about how to create this file, see Creating clusters.

    For a complete list of available Managed Service for OpenSearch cluster configuration fields, see the Terraform provider documentation.

  2. To update the configuration of an OpenSearch host group, edit the parameters of the required node_groups section in the opensearch section:

    resource "yandex_mdb_opensearch_cluster" "<cluster_name>" {
  ...
  opensearch {
    ...
    node_groups {
      name             = "<virtual_host_group_name>"
      assign_public_ip = <public_access>
      hosts_count      = <number_of_hosts>
      roles            = ["<list_of_roles>"]
      resources {
        resource_preset_id = "<host_class>"
        disk_size          = <storage_size_in_bytes>
        disk_type_id       = "<disk_type>"
      }
    }
    ...
  }
}

    Where:

    • assign_public_ip: Public access to the host, true or false.
    • roles: DATA and MANAGER host roles.

  3. To update the configuration of a Dashboards host group, edit the dashboards section parameters:

    resource "yandex_mdb_opensearch_cluster" "<cluster_name>" {
  ...
  dashboards {
    node_groups {
      name             = "<virtual_host_group_name>"
      assign_public_ip = <public_access>
      hosts_count      = <number_of_hosts>
      resources {
        resource_preset_id = "<host_class>"
        disk_size          = <storage_size_in_bytes>
        disk_type_id       = "<disk_type>"
      }
    }
  }
}

    Where assign_public_ip is public access to the host, true or false.

  4. Make sure the settings are correct.

    1. Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.

    2. Run the command:

      terraform validate

      If there are errors in the configuration files, Terraform will point to them.

  5. Confirm updating the resources.

    1. Run the command to view planned changes:

      terraform plan

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

    2. If you are happy with the planned changes, apply them:

      1. Run the command:

        terraform apply

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

    Time limits

    A Terraform provider sets the timeout for Managed Service for Elasticsearch cluster operations:

    • Creating a cluster, including by restoring one from a backup: 30 minutes.
    • Editing a cluster: 60 minutes.
    • Deleting a cluster: 15 minutes.

    Operations exceeding the set timeout are interrupted.

    How do I change these limits?

    Add the timeouts block to the cluster description, for example:

    resource "yandex_mdb_elasticsearch_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

To update the configuration of an OpenSearch host group, use the updateOpenSearchNodeGroup REST API method for the Cluster resource or the ClusterService/UpdateOpenSearchNodeGroup gRPC API call.

To update the configuration of a Dashboards host group, use the updateDashboardsNodeGroup REST API method for the Cluster resource or the ClusterService/UpdateDashboardsNodeGroup gRPC API call.

Provide a new configuration under nodeGroupSpec in the request:

  • Host class in the resources.resourcePresetId parameter.
  • Disk type in the resources.diskTypeId parameter.
  • Data storage size in the resources.diskSize parameter.
  • Number of group hosts in the hostsCount parameter.
  • List of host roles in the roles parameter (for OpenSearch host groups only).

Warning

This API method overrides all parameters of the object being modified that were not explicitly passed in the request to the default values. To avoid this, list the settings you want to change in the updateMask parameter (one line separated by commas).

Deleting a host group

When deleting a host group, the following limitation applies: you cannot delete the only host group with the DATA role.

  1. In the management console, go to the folder page and select Managed Service for OpenSearch.
  2. Click the name of the cluster you need and select the  Node groups tab.
  3. Click in the line with the appropriate group and select Delete.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To delete a host group, run the command:

yc managed-opensearch node-group delete --cluster-name <cluster_name> \
   --node-group-name <host_group_name>

In the command, specify the host group you want to delete.

To remove a host group from a cluster:

  1. Open the current Terraform configuration file with an infrastructure plan.

    For more information about how to create this file, see Creating clusters.

    For a complete list of available Managed Service for OpenSearch cluster configuration fields, see the Terraform provider documentation.

  2. To delete an OpenSearch host group, remove the corresponding node_groups section from the opensearch section.

  3. To delete a Dashboards host group, remove the dashboards section.

  4. Make sure the settings are correct.

    1. Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.

    2. Run the command:

      terraform validate

      If there are errors in the configuration files, Terraform will point to them.

  5. Confirm the deletion of resources.

    1. Run the command to view planned changes:

      terraform plan

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

    2. If you are happy with the planned changes, apply them:

      1. Run the command:

        terraform apply

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

    Time limits

    A Terraform provider sets the timeout for Managed Service for Elasticsearch cluster operations:

    • Creating a cluster, including by restoring one from a backup: 30 minutes.
    • Editing a cluster: 60 minutes.
    • Deleting a cluster: 15 minutes.

    Operations exceeding the set timeout are interrupted.

    How do I change these limits?

    Add the timeouts block to the cluster description, for example:

    resource "yandex_mdb_elasticsearch_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

To delete an OpenSearch host group, use the deleteOpenSearchNodeGroup REST API method for the Cluster resource or the ClusterService/DeleteOpenSearchNodeGroup gRPC API call.

To delete a Dashboards host group, use the deleteDashboardsNodeGroup REST API method for the Cluster resource or the ClusterService/DeleteDashboardsNodeGroup gRPC API call.

Provide the following in the request:

  • Cluster ID in the clusterID parameter.

    You can get the cluster ID with a list of clusters in the folder.

  • Name of the host group to delete, in the name parameter.

Getting a list of cluster hosts

  1. Go to the folder page and select Managed Service for OpenSearch.
  2. Click the cluster name and select the Hosts tab.

To get a list of cluster hosts, use the listHosts REST API method for the Cluster resource or the ClusterService/ListHosts gRPC API call and provide the cluster ID in the clusterId request parameter.

To find out the cluster ID, get a list of clusters in the folder.

Previous
Managing plugins
Next
Migrating host groups to a different availability zone