Creating an OpenSearch cluster
A Managed Service for OpenSearch cluster is a group of multiple interlinked OpenSearch and DashboardsDATA role. To learn more about roles in the cluster, see Host roles.
Available disk types depend on the selected host class.
For more information, see Resource relationships in the service.
Roles for creating a cluster
To create a Managed Service for OpenSearch cluster, you will need the vpc.user and managed-opensearch.editor roles or higher.
To link a service account to a cluster, e.g., to use Yandex Object Storage, make sure your Yandex Cloud account has the iam.serviceAccounts.user role or higher.
For more information about assigning roles, see this Yandex Identity and Access Management guide.
Creating a cluster
When creating a cluster, you need to specify individual parameters for each host group.
To create a Managed Service for OpenSearch cluster:
-
In the management console
, select the folder where you want to create a cluster. -
Select Managed Service for OpenSearch.
-
Click Create cluster.
-
Under Basic parameters:
-
Enter a name for the cluster. It must be unique within the folder.
-
Optionally, enter a description for the cluster.
-
Select the environment where you want to create your cluster (the environment cannot be changed after cluster creation):
PRODUCTION: For stable versions of your applications.PRESTABLE: For testing purposes. The prestable environment is similar to the production environment and likewise covered by the SLA, but it is the first of the two to get new features, improvements, and bug fixes. In the prestable environment, you can test the new versions for compatibility with your application.
-
Select the OpenSearch version.
-
Select the plugins you want to install in the cluster.
-
-
Under Network settings, select a cloud network to host the cluster and security groups for cluster network traffic. You may need to configure the security groups to connect to the cluster.
-
Under Virtual node group 1, configure the
OpenSearchhost group:-
Select the host group type:
OpenSearch. -
Enter a name for the host group. It must be unique within the cluster.
-
Select the
DATAandMANAGERhost roles. -
Select the platform, host type, and host class.
The host class defines the technical characteristics of virtual machines that OpenSearch nodes are deployed on. All available options are listed under Host classes.
-
Select the disk type and data storage size.
The selected type determines the increments in which you can change your disk size:
- Network HDD and SSD storage: In increments of 1 GB.
- Local SSD storage:
- For Intel Cascade Lake: In increments of 100 GB.
- For Intel Ice Lake: In increments of 368 GB.
- Non-replicated SSD storage: In increments of 93 GB.
-
Optionally, select Encrypted disk to encrypt the disk with a custom KMS key.
-
To create a new key, click Create.
-
To use the key you created earlier, select it in the KMS key field.
To learn more about disk encryption, see Storage.
-
-
Optionally, set up automatic increase of disk size:
-
In the Increase size field, specify the conditions for the actions below:
-
Storage size increase during the next maintenance window once the fill level exceeds the specified percentage. If you set this condition, configure the maintenance window schedule.
-
Storage size increase immediately once the fill level exceeds the specified percentage.
Warning
If you set both conditions, make sure the immediate increase threshold is higher than the scheduled one.
-
-
Specify maximum storage size after the increase in the Maximum storage size field.
Warning
- You cannot decrease the storage size.
- While resizing the storage, cluster hosts will be unavailable.
-
-
Specify how hosts should be distributed across availability zones and subnets.
-
Select the number of hosts to create.
-
Enable Public access if you want to allow connecting to hosts over the internet.
Tip
For security reasons, we do not recommend enabling public access for hosts with the
MANAGERrole.
-
-
Configure the
Dashboardshost group under Virtual node group 2, if required:-
Select the platform, host type, and host class.
-
Set up storage in the same way as for
OpenSearchhosts. -
Specify how hosts should be distributed across availability zones and subnets.
-
Select the number of hosts to create.
-
Enable Public access if you want to allow connecting to hosts over the internet.
Tip
You can use OpenSearch Dashboards even if you can't request public access to the hosts (for example, for security reasons). To do this, proxy the connections via the virtual machine in Yandex Compute Cloud that is hosted in the same network as the cluster. For more information, see Connecting to OpenSearch Dashboards.
-
-
If required, click Add virtual node group to add another host group or more.
-
Under Service settings:
-
Enter the password for the
adminuser.The password must contain three groups of characters out of these four:
- Lowercase Latin letters
- Uppercase Latin letters
- Numbers
- Special characters
The password must be from 10 to 72 characters long.
adminis a dedicated user required to manage the cluster and cannot be deleted. Such a user has thesuperuserrole and can perform any operations with the cluster.Tip
This user is not intended for routine jobs; for those, we recommend creating regular users. For more information, see Managing OpenSearch users.
-
If required, change additional cluster settings:
-
Maintenance window: Maintenance window settings:
- To enable maintenance at any time, select arbitrary (default).
- To specify the preferred maintenance start time, select by schedule and specify the desired day of the week and UTC hour. For example, you can choose a time when the cluster is least loaded.
Maintenance operations are carried out both on enabled and disabled clusters. They may include updating the DBMS, applying patches, and so on.
-
Service account is an account to access Yandex Object Storage as a repository of OpenSearch snapshots. For more detail on service accounts, see the Yandex Identity and Access Management documentation.
-
Deletion protection: Manages cluster protection against accidental deletion.
Even with cluster deletion protection enabled, one can still delete a user or connect to the cluster manually and delete the data.
-
-
-
Click Create cluster.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To create a Managed Service for OpenSearch cluster:
-
See the description of the CLI command for creating a cluster:
yc managed-opensearch cluster create --help -
Specify the cluster settings in the creation command (our example does not include all available settings):
yc managed-opensearch cluster create \ --name <cluster_name> \ --description <cluster_description> \ --labels <labels> \ --environment <environment> \ --network-name <network_name> \ --security-group-ids <security_group_IDs> \ --service-account-name <service_account_name> \ --delete-protection \ --maintenance schedule=<maintenance_type>,` `weekday=<day_of_week>,` `hour=<hour> \ --disk-encryption-key-id <KMS_key_ID> \ --version <OpenSearch_version> \ --read-admin-password \ --data-transfer-access=<allow_access_from_Data_Transfer> \ --serverless-access=<allow_access_from_Serverless_Containers> \ --plugins <OpenSearch_plugins> \ --advanced-params <additional_parameters> \ --opensearch-node-group name=<OpenSearch_host_group_name>,` `resource-preset-id=<host_class>,` `disk-size=<disk_size_in_bytes>,` `disk-type-id=<network-hdd|network-ssd|network-ssd-io-m3|network-ssd-nonreplicated|local-ssd>,` `hosts-count=<number_of_hosts_in_group>,` `zone-ids=<availability_zones>,` `subnet-names=<subnet_names>,` `assign-public-ip=<allow_public_access_to_hosts>,` `roles=<host_roles> \ --dashboards-node-group name=<Dashboards_host_group_name>,` `resource-preset-id=<host_class>,` `disk-size=<disk_size_in_bytes>,` `disk-type-id=<network-ssd>,` `hosts-count=<number_of_hosts_in_group>,` `zone-ids=<availability_zones>,` `subnet-names=<subnet_names>,` `assign-public-ip=<allow_public_access_to_hosts>Where:
-
--labels: Yandex Cloud labels in<key>=<value>format. You can use them to logically separate resources. -
--environment: Environment:production: For stable versions of your apps.prestable: For testing purposes. The prestable environment is similar to the production environment and likewise covered by the SLA, but it is the first of the two to get new features, improvements, and bug fixes. In the prestable environment, you can test the new versions for compatibility with your application.
-
--service-account-name: Name of the service account for access to Yandex Object Storage as a repository of OpenSearch snapshots. For more information on service accounts, see the Yandex Identity and Access Management documentation. -
--deletion-protection: Cluster protection from accidental deletion,trueorfalse.Even with cluster deletion protection enabled, one can still delete a user or connect to the cluster manually and delete the data.
-
--maintenance: Maintenance window settings:- To allow maintenance at any time, do not specify the
--maintenanceparameter in the command (default configuration) or specify--maintenance schedule=anytime. - To specify the preferred start time for maintenance, specify this parameter in the command:
--maintenance schedule=weekly,weekday=<day_of_week>,hour=<hour_in_UTC>. In this case, maintenance will take place every week on a specified day at a specified time.
Both enabled and disabled clusters undergo maintenance. Maintenance may involve such operations as applying patches or updating DBMS's.
- To allow maintenance at any time, do not specify the
-
--disk-encryption-key-id: Disk encryption using a custom KMS key.To learn more about disk encryption, see Storage.
-
--read-admin-password:adminuser password. If you specify this parameter in the command, it will prompt you to enter a password.The password must contain three groups of characters out of these four:
- Lowercase Latin letters
- Uppercase Latin letters
- Numbers
- Special characters
The password must be from 10 to 72 characters long.
adminis a dedicated user required to manage the cluster and cannot be deleted. Such a user has thesuperuserrole and can perform any operations with the cluster.Tip
This user is not intended for routine jobs; for those, we recommend creating regular users. For more information, see Managing OpenSearch users.
-
--serverless-access: Access from Yandex Serverless Containers,trueorfalse. -
--plugins: OpenSearch plugins to install in the cluster. -
--advanced-params: Additional cluster parameters. The possible values are:max-clause-count: Maximum allowed number of boolean clauses per query. For more information, see this OpenSearch guide .fielddata-cache-size: JVM heap size allocated for thefielddatadata structure. You can specify either an absolute value or percentage, e.g.,512mbor50%. For more information, see the OpenSearch documentation .reindex-remote-whitelist: List of remote hosts whose indexes contain documents to copy for reindexing. Specify the parameter value as<host_address>:<port>. If you need to specify more than one host, list values separated by commas. For more information, see this OpenSearch guide .
-
--opensearch-node-group:OpenSearchhost group configuration, where:-
resource-preset-id: Host class that defines the configuration of virtual machines the OpenSearch nodes will be deployed on. All available options are listed under Host classes. -
disk-size: Disk size in bytes. The minimum and maximum values depend on the selected host class. -
disk-type-id: Disk type. -
zone-ids: Availability zones Separate zones with commas and enclose them in square brackets. Here is an example:zone-ids=[ru-central1-a,ru-central1-b,ru-central1-d] -
subnet-names: Name of the subnets in the specified availability zones. Separate subnets with commas and enclose them in square brackets. Here is an example:subnet-names=[default-ru-central1-a,default-ru-central1-b,default-ru-central1-d]You can specify the
subnet-idsparameter with network IDs instead ofsubnet-names. Separate IDs with commas and enclosed them in square brackets. Here is an example:subnet-ids=[e9bp8qmchqh2********,e2l963gkhobo********,fl8klaabecc3********] -
roles: Host roles. The possible values are as follows:data: Assigns theDATArole only.manager: Assigns theMANAGERrole only.data+managerormanager+data: Assigns both roles.
Tip
For security reasons, we do not recommend enabling public access to hosts with the
MANAGERrole.
-
-
--dashboards-node-group:Dashboardshost group configuration. It is configured in the same way as theOpenSearchhost group, except for the host roles. You do not need to configure any roles for theDashboardsgroup.
-
With Terraform
Terraform is distributed under the Business Source License
For more information about the provider resources, see the relevant documentation on the Terraform
If you do not have Terraform yet, install it and configure the Yandex Cloud provider.
To create a Managed Service for OpenSearch cluster:
-
In the configuration file, describe the properties of resources you want to create:
-
DB cluster: Description of the Managed Service for OpenSearch cluster and its hosts
-
Network: Description of the cloud network where a cluster will be located. If you already have a suitable network, you don't have to describe it again.
-
Subnets: Description of the subnets to connect the cluster hosts to. If you already have suitable subnets, you don't have to describe them again.
Here is an example of the configuration file structure:
resource "yandex_mdb_opensearch_cluster" "<cluster_name>" { name = "<cluster_name>" environment = "<environment>" network_id = "<network_ID>" security_group_ids = ["<list_of_security_group_IDs>"] disk_encryption_key_id = <KMS_key_ID> deletion_protection = "<protect_cluster_from_deletion>" config { version = "<OpenSearch_version>" admin_password = "<admin_user_password>" opensearch { node_groups { name = "<virtual_host_group_name>" assign_public_ip = <allow_public_access_to_hosts> hosts_count = <number_of_hosts> zone_ids = ["<list_of_availability_zones>"] subnet_ids = ["<list_of_subnet_IDs>"] roles = ["<role_list>"] resources { resource_preset_id = "<host_class>" disk_size = <storage_size_in_bytes> disk_type_id = "<disk_type>" } } plugins = ["<list_of_plugin_names>"] } dashboards { node_groups { name = "<virtual_host_group_name>" assign_public_ip = <allow_public_access_to_hosts> hosts_count = <number_of_hosts> zone_ids = ["<list_of_availability_zones>"] subnet_ids = ["<list_of_subnet_IDs>"] resources { resource_preset_id = "<host_class>" disk_size = <storage_size_in_bytes> disk_type_id = "<disk_type>" } } } } maintenance_window { type = <maintenance_type> day = <day_of_week> hour = <hour> } } resource "yandex_vpc_network" "<network_name>" { name = "<network_name>" } resource "yandex_vpc_subnet" "<subnet_name>" { name = "<subnet_name>" zone = "<availability_zone>" network_id = "<network_ID>" v4_cidr_blocks = ["<range>"] }Where:
-
environment: Environment,PRESTABLEorPRODUCTION. -
disk_encryption_key_id: Disk encryption with a custom KMS key.To learn more about disk encryption, see Storage.
-
deletion_protection: Cluster protection from accidental deletion,trueorfalse.Even with cluster deletion protection enabled, one can still delete a user or connect to the cluster manually and delete the data.
-
admin_password:adminuser password.The password must contain three groups of characters out of these four:
- Lowercase Latin letters
- Uppercase Latin letters
- Numbers
- Special characters
The password must be from 10 to 72 characters long.
adminis a dedicated user required to manage the cluster and cannot be deleted. Such a user has thesuperuserrole and can perform any operations with the cluster.Tip
This user is not intended for routine jobs; for those, we recommend creating regular users. For more information, see Managing OpenSearch users.
-
assign_public_ip: Public access to the host,trueorfalse. -
roles:DATAandMANAGERhost roles. -
maintenance_window: Maintenance window settings (including those for disabled clusters):type: Maintenance type. The possible values include:ANYTIME: Any time.WEEKLY: On a schedule.
day: Day of week inDDDformat for theWEEKLYtype, e.g.,MON.hour: Time of day (UTC) inHHformat for theWEEKLYtype, e.g.,21.
For a complete list of configurable Managed Service for OpenSearch cluster fields, refer to the Terraform provider guides.
-
-
Validate your configuration.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Create a cluster.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
Timeouts
The Terraform provider sets the following timeouts for Managed Service for OpenSearch cluster operations:
- Creating a cluster, including restoring from a backup: 30 minutes.
- Editing a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the
timeoutsblock to the cluster description, for example:resource "yandex_mdb_opensearch_cluster" "<cluster_name>" { ... timeouts { create = "1h30m" # 1 hour 30 minutes update = "2h" # 2 hours delete = "30m" # 30 minutes } } -
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Create a file named
body.jsonand paste the following code into it:{ "folderId": "<folder_ID>", "name": "<cluster_name>", "environment": "<environment>", "networkId": "<network_ID>", "securityGroupIds": [ "<security_group_1_ID>", "<security_group_2_ID>", ... "<security_group_N_ID>" ], "serviceAccountId": "<service_account_ID>", "deletionProtection": <protect_cluster_from_deletion>, "configSpec": { "version": "<OpenSearch_version>", "adminPassword": "<admin_password>", "opensearchSpec": { "plugins": [ "<OpenSearch_pugin_1>", "<OpenSearch_pugin_2>", ... "<OpenSearch_pugin_N>" ], "nodeGroups": [ { "name": "<host_group_name>", "resources": { "resourcePresetId": "<host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" }, "roles": ["<role_1>","<role_2>"], "hostsCount": "<number_of_hosts>", "zoneIds": [ "<availability_zone_1>", "<availability_zone_2>", "<availability_zone_3>" ], "subnetIds": [ "<subnet_1_ID>", "<subnet_2_ID>", "<subnet_3_ID>" ], "assignPublicIp": <allow_public_access_to_hosts>, "diskSizeAutoscaling": { "plannedUsageThreshold": "<scheduled_increase_percentage>", "emergencyUsageThreshold": "<immediate_increase_percentage>", "diskSizeLimit": "<maximum_storage_size_in_bytes>" } }, ... ] }, "dashboardsSpec": { "nodeGroups": [ { "name": "<host_group_name>", "resources": { "resourcePresetId": "<host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" }, "hostsCount": "<number_of_hosts>", "zoneIds": ["<availability_zone>"], "subnetIds": ["<subnet_ID>"], "assignPublicIp": <allow_public_access_to_hosts>, "diskSizeAutoscaling": { "plannedUsageThreshold": "<scheduled_increase_percentage>", "emergencyUsageThreshold": "<immediate_increase_percentage>", "diskSizeLimit": "<maximum_storage_size_in_bytes>" } } ] }, "access": { "dataTransfer": <allow_access_from_Data_Transfer>, "serverless": <allow_access_from_Serverless_Containers> } }, "maintenanceWindow": { "weeklyMaintenanceWindow": { "day": "<day_of_week>", "hour": "<hour>" } } }Where:
-
folderId: Folder ID. You can get it from the cloud’s folder list. -
name: Cluster name. -
environment: Cluster environment,PRODUCTIONorPRESTABLE. -
networkId: ID of the network the cluster will be in. -
securityGroupIds: Security group IDs. -
serviceAccountId: ID of the service account used for cluster operations. -
deletionProtection: Cluster protection from accidental deletion,trueorfalse.Even with cluster deletion protection enabled, one can still delete a user or connect to the cluster manually and delete the data.
-
configSpec: Cluster settings:-
version: OpenSearch version. -
adminPassword:adminuser password.The password must contain three groups of characters out of these four:
- Lowercase Latin letters
- Uppercase Latin letters
- Numbers
- Special characters
The password must be from 10 to 72 characters long.
adminis a dedicated user required to manage the cluster and cannot be deleted. Such a user has thesuperuserrole and can perform any operations with the cluster.Tip
This user is not intended for routine jobs; for those, we recommend creating regular users. For more information, see Managing OpenSearch users.
-
opensearchSpec:OpenSearchhost group settings:-
plugins: List of OpenSearch plugins you should additionally install in the cluster. -
nodeGroups: Host settings as an array of elements, one for each host group. Each element has the following structure:-
name: Host group name. -
resources: Cluster resources:resourcePresetId: Host class.diskSize: Disk size in bytes.diskTypeId: Disk type.
-
roles: List of host roles. A cluster must include at least one group ofDATAhosts and one group ofMANAGERhosts. This can be a single group with two roles or several groups with different roles. -
hostsCount: Number of hosts in the group. Minimum number ofDATAhosts: one; minimum number ofMANAGERhosts: three. -
zoneIds: List of availability zones the cluster hosts are located in. -
subnetIds: List of subnet IDs. -
assignPublicIp: Permission to connect to the host from the internet,trueorfalse. -
diskSizeAutoscaling: Automatic storage size increase settings:-
plannedUsageThreshold: Storage utilization percentage to trigger a storage increase during the next maintenance window.Use a value between
0and100%. The default setting is0(automatic increase disabled).If you set this condition, configure the maintenance window schedule in the
maintenanceWindowparameter. -
emergencyUsageThreshold: Storage utilization percentage to trigger an immediate storage increase.Use a value between
0and100%. The default setting is0(automatic increase disabled).Warning
If you specify both thresholds,
emergencyUsageThresholdmust not be less thanplannedUsageThreshold. -
diskSizeLimit: Maximum storage size, in bytes, that can be set when utilization reaches one of the specified percentages.
Warning
- You cannot decrease the storage size.
- While resizing the storage, cluster hosts will be unavailable.
-
-
-
-
dashboardsSpec: Settings forDashboardshost groups. Contains thenodeGroupsparameter of the same structure asopensearchSpec.nodeGroups. Therolesparameter is the exception: theDashboardshosts can only have one role,DASHBOARDS, so there is no need to specify it. -
access: Settings for cluster access to the following Yandex Cloud services:dataTransfer: Yandex Data Transferserverless: Yandex Serverless Containers
The possible setting values are
trueorfalse.
-
-
maintenance_window.weeklyMaintenanceWindow: Maintenance window schedule:day: Day of the week, inDDDformat, for scheduled maintenance.hour: Hour, inHHformat, for scheduled maintenance. The possible values range from1to24. Use the UTC time zone.
-
-
Use the Cluster.Create method and send the following request, e.g., via cURL
:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-opensearch/v1/clusters' \ --data "@body.json" -
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Create a file named
body.jsonand paste the following code into it:{ "folder_id": "<folder_ID>", "name": "<cluster_name>", "environment": "<environment>", "network_id": "<network_ID>", "security_group_ids": [ "<security_group_1_ID>", "<security_group_2_ID>", ... "<security_group_N_ID>" ], "service_account_id": "<service_account_ID>", "deletion_protection": <protect_cluster_from_deletion>, "config_spec": { "version": "<OpenSearch_version>", "admin_password": "<admin_password>", "opensearch_spec": { "plugins": [ "<OpenSearch_pugin_1>", "<OpenSearch_pugin_2>", ... "<OpenSearch_pugin_N>" ], "node_groups": [ { "name": "<host_group_name>", "resources": { "resource_preset_id": "<host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" }, "roles": ["<role_1>","<role_2>"], "hosts_count": "<number_of_hosts>", "zone_ids": [ "<availability_zone_1>", "<availability_zone_2>", "<availability_zone_3>" ], "subnet_ids": [ "<subnet_1_ID>", "<subnet_2_ID>", "<subnet_3_ID>" ], "assign_public_ip": <allow_public_access_to_hosts>, "disk_size_autoscaling": { "planned_usage_threshold": "<scheduled_increase_percentage>", "emergency_usage_threshold": "<immediate_increase_percentage>", "disk_size_limit": "<maximum_storage_size_in_bytes>" } }, ... ] }, "dashboards_spec": { "node_groups": [ { "name": "<host_group_name>", "resources": { "resource_preset_id": "<host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" }, "hosts_count": "<number_of_hosts>", "zone_ids": ["<availability_zone>"], "subnet_ids": ["<subnet_ID>"], "assign_public_ip": <allow_public_access_to_hosts>, "disk_size_autoscaling": { "planned_usage_threshold": "<scheduled_increase_percentage>", "emergency_usage_threshold": "<immediate_increase_percentage>", "disk_size_limit": "<maximum_storage_size_in_bytes>" } } ] }, "access": { "data_transfer": <allow_access_from_Data_Transfer>, "serverless": <allow_access_from_Serverless_Containers> } }, "maintenance_window": { "weekly_maintenance_window": { "day": "<day_of_week>", "hour": "<hour>" } } }Where:
-
folder_id: Folder ID. You can request it with the list of folders in the cloud. -
name: Cluster name. -
environment: Cluster environment,PRODUCTIONorPRESTABLE. -
network_id: ID of the network the cluster will be in. -
security_group_ids: Security group IDs. -
service_account_id: ID of the service account used for cluster operations. -
deletion_protection: Cluster protection from accidental deletion,trueorfalse.Even with cluster deletion protection enabled, one can still delete a user or connect to the cluster manually and delete the data.
-
config_spec: Cluster settings:-
version: OpenSearch version. -
admin_password:adminuser password.The password must contain three groups of characters out of these four:
- Lowercase Latin letters
- Uppercase Latin letters
- Numbers
- Special characters
The password must be from 10 to 72 characters long.
adminis a dedicated user required to manage the cluster and cannot be deleted. Such a user has thesuperuserrole and can perform any operations with the cluster.Tip
This user is not intended for routine jobs; for those, we recommend creating regular users. For more information, see Managing OpenSearch users.
-
opensearch_spec:OpenSearchhost group settings:-
plugins: List of OpenSearch plugins you should additionally install in the cluster. -
node_groups: Host settings as an array of elements, one for each host group. Each element has the following structure:-
name: Host group name. -
resources: Cluster resources:resource_preset_id: Host class.disk_size: Disk size in bytes.disk_type_id: Disk type.
-
roles: List of host roles. A cluster must include at least one group ofDATAhosts and one group ofMANAGERhosts. This can be a single group with two roles or several groups with different roles. -
hosts_count: Number of hosts in the group. Minimum number ofDATAhosts: one; minimum number ofMANAGERhosts: three. -
zone_ids: List of availability zones the cluster hosts are located in. -
subnet_ids: List of subnet IDs. -
assign_public_ip: Permission to connect to the host from the internet,trueorfalse. -
disk_size_autoscaling: Automatic storage size increase settings:-
planned_usage_threshold: Storage utilization percentage to trigger a storage increase during the next maintenance window.Use a value between
0and100%. The default setting is0(automatic increase disabled).If you set this condition, configure the maintenance window schedule in the
maintenance_windowparameter. -
emergency_usage_threshold: Storage utilization percentage to trigger an immediate storage increase.Use a value between
0and100%. The default setting is0(automatic increase disabled).Warning
If you specify both thresholds,
emergency_usage_thresholdmust not be less thanplanned_usage_threshold. -
disk_size_limit: Maximum storage size, in bytes, that can be set when utilization reaches one of the specified percentages.
Warning
- You cannot decrease the storage size.
- While resizing the storage, cluster hosts will be unavailable.
-
-
-
-
dashboards_spec: Settings forDashboardshost groups. Contains thenode_groupsparameter of the same structure asopensearch_spec.node_groups. Therolesparameter is the exception: theDashboardshosts can only have one role,DASHBOARDS, so there is no need to specify it. -
access: Settings for cluster access to the following Yandex Cloud services:data_transfer: Yandex Data Transferserverless: Yandex Serverless Containers
The possible setting values are
trueorfalse.
-
-
maintenance_window.weekly_maintenance_window: Maintenance window schedule:day: Day of the week, inDDDformat, for scheduled maintenance.hour: Hour, inHHformat, for scheduled maintenance. The possible values range from1to24. Use the UTC time zone.
-
-
Use the ClusterService.Create call to execute the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/opensearch/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d @ \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.opensearch.v1.ClusterService.Create \ < body.json -
Check the server response to make sure your request was successful.
Creating a cluster copy
You can create an OpenSearch cluster with the settings of another one you previously created. To do this, import the OpenSearch source cluster configuration to Terraform. This way you can either create an identical copy or use the imported configuration as the baseline and modify it as needed. Import is convenient when the source OpenSearch cluster has many settings and you need to create a similar cluster.
To create an OpenSearch cluster copy:
-
If you do not have Terraform yet, install it.
-
Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.
-
Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it
. -
Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.
-
In your current working directory, create a
.tffile with the following content:resource "yandex_mdb_opensearch_cluster" "old" { } -
Save the ID of the original OpenSearch cluster to an environment variable:
export OPENSEARCH_CLUSTER_ID=<cluster_ID>You can get the ID from the folder’s cluster list.
-
Import the original OpenSearch cluster settings to the Terraform configuration:
terraform import yandex_mdb_opensearch_cluster.old ${OPENSEARCH_CLUSTER_ID} -
Display the imported configuration:
terraform show -
Copy it from the terminal and paste it into the
.tffile. -
Create a new directory named
imported-clusterand move your file there. -
Modify the configuration so it can be used to create a new cluster:
- Specify the new cluster name in the
resourcestring and thenameparameter. - Delete
created_at,health,id, andstatus. - Add the
admin_passwordparameter to theconfigsection. - If the
maintenance_windowsection hastype = "ANYTIME", delete thehourparameter. - Optionally, you can customize the configuration further if needed.
- Specify the new cluster name in the
-
Get the authentication credentials in the
imported-clusterdirectory. -
In the same directory, configure and initialize a provider. To avoid creating the provider configuration file manually, you can download it here
. -
Move the configuration file to the
imported-clusterdirectory and edit it to include your required values. If you have not added your authentication credentials to the environment variables, specify them in the configuration file. -
Validate your Terraform configuration files:
terraform validateTerraform will display any configuration errors detected in your files.
-
Create the required infrastructure:
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console
. -
Timeouts
The Terraform provider sets the following timeouts for Managed Service for OpenSearch cluster operations:
- Creating a cluster, including restoring from a backup: 30 minutes.
- Editing a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_opensearch_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
Examples
Create a Managed Service for OpenSearch cluster with the following test specifications:
-
Name:
my-os-clstr. -
Description:
My OS cluster. -
Label:
label-keywithlabel-value. -
Environment:
production. -
Network name:
default. -
Security group ID:
enp6saqnq4ie244g67sb. -
Service account name:
os-account. -
Deletion protection: Disabled.
-
Maintenance time: Every Monday from 13:00 till 14:00.
-
OpenSearch version:
2.12. -
adminuser password: Specified after entering the cluster create command. -
Access to Data Transfer: Enabled.
-
Access to Serverless Containers: Enabled.
-
OpenSearch added plugin: analysis-icu.
-
OpenSearch additional parameter:
fielddata-cache-size=50%. -
OpenSearchnode group configuration:- Node group name:
os-group. - Host class:
s2.micro. - Disk size:
10737418240(in bytes). - Disk type:
network-ssd. - Number of hosts:
3. - Availability zone:
ru-central1-a. - Subnet:
default-ru-central1-a. - Public address: Assigned.
- Host group roles:
DATAandMANAGER.
- Node group name:
-
Dashboardshost group configuration:- Host group name:
dashboard-group. - Host class:
s2.micro. - Disk size:
10737418240(in bytes). - Disk type:
network-ssd. - Number of hosts:
1. - Availability zone:
ru-central1-a. - Subnet:
default-ru-central1-a. - Public address: Assigned.
- Host group name:
Run this command:
yc managed-opensearch cluster create \
--name my-os-clstr \
--description "My OS cluster" \
--labels label-key=label-value \
--environment production \
--network-name default \
--security-group-ids enp6saqnq4ie244g67sb \
--service-account-name os-account \
--delete-protection \
--maintenance schedule=weekly,`
`weekday=mon,`
`hour=14 \
--version 2.12 \
--read-admin-password \
--data-transfer-access=true \
--serverless-access=true \
--plugins analysis-icu \
--advanced-params fielddata-cache-size=50% \
--opensearch-node-group name=os-group,`
`resource-preset-id=s2.micro,`
`disk-size=10737418240,`
`disk-type-id=network-ssd,`
`hosts-count=3,`
`zone-ids=ru-central1-a,`
`subnet-names=default-ru-central1-a,`
`assign-public-ip=true,`
`roles=data+manager \
--dashboards-node-group name=dashboard-group,`
`resource-preset-id=s2.micro,`
`disk-size=10737418240,`
`disk-type-id=network-ssd,`
`hosts-count=1,`
`zone-ids=ru-central1-a,`
`subnet-names=default-ru-central1-a,`
`assign-public-ip=true
Create a Managed Service for OpenSearch cluster with the following test specifications:
-
Name:
my-os-clstr. -
Environment:
PRODUCTION. -
OpenSearch version:
2.12. -
adminuser password:osAdminpwd1. -
OpenSearchnode group configuration:OpenSearchnode group name:os-group.- Host class:
s2.micro. - Disk size:
10737418240(in bytes). - Disk type:
network-ssd. - Number of hosts:
3. - Availability zone:
ru-central1-a. - Public address: Assigned.
- Host group roles:
DATAandMANAGER.
-
Dashboardshost group configuration:- Host group name:
dashboard-group. - Host class:
s2.micro. - Disk size:
10737418240(in bytes). - Disk type:
network-ssd. - Number of hosts:
1. - Availability zone:
ru-central1-a. - Public address: Assigned.
- Host group name:
-
Maintenance time: Every Monday from 13:00 till 14:00.
-
Network name:
mynet. -
Subnet name:
mysubnet. -
Availability zone:
ru-central1-a. -
Address range:
10.1.0.0/16. -
Security group name:
os-sg. The security group enables connecting to the cluster host from any network (including the internet) on port9200.
The configuration file for this cluster looks like this:
resource "yandex_mdb_opensearch_cluster" "my-os-clstr" {
name = "my-os-clstr"
environment = "PRODUCTION"
network_id = yandex_vpc_network.mynet.id
security_group_ids = [yandex_vpc_security_group.os-sg.id]
config {
version = "2.12"
admin_password = "osAdminpwd1"
opensearch {
node_groups {
name = "os-group"
assign_public_ip = true
hosts_count = 3
zone_ids = ["ru-central1-a"]
subnet_ids = [yandex_vpc_subnet.mysubnet.id]
roles = ["DATA", "MANAGER"]
resources {
resource_preset_id = "s2.micro"
disk_size = 10737418240
disk_type_id = "network-ssd"
}
}
}
dashboards {
node_groups {
name = "dashboard-group"
assign_public_ip = true
hosts_count = 1
zone_ids = ["ru-central1-a"]
subnet_ids = [yandex_vpc_subnet.mysubnet.id]
resources {
resource_preset_id = "s2.micro"
disk_size = 10737418240
disk_type_id = "network-ssd"
}
}
}
}
maintenance_window {
type = "WEEKLY"
day = "MON"
hour = 14
}
}
resource "yandex_vpc_network" "mynet" {
name = "mynet"
}
resource "yandex_vpc_subnet" "mysubnet" {
name = "mysubnet"
zone = "ru-central1-a"
network_id = yandex_vpc_network.mynet.id
v4_cidr_blocks = ["10.1.0.0/16"]
}
resource "yandex_vpc_security_group" "os-sg" {
name = "os-sg"
network_id = yandex_vpc_network.mynet.id
ingress {
description = "Allow connections to the Managed Service for OpenSearch cluster from the Internet"
protocol = "TCP"
port = 9200
v4_cidr_blocks = ["0.0.0.0/0"]
}
egress {
description = "The rule allows all outgoing traffic"
protocol = "ANY"
v4_cidr_blocks = ["0.0.0.0/0"]
from_port = 0
to_port = 65535
}
}
Managing database connection parameters using Connection Manager
If your cloud or folder has access to Connection Manager public preview, a new connection entity will appear in your folder after you create a cluster. You can use it to manage database connection parameters.
Passwords and other sensitive data will be stored in a Yandex Lockbox secret. To see which secrets store connection information for your cluster, select Lockbox in the list of services in your folder. You will find you cluster's ID on the Secrets page in the secret dependencies column.
You can also use Connection Manager to configure access to connections.