Creating an OpenSearch cluster
A Managed Service for OpenSearch cluster is a group of multiple interlinked OpenSearch and DashboardsDATA role. To learn more about roles in the cluster, see Host roles.
The available disk types depend on the selected host class.
For more information, see Resource relationships.
Roles for creating a cluster
To create a Managed Service for OpenSearch cluster, you need the vpc.user role and the managed-opensearch.editor role or higher.
To attach your service account to a cluster, e.g., to use Yandex Object Storage, make sure your Yandex Cloud account has the iam.serviceAccounts.user role or higher.
For more information about assigning roles, see this Yandex Identity and Access Management guide.
Creating a cluster
When creating a cluster, you need to specify individual parameters for each host group.
To create a Managed Service for OpenSearch cluster:
-
In the management console
, select the folder where you want to create a cluster. -
Navigate to the Managed Service for OpenSearch service.
-
Click Create cluster.
-
Under Basic parameters:
-
Enter a name for the cluster. It must be unique within the folder.
-
Optionally, enter a description for the cluster.
-
Select the environment where you want to create your cluster (you cannot change the environment once the cluster is created):
PRODUCTION: For stable versions of your applications.PRESTABLE: For testing purposes. The prestable environment is similar to the production environment and likewise covered by an SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test new versions for compatibility with your application.
-
Select the OpenSearch version.
-
Select the plugins you want to install in the cluster.
-
-
Under Network settings, select the cloud network to host your cluster and security groups for cluster network traffic. You may need to additionally set up security groups to be able connect to the cluster.
-
Under Virtual node group 1, configure your
OpenSearchhost group:-
Select the host group type:
OpenSearch. -
Enter a name for the host group. It must be unique within the cluster.
-
Select the
DATAandMANAGERhost roles. -
Select the platform, host type, and host class.
The host class defines the specifications of virtual machines OpenSearch nodes will be deployed on. All available options are listed under Host classes.
-
Select the disk type and data storage size.
The selected type determines the increments in which you can change your disk size:
- Network HDD and SSD storage: In increments of 1 GB.
- Local SSD storage:
- For Intel Cascade Lake: In increments of 100 GB.
- For Intel Ice Lake: In increments of 368 GB.
- Non-replicated SSD storage: In increments of 93 GB.
-
Optionally, select Encrypted disk to encrypt the disk with a custom KMS key.
-
To create a new key, click Create.
-
To use the key you created earlier, select it in the KMS key field.
To learn more about disk encryption, see Storage.
-
-
Optionally, set up automatic storage expansion:
-
In the Increase size field, specify the conditions for the actions below:
-
Storage expansion during the next maintenance window once the fill level exceeds the specified percentage. If you set this condition, configure the maintenance window schedule.
-
Storage expansion immediately once the fill level exceeds the specified percentage.
Warning
If you set both conditions, make sure the immediate expansion threshold is higher than the scheduled one.
-
-
Specify the maximum storage size after the expansion in the Maximum storage size field.
Warning
- You cannot decrease the storage size.
- When scaling your storage, the cluster hosts will be unavailable.
-
-
Specify how hosts should be distributed across availability zones and subnets.
-
Select the number of hosts to create.
-
Enable Public access if you want to allow connections to hosts over the internet.
Tip
For security reasons, we do not recommend enabling public access for hosts with the
MANAGERrole.
-
-
Configure the
Dashboardshost group under Virtual node group 2, if required:-
Select the platform, host type, and host class.
-
Configure the storage the same way as for
OpenSearchhosts. -
Specify how hosts should be distributed across availability zones and subnets.
-
Select the number of hosts to create.
-
Enable Public access if you want to allow connections to hosts over the internet.
Tip
You can use OpenSearch Dashboards even if you cannot get public access to your hosts, e.g., for security reasons. To do this, proxy the connections via the virtual machine in Yandex Compute Cloud that resides in the same network as the cluster. For more information, see Connecting to OpenSearch Dashboards.
-
-
If required, click Add virtual node group to add another host group or more.
-
Under Service settings:
-
Enter the password for
admin.The password must include three groups of characters out of these four:
- Lowercase Latin letters
- Uppercase Latin letters
- Numbers
- Special characters
The password must be from 10 to 72 characters long.
adminis a dedicated user required to manage the cluster and cannot be deleted. Such a user has thesuperuserrole and can perform any operations with the cluster.Tip
However, for routine jobs, we recommend creating individual regular users. For more information, see Managing OpenSearch users.
-
If required, change additional cluster settings:
-
Maintenance window: Maintenance window settings:
- To enable maintenance at any time, select arbitrary (default).
- To specify the preferred maintenance start time, select by schedule and specify the desired day of the week and UTC hour. For example, you can choose a time when the cluster is least loaded.
Maintenance operations are carried out both on enabled and disabled clusters. They may include updating the DBMS, applying patches, and so on.
-
Service account: Account for accessing Yandex Object Storage as a repository of OpenSearch snapshots. Learn more about service accounts in this Yandex Identity and Access Management guide.
-
Deletion protection: Manages cluster protection against accidental deletion.
Even with cluster deletion protection enabled, it is still possible to delete a user or connect to the cluster manually and delete the data.
-
-
-
Click Create cluster.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To create a Managed Service for OpenSearch cluster:
-
View the description of the CLI command for creating a cluster:
yc managed-opensearch cluster create --help -
Specify the cluster properties in this command (the example does not show all that are available):
yc managed-opensearch cluster create \ --name <cluster_name> \ --description <cluster_description> \ --labels <labels> \ --environment <environment> \ --network-name <network_name> \ --security-group-ids <security_group_IDs> \ --service-account-name <service_account_name> \ --delete-protection \ --maintenance schedule=<maintenance_type>,` `weekday=<day_of_week>,` `hour=<hour> \ --disk-encryption-key-id <KMS_key_ID> \ --version <OpenSearch_version> \ --read-admin-password \ --data-transfer-access=<allow_access_from_Data_Transfer> \ --serverless-access=<allow_access_from_Serverless_Containers> \ --plugins <OpenSearch_plugins> \ --advanced-params <additional_parameters> \ --opensearch-node-group name=<OpenSearch_host_group_name>,` `resource-preset-id=<host_class>,` `disk-size=<disk_size_in_bytes>,` `disk-type-id=<network-hdd|network-ssd|network-ssd-io-m3|network-ssd-nonreplicated|local-ssd>,` `hosts-count=<number_of_hosts_in_group>,` `zone-ids=<availability_zones>,` `subnet-names=<subnet_names>,` `assign-public-ip=<allow_public_access_to_hosts>,` `roles=<host_roles> \ --dashboards-node-group name=<Dashboards_host_group_name>,` `resource-preset-id=<host_class>,` `disk-size=<disk_size_in_bytes>,` `disk-type-id=<network-ssd>,` `hosts-count=<number_of_hosts_in_group>,` `zone-ids=<availability_zones>,` `subnet-names=<subnet_names>,` `assign-public-ip=<allow_public_access_to_hosts>Where:
-
--labels: Yandex Cloud labels in<key>=<value>format. You can use them to logically separate resources. -
--environment: Environment:production: For stable versions of your applications.prestable: For testing purposes. The prestable environment is similar to the production environment and likewise covered by an SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test new versions for compatibility with your application.
-
--service-account-name: Name of the service account for accessing Yandex Object Storage as a repository of OpenSearch snapshots. Learn more about service accounts in this Yandex Identity and Access Management guide. -
--deletion-protection: Cluster protection from accidental deletion,trueorfalse.Even with cluster deletion protection enabled, it is still possible to delete a user or connect to the cluster manually and delete the data.
-
--maintenance: Maintenance window settings:- To allow maintenance at any time, do not specify the
--maintenanceparameter in the command (the default configuration) or specify--maintenance schedule=anytime. - To specify the preferred start time for maintenance, specify this parameter in the command:
--maintenance schedule=weekly,weekday=<day_of_week>,hour=<hour_in_UTC>. In this case, maintenance will take place every week on a specified day at a specified time.
Both active and stopped clusters undergo maintenance. Maintenance may involve such operations as applying patches or updating DBMS's.
- To allow maintenance at any time, do not specify the
-
--disk-encryption-key-id: Disk encryption with a custom KMS key.To learn more about disk encryption, see Storage.
-
--read-admin-password:adminpassword. If you specify this parameter in the command, it will prompt you to enter a password.The password must include three groups of characters out of these four:
- Lowercase Latin letters
- Uppercase Latin letters
- Numbers
- Special characters
The password must be from 10 to 72 characters long.
adminis a dedicated user required to manage the cluster and cannot be deleted. Such a user has thesuperuserrole and can perform any operations with the cluster.Tip
However, for routine jobs, we recommend creating individual regular users. For more information, see Managing OpenSearch users.
-
--serverless-access: Access from Yandex Serverless Containers,trueorfalse. -
--plugins: OpenSearch plugins to install in the cluster. -
--advanced-params: Additional cluster parameters. The possible values are:max-clause-count: Maximum allowed number of boolean clauses per query. For more information, see this OpenSearch guide .fielddata-cache-size: JVM heap size allocated for thefielddatadata structure. You can specify either an absolute value or percentage, e.g.,512mbor50%. For more information, see this OpenSearch guide .reindex-remote-whitelist: List of remote hosts whose indexes contain documents to copy for reindexing. Specify the parameter value as<host_address>:<port>. If you need to specify more than one host, list values separated by commas. For more information, see this OpenSearch guide .
-
--opensearch-node-group:OpenSearchhost group configuration, where:-
resource-preset-id: Host class that defines the specifications of virtual machines the OpenSearch nodes will be deployed on. All available options are listed under Host classes. -
disk-size: Disk size, in bytes. The minimum and maximum values depend on the selected host class. -
disk-type-id: Disk type. -
zone-ids: Availability zones. Separate zones with commas and enclose them in square brackets, e.g.:zone-ids=[ru-central1-a,ru-central1-b,ru-central1-d] -
subnet-names: Names of the subnets in the specified availability zones. Separate subnets with commas and enclose them in square brackets, e.g.:subnet-names=[default-ru-central1-a,default-ru-central1-b,default-ru-central1-d]You can specify the
subnet-idsparameter with network IDs instead ofsubnet-names. Separate IDs with commas and enclosed them in square brackets, e.g.:subnet-ids=[e9bp8qmchqh2********,e2l963gkhobo********,fl8klaabecc3********] -
roles: Host roles. The possible values are:data: Assigns theDATArole only.manager: Assigns theMANAGERrole only.data+managerormanager+data: Assigns both roles.
Tip
For security reasons, we do not recommend enabling public access to hosts with the
MANAGERrole.
-
-
--dashboards-node-group:Dashboardshost group configuration. It is configured just as theOpenSearchhost group, except for the host roles. You do not need to set up any roles for theDashboardsgroup.
-
With Terraform
Terraform is distributed under the Business Source License
For more information about the provider resources, see the relevant documentation on the Terraform
If you do not have Terraform yet, install it and configure the Yandex Cloud provider.
To create a Managed Service for OpenSearch cluster:
-
In the configuration file, describe the resources you want to create:
-
Database cluster: Description of the Managed Service for OpenSearch cluster and its hosts.
-
Network: Description of the cloud network where a cluster will be located. If you already have a suitable network, you don't have to describe it again.
-
Subnets: Description of the subnets to connect the cluster hosts to. If you already have suitable subnets, you don't have to describe them again.
Here is an example of the configuration file structure:
resource "yandex_mdb_opensearch_cluster" "<cluster_name>" { name = "<cluster_name>" environment = "<environment>" network_id = "<network_ID>" security_group_ids = ["<list_of_security_group_IDs>"] disk_encryption_key_id = <KMS_key_ID> deletion_protection = "<protect_cluster_against_deletion>" config { version = "<OpenSearch_version>" admin_password = "<admin_password>" opensearch { node_groups { name = "<virtual_host_group_name>" assign_public_ip = <allow_public_access_to_hosts> hosts_count = <number_of_hosts> zone_ids = ["<list_of_availability_zones>"] subnet_ids = ["<list_of_subnet_IDs>"] roles = ["<role_list>"] resources { resource_preset_id = "<host_class>" disk_size = <storage_size_in_bytes> disk_type_id = "<disk_type>" } } plugins = ["<list_of_plugin_names>"] } dashboards { node_groups { name = "<virtual_host_group_name>" assign_public_ip = <allow_public_access_to_hosts> hosts_count = <number_of_hosts> zone_ids = ["<list_of_availability_zones>"] subnet_ids = ["<list_of_subnet_IDs>"] resources { resource_preset_id = "<host_class>" disk_size = <storage_size_in_bytes> disk_type_id = "<disk_type>" } } } } maintenance_window { type = <maintenance_type> day = <day_of_week> hour = <hour> } } resource "yandex_vpc_network" "<network_name>" { name = "<network_name>" } resource "yandex_vpc_subnet" "<subnet_name>" { name = "<subnet_name>" zone = "<availability_zone>" network_id = "<network_ID>" v4_cidr_blocks = ["<range>"] }Where:
-
environment: Environment,PRESTABLEorPRODUCTION. -
disk_encryption_key_id: Disk encryption with a custom KMS key.To learn more about disk encryption, see Storage.
-
deletion_protection: Cluster protection against accidental deletion,trueorfalse.Even with cluster deletion protection enabled, it is still possible to delete a user or connect to the cluster manually and delete the data.
-
admin_password:adminpassword.The password must include three groups of characters out of these four:
- Lowercase Latin letters
- Uppercase Latin letters
- Numbers
- Special characters
The password must be from 10 to 72 characters long.
adminis a dedicated user required to manage the cluster and cannot be deleted. Such a user has thesuperuserrole and can perform any operations with the cluster.Tip
However, for routine jobs, we recommend creating individual regular users. For more information, see Managing OpenSearch users.
-
assign_public_ip: Public access to the host,trueorfalse. -
roles: Host roles,DATAandMANAGER. -
maintenance_window: Maintenance window (for stopped clusters as well):type: Maintenance type. The possible values include:ANYTIME: At any time.WEEKLY: On schedule.
day: Day of week inDDDformat for theWEEKLYtype, e.g.,MON.hour: Time of day (UTC) inHHformat for theWEEKLYtype, e.g.,21.
For a complete list of Managed Service for OpenSearch cluster configuration fields you can update, see this Terraform provider guide.
-
-
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Create a cluster.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
Timeouts
The Terraform provider sets the following timeouts for Managed Service for OpenSearch cluster operations:
- Creating a cluster, including by restoring it from a backup: 30 minutes.
- Updating a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the timeout are aborted.
How do I change these limits?
Add a
timeoutssection to the cluster description, e.g.:resource "yandex_mdb_opensearch_cluster" "<cluster_name>" { ... timeouts { create = "1h30m" # 1 hour 30 minutes update = "2h" # 2 hours delete = "30m" # 30 minutes } } -
-
Get an IAM token for API authentication and put it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Create a file named
body.jsonand paste the following code into it:{ "folderId": "<folder_ID>", "name": "<cluster_name>", "environment": "<environment>", "networkId": "<network_ID>", "securityGroupIds": [ "<security_group_1_ID>", "<security_group_2_ID>", ... "<security_group_N_ID>" ], "serviceAccountId": "<service_account_ID>", "deletionProtection": <protect_cluster_against_deletion>, "configSpec": { "version": "<OpenSearch_version>", "adminPassword": "<admin_password>", "opensearchSpec": { "plugins": [ "<OpenSearch_plugin_1>", "<OpenSearch_plugin_2>", ... "<OpenSearch_plugin_N>" ], "nodeGroups": [ { "name": "<host_group_name>", "resources": { "resourcePresetId": "<host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" }, "roles": ["<role_1>","<role_2>"], "hostsCount": "<number_of_hosts>", "zoneIds": [ "<availability_zone_1>", "<availability_zone_2>", "<availability_zone_3>" ], "subnetIds": [ "<subnet_1_ID>", "<subnet_2_ID>", "<subnet_3_ID>" ], "assignPublicIp": <allow_public_access_to_hosts>, "diskSizeAutoscaling": { "plannedUsageThreshold": "<scheduled_expansion_percentage>", "emergencyUsageThreshold": "<immediate_expansion_percentage>", "diskSizeLimit": "<maximum_storage_size_in_bytes>" } }, ... ] }, "dashboardsSpec": { "nodeGroups": [ { "name": "<host_group_name>", "resources": { "resourcePresetId": "<host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" }, "hostsCount": "<number_of_hosts>", "zoneIds": ["<availability_zone>"], "subnetIds": ["<subnet_ID>"], "assignPublicIp": <allow_public_access_to_hosts>, "diskSizeAutoscaling": { "plannedUsageThreshold": "<scheduled_expansion_percentage>", "emergencyUsageThreshold": "<immediate_expansion_percentage>", "diskSizeLimit": "<maximum_storage_size_in_bytes>" } } ] }, "access": { "dataTransfer": <allow_access_from_Data_Transfer>, "serverless": <allow_access_from_Serverless_Containers> } }, "maintenanceWindow": { "weeklyMaintenanceWindow": { "day": "<day_of_week>", "hour": "<hour>" } } }Where:
-
folderId: Folder ID. You can get it with the list of folders in the cloud. -
name: Cluster name. -
environment: Cluster environment,PRODUCTIONorPRESTABLE. -
networkId: ID of the network where the cluster will be deployed. -
securityGroupIds: Security group IDs. -
serviceAccountId: ID of the service account used for cluster operations. -
deletionProtection: Cluster protection against accidental deletion,trueorfalse.Even with cluster deletion protection enabled, it is still possible to delete a user or connect to the cluster manually and delete the data.
-
configSpec: Cluster settings:-
version: OpenSearch version. -
adminPassword:adminpassword.The password must include three groups of characters out of these four:
- Lowercase Latin letters
- Uppercase Latin letters
- Numbers
- Special characters
The password must be from 10 to 72 characters long.
adminis a dedicated user required to manage the cluster and cannot be deleted. Such a user has thesuperuserrole and can perform any operations with the cluster.Tip
However, for routine jobs, we recommend creating individual regular users. For more information, see Managing OpenSearch users.
-
opensearchSpec:OpenSearchhost group settings:-
plugins: List of OpenSearch plugins to additionally install in the cluster. -
nodeGroups: Host settings as an array of elements, one per host group. Each element has the following structure:-
name: Host group name. -
resources: Cluster resources:resourcePresetId: Host class.diskSize: Disk size, in bytes.diskTypeId: Disk type.
-
roles: List of host roles. A cluster must include at least one group ofDATAhosts and one group ofMANAGERhosts. This can be a single group with two roles or multiple groups with different roles. -
hostsCount: Number of hosts in the group. The minimum number ofDATAhosts is one, while the minimum number ofMANAGERhosts is three. -
zoneIds: List of availability zones the cluster hosts are located in. -
subnetIds: List of subnet IDs. -
assignPublicIp: Permission to connect to the host from the internet,trueorfalse. -
diskSizeAutoscaling: Automatic storage expansion settings:-
plannedUsageThreshold: Storage usage percentage to trigger a storage expansion during the next maintenance window.Use a value between
0and100%. The default value is0, i.e., automatic expansion is disabled.If you set this condition, configure the maintenance window schedule in the
maintenanceWindowparameter. -
emergencyUsageThreshold: Storage usage percentage to trigger an immediate storage expansion.Use a value between
0and100%. The default value is0, i.e., automatic expansion is disabled.Warning
If you specify both thresholds,
emergencyUsageThresholdmust not be less thanplannedUsageThreshold. -
diskSizeLimit: Maximum storage size, in bytes, to set when storage usage reaches one of the specified thresholds.
Warning
- You cannot decrease the storage size.
- When scaling your storage, the cluster hosts will be unavailable.
-
-
-
-
dashboardsSpec:Dashboardshost group settings that contain thenodeGroupsparameter of the same structure asopensearchSpec.nodeGroups, except for therolesparameter. TheDashboardshosts have only one role,DASHBOARDS, so there is no need to specify it. -
access: Cluster access settings for the following Yandex Cloud services:dataTransfer: Yandex Data Transferserverless: Yandex Serverless Containers
The possible setting values are
trueorfalse.
-
-
maintenance_window.weeklyMaintenanceWindow: Maintenance window schedule:day: Day of the week, inDDDformat, for scheduled maintenance.hour: Hour, inHHformat, for scheduled maintenance. The possible values range from1to24. Use the UTC time zone.
-
-
Call the Cluster.Create method, e.g., via the following cURL
request:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-opensearch/v1/clusters' \ --data "@body.json" -
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume that the repository contents reside in the
~/cloudapi/directory. -
Create a file named
body.jsonand paste the following code into it:{ "folder_id": "<folder_ID>", "name": "<cluster_name>", "environment": "<environment>", "network_id": "<network_ID>", "security_group_ids": [ "<security_group_1_ID>", "<security_group_2_ID>", ... "<security_group_N_ID>" ], "service_account_id": "<service_account_ID>", "deletion_protection": <protect_cluster_against_deletion>, "config_spec": { "version": "<OpenSearch_version>", "admin_password": "<admin_password>", "opensearch_spec": { "plugins": [ "<OpenSearch_plugin_1>", "<OpenSearch_plugin_2>", ... "<OpenSearch_plugin_N>" ], "node_groups": [ { "name": "<host_group_name>", "resources": { "resource_preset_id": "<host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" }, "roles": ["<role_1>","<role_2>"], "hosts_count": "<number_of_hosts>", "zone_ids": [ "<availability_zone_1>", "<availability_zone_2>", "<availability_zone_3>" ], "subnet_ids": [ "<subnet_1_ID>", "<subnet_2_ID>", "<subnet_3_ID>" ], "assign_public_ip": <allow_public_access_to_hosts>, "disk_size_autoscaling": { "planned_usage_threshold": "<scheduled_expansion_percentage>", "emergency_usage_threshold": "<immediate_expansion_percentage>", "disk_size_limit": "<maximum_storage_size_in_bytes>" } }, ... ] }, "dashboards_spec": { "node_groups": [ { "name": "<host_group_name>", "resources": { "resource_preset_id": "<host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" }, "hosts_count": "<number_of_hosts>", "zone_ids": ["<availability_zone>"], "subnet_ids": ["<subnet_ID>"], "assign_public_ip": <allow_public_access_to_hosts>, "disk_size_autoscaling": { "planned_usage_threshold": "<scheduled_expansion_percentage>", "emergency_usage_threshold": "<immediate_expansion_percentage>", "disk_size_limit": "<maximum_storage_size_in_bytes>" } } ] }, "access": { "data_transfer": <allow_access_from_Data_Transfer>, "serverless": <allow_access_from_Serverless_Containers> } }, "maintenance_window": { "weekly_maintenance_window": { "day": "<day_of_week>", "hour": "<hour>" } } }Where:
-
folder_id: Folder ID. You can get it with the list of folders in the cloud. -
name: Cluster name. -
environment: Cluster environment,PRODUCTIONorPRESTABLE. -
network_id: ID of the network where the cluster will be deployed. -
security_group_ids: Security group IDs. -
service_account_id: ID of the service account used for cluster operations. -
deletion_protection: Cluster protection against accidental deletion,trueorfalse.Even with cluster deletion protection enabled, it is still possible to delete a user or connect to the cluster manually and delete the data.
-
config_spec: Cluster settings:-
version: OpenSearch version. -
admin_password:adminpassword.The password must include three groups of characters out of these four:
- Lowercase Latin letters
- Uppercase Latin letters
- Numbers
- Special characters
The password must be from 10 to 72 characters long.
adminis a dedicated user required to manage the cluster and cannot be deleted. Such a user has thesuperuserrole and can perform any operations with the cluster.Tip
However, for routine jobs, we recommend creating individual regular users. For more information, see Managing OpenSearch users.
-
opensearch_spec:OpenSearchhost group settings:-
plugins: List of OpenSearch plugins to additionally install in the cluster. -
node_groups: Host settings as an array of elements, one per host group. Each element has the following structure:-
name: Host group name. -
resources: Cluster resources:resource_preset_id: Host class.disk_size: Disk size, in bytes.disk_type_id: Disk type.
-
roles: List of host roles. A cluster must include at least one group ofDATAhosts and one group ofMANAGERhosts. This can be a single group with two roles or multiple groups with different roles. -
hosts_count: Number of hosts in the group. The minimum number ofDATAhosts is one, while the minimum number ofMANAGERhosts is three. -
zone_ids: List of availability zones the cluster hosts are located in. -
subnet_ids: List of subnet IDs. -
assign_public_ip: Permission to connect to the host from the internet,trueorfalse. -
disk_size_autoscaling: Automatic storage expansion settings:-
planned_usage_threshold: Storage usage percentage to trigger a storage expansion during the next maintenance window.Use a value between
0and100%. The default value is0, i.e., automatic expansion is disabled.If you set this condition, configure the maintenance window schedule in the
maintenance_windowparameter. -
emergency_usage_threshold: Storage usage percentage to trigger an immediate storage expansion.Use a value between
0and100%. The default value is0, i.e., automatic expansion is disabled.Warning
If you specify both thresholds,
emergency_usage_thresholdmust not be less thanplanned_usage_threshold. -
disk_size_limit: Maximum storage size, in bytes, to set when storage usage reaches one of the specified thresholds.
Warning
- You cannot decrease the storage size.
- When scaling your storage, the cluster hosts will be unavailable.
-
-
-
-
dashboards_spec:Dashboardshost group settings that contain thenode_groupsparameter of the same structure asopensearch_spec.node_groups, except for therolesparameter. TheDashboardshosts have only one role,DASHBOARDS, so there is no need to specify it. -
access: Cluster access settings for the following Yandex Cloud services:data_transfer: Yandex Data Transferserverless: Yandex Serverless Containers
The possible setting values are
trueorfalse.
-
-
maintenance_window.weekly_maintenance_window: Maintenance window schedule:day: Day of the week, inDDDformat, for scheduled maintenance.hour: Hour, inHHformat, for scheduled maintenance. The possible values range from1to24. Use the UTC time zone.
-
-
Call the ClusterService.Create method, e.g., via the following gRPCurl
request:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/opensearch/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d @ \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.opensearch.v1.ClusterService.Create \ < body.json -
View the server response to make sure your request was successful.
Creating a cluster copy
You can create an OpenSearch cluster with the settings of another one created earlier. To do this, import the original OpenSearch cluster configuration to Terraform. This way, you can either create an identical copy or use the imported configuration as the baseline and modify it as needed. The import feature is useful when you need to replicate an OpenSearch cluster with multiple settings.
To create an OpenSearch cluster copy:
-
If you do not have Terraform yet, install it.
-
Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.
-
Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it
. -
Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.
-
In the same working directory, place a
.tffile with the following contents:resource "yandex_mdb_opensearch_cluster" "old" { } -
Save the ID of the original OpenSearch cluster to an environment variable:
export OPENSEARCH_CLUSTER_ID=<cluster_ID>You can get the ID with the list of clusters in the folder.
-
Import the original OpenSearch cluster settings to the Terraform configuration:
terraform import yandex_mdb_opensearch_cluster.old ${OPENSEARCH_CLUSTER_ID} -
Get the imported configuration:
terraform show -
Copy it from the terminal and paste it into the
.tffile. -
Place the file in the new
imported-clusterdirectory. -
Edit the copied configuration so that you can create a new cluster from it:
- Specify the new cluster name in the
resourcestring and thenameparameter. - Delete the
created_at,health,id, andstatusparameters. - Add the
admin_passwordparameter to theconfigsection. - If you have
type = "ANYTIME"in themaintenance_windowsection, delete thehourparameter. - Optionally, make further changes if you need a customized configuration.
- Specify the new cluster name in the
-
Get the authentication credentials in the
imported-clusterdirectory. -
In the same directory, configure and initialize the provider. There is no need to create a provider configuration file manually, as you can download it
. -
Place the configuration file in the
imported-clusterdirectory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file. -
Make sure the Terraform configuration files are correct:
terraform validateTerraform will display any configuration errors detected in your files.
-
Create the required infrastructure:
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console
. -
Timeouts
The Terraform provider sets the following timeouts for Managed Service for OpenSearch cluster operations:
- Creating a cluster, including by restoring it from a backup: 30 minutes.
- Updating a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the timeout are aborted.
How do I change these limits?
Add a timeouts section to the cluster description, e.g.:
resource "yandex_mdb_opensearch_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
Examples
Create a Managed Service for OpenSearch cluster with the following test specifications:
-
Name:
my-os-clstr. -
Description:
My OS cluster. -
Label:
label-keywithlabel-value. -
Environment:
production. -
Network name:
default. -
Security group ID:
enp6saqnq4ie244g67sb. -
Service account name:
os-account. -
Deletion protection: Disabled.
-
Maintenance time: Every Monday from 13:00 till 14:00.
-
OpenSearch version:
2.12. -
adminpassword: Specified after entering the cluster create command. -
Access to Data Transfer: Enabled.
-
Access to Serverless Containers: Enabled.
-
OpenSearch added plugin: analysis-icu.
-
OpenSearch additional parameter:
fielddata-cache-size=50%. -
OpenSearchnode group configuration:- Node group name:
os-group. - Host class:
s2.micro. - Disk size:
10737418240(in bytes). - Disk type:
network-ssd. - Number of hosts:
3. - Availability zone:
ru-central1-a. - Subnet:
default-ru-central1-a. - Public address: Assigned.
- Host group roles:
DATAandMANAGER.
- Node group name:
-
Dashboardshost group configuration:- Host group name:
dashboard-group. - Host class:
s2.micro. - Disk size:
10737418240(in bytes). - Disk type:
network-ssd. - Number of hosts:
1. - Availability zone:
ru-central1-a. - Subnet:
default-ru-central1-a. - Public address: Assigned.
- Host group name:
Run this command:
yc managed-opensearch cluster create \
--name my-os-clstr \
--description "My OS cluster" \
--labels label-key=label-value \
--environment production \
--network-name default \
--security-group-ids enp6saqnq4ie244g67sb \
--service-account-name os-account \
--delete-protection \
--maintenance schedule=weekly,`
`weekday=mon,`
`hour=14 \
--version 2.12 \
--read-admin-password \
--data-transfer-access=true \
--serverless-access=true \
--plugins analysis-icu \
--advanced-params fielddata-cache-size=50% \
--opensearch-node-group name=os-group,`
`resource-preset-id=s2.micro,`
`disk-size=10737418240,`
`disk-type-id=network-ssd,`
`hosts-count=3,`
`zone-ids=ru-central1-a,`
`subnet-names=default-ru-central1-a,`
`assign-public-ip=true,`
`roles=data+manager \
--dashboards-node-group name=dashboard-group,`
`resource-preset-id=s2.micro,`
`disk-size=10737418240,`
`disk-type-id=network-ssd,`
`hosts-count=1,`
`zone-ids=ru-central1-a,`
`subnet-names=default-ru-central1-a,`
`assign-public-ip=true
Create a Managed Service for OpenSearch cluster with the following test specifications:
-
Name:
my-os-clstr. -
Environment:
PRODUCTION. -
OpenSearch version:
2.12. -
adminpassword:osAdminpwd1. -
OpenSearchnode group configuration:OpenSearchnode group name:os-group.- Host class:
s2.micro. - Disk size:
10737418240(in bytes). - Disk type:
network-ssd. - Number of hosts:
3. - Availability zone:
ru-central1-a. - Public address: Assigned.
- Host group roles:
DATAandMANAGER.
-
Dashboardshost group configuration:- Host group name:
dashboard-group. - Host class:
s2.micro. - Disk size:
10737418240(in bytes). - Disk type:
network-ssd. - Number of hosts:
1. - Availability zone:
ru-central1-a. - Public address: Assigned.
- Host group name:
-
Maintenance time: Every Monday from 13:00 till 14:00.
-
Network name:
mynet. -
Subnet name:
mysubnet. -
Availability zone:
ru-central1-a. -
Address range:
10.1.0.0/16. -
Security group name:
os-sg. The security group allows connecting to the cluster host from any network (including the internet) on port9200.
The configuration file for this cluster is as follows:
resource "yandex_mdb_opensearch_cluster" "my-os-clstr" {
name = "my-os-clstr"
environment = "PRODUCTION"
network_id = yandex_vpc_network.mynet.id
security_group_ids = [yandex_vpc_security_group.os-sg.id]
config {
version = "2.12"
admin_password = "osAdminpwd1"
opensearch {
node_groups {
name = "os-group"
assign_public_ip = true
hosts_count = 3
zone_ids = ["ru-central1-a"]
subnet_ids = [yandex_vpc_subnet.mysubnet.id]
roles = ["DATA", "MANAGER"]
resources {
resource_preset_id = "s2.micro"
disk_size = 10737418240
disk_type_id = "network-ssd"
}
}
}
dashboards {
node_groups {
name = "dashboard-group"
assign_public_ip = true
hosts_count = 1
zone_ids = ["ru-central1-a"]
subnet_ids = [yandex_vpc_subnet.mysubnet.id]
resources {
resource_preset_id = "s2.micro"
disk_size = 10737418240
disk_type_id = "network-ssd"
}
}
}
}
maintenance_window {
type = "WEEKLY"
day = "MON"
hour = 14
}
}
resource "yandex_vpc_network" "mynet" {
name = "mynet"
}
resource "yandex_vpc_subnet" "mysubnet" {
name = "mysubnet"
zone = "ru-central1-a"
network_id = yandex_vpc_network.mynet.id
v4_cidr_blocks = ["10.1.0.0/16"]
}
resource "yandex_vpc_security_group" "os-sg" {
name = "os-sg"
network_id = yandex_vpc_network.mynet.id
ingress {
description = "Allow connections to the Managed Service for OpenSearch cluster from the Internet"
protocol = "TCP"
port = 9200
v4_cidr_blocks = ["0.0.0.0/0"]
}
egress {
description = "The rule allows all outgoing traffic"
protocol = "ANY"
v4_cidr_blocks = ["0.0.0.0/0"]
from_port = 0
to_port = 65535
}
}
Managing database connection parameters using Connection Manager
If your cloud or folder has access to Connection Manager public preview, a new connection entity will appear in your folder after you create a cluster. You can use it to manage database connection parameters.
Passwords and other sensitive data will be stored in a Yandex Lockbox secret. To see which secrets store connection information for your cluster, select Lockbox in the list of services in your folder. You will find you cluster's ID on the Secrets page in the secret dependencies column.
You can also use Connection Manager to configure access to connections.