Connecting to an OpenSearch cluster
You can connect to Managed Service for OpenSearch cluster hosts with the DATA
role:
-
Over the internet, if you configured public access for the appropriate host group.
-
From Yandex Cloud VMs residing in the same virtual network.
Regardless of the connection method, Managed Service for OpenSearch only supports cluster host connections with an SSL certificate.
Configuring security groups
To connect to a cluster, security groups must include rules allowing traffic from certain ports, IP addresses, or from other security groups.
Settings of access policies depend on the connection method you select:
Configure all the cluster security groups to allow incoming traffic on ports 443 (Dashboards) and 9200 (OpenSearch) from any IP address. To do this, create the following rules for incoming traffic:
- Port range:
443
,9200
- Protocol:
TCP
- Source:
CIDR
- CIDR blocks:
0.0.0.0/0
A separate rule is created for each port.
-
Configure all the cluster security groups to allow incoming traffic on ports 443 (Dashboards) and 9200 (OpenSearch) from the security group where the VM is located. To do this, create the following rules for incoming traffic in these security groups:
- Port range:
443
,9200
- Protocol:
TCP
- Source:
Security group
- Security group: If your cluster and VM are in the same security group, select
Current
(Self
). Otherwise, specify the VM security group.
A separate rule is created for each port.
- Port range:
-
Configure all the security groups where your VM is located to allow connections to the VM and traffic between the VM and cluster hosts.
For example, you can set the following rules for a VM:
-
For incoming traffic:
- Port range:
22
,443
,9200
- Protocol:
TCP
- Source:
CIDR
- CIDR blocks:
0.0.0.0/0
A separate rule is created for each port.
- Port range:
-
For outgoing traffic:
- Port range:
0-65535
- Protocol:
Any
(Any
) - Destination name:
CIDR
- CIDR blocks:
0.0.0.0/0
This rule allows all outgoing traffic, thus enabling you not only to connect to the cluster but also to install the certificates and utilities your VM needs for the connection.
- Port range:
-
Note
You can specify more detailed rules for your security groups, e.g., to allow traffic only in specific subnets.
You must configure security groups correctly for all subnets in which the cluster hosts will reside. If security group settings are incomplete or incorrect, you may lose access to the cluster.
For more information about security groups, see Security groups.
Getting an SSL certificate
To use an encrypted connection, get an SSL certificate:
mkdir -p ~/.opensearch && \
wget "https://storage.yandexcloud.net/cloud-certs/CA.pem" \
--output-document ~/.opensearch/root.crt && \
chmod 0600 ~/.opensearch/root.crt
The certificate will be saved to the ~/.opensearch/root.crt
file.
mkdir $HOME\.opensearch; curl --output $HOME\.opensearch\root.crt https://storage.yandexcloud.net/cloud-certs/CA.pem
The certificate will be saved to the $HOME\.opensearch\root.crt
file.
Corporate policies and antivirus software can block the download of certificates. For more information, see FAQ.
OpenSearch host FQDN
To connect to a host, you need its fully qualified domain name (FQDN). You can obtain it in one of the following ways:
-
In the management console
, copy the command for connecting to the cluster. This command contains the host FQDN. To get the command, go to the cluster page and click Connect. -
Look up the FQDN in the management console:
- Go to the cluster page.
- Go to Hosts.
- Copy the Host FQDN column value.
Hosts with the DASHBOARDS
role also use special FQDNs.
Special FQDN
Alongside regular FQDNs, Managed Service for OpenSearch provides a special FQDN, which you can also use when connecting to a cluster.
An FQDN in c-<cluster_ID>.rw.mdb.yandexcloud.net
format always points to the available OpenSearch host with the DASHBOARDS
role in the cluster. You can get the cluster ID with a list of clusters in the folder.
The service does not provide special FQDNs for hosts with the DATA
role.
Connecting to OpenSearch Dashboards
You can connect to OpenSearch Dashboards:
- Over the internet if public access is enabled for a host with the
DASHBOARDS
role. - Via a VM instance in Yandex Cloud if public access is not enabled for any of your hosts with the
DASHBOARDS
role.
-
Install the SSL certificate in your browser's trusted root certificate store (instructions
for Mozilla Firefox). -
On the cluster page in the management console, click OpenSearch Dashboards or go to
https://c-<cluster_ID>.rw.mdb.yandexcloud.net>
in your browser.You can get the cluster ID with a list of clusters in the folder.
-
Enter
admin
for the username and the password you set when creating the cluster.
-
Create a Linux VM in the same virtual network as the cluster.
-
Connect to the virtual machine over SSH.
-
Install the dependencies:
sudo apt update && \ sudo apt install --yes nginx ssl-cert
-
Copy the downloaded SSL certificate to the
/etc/nginx/
directory:sudo cp ~/.opensearch/root.crt /etc/nginx/root.crt
-
Edit the NGINX default configuration file, for example, like this:
/etc/nginx/sites-available/default
upstream os-dashboards-nodes { server <FQDN_of_host_1_with_DASHBOARDS_role>:443; ... server <FQDN_of_host_N_with_DASHBOARDS_role>:443; } server { listen 443 ssl; ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem; ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key; server_name _; location / { proxy_pass https://os-dashboards-nodes; proxy_ssl_trusted_certificate /etc/nginx/root.crt; proxy_ssl_session_reuse on; } }
Warning
This configuration file example uses a self-signed
snakeoil
certificate from thessl-cert
package. It is not safe to use this certificate in a real cluster. Instead of the self-signed certificate, specify the path to your public and private SSL certificate keys in thessl_certificate
andssl_certificate_key
directives. -
Restart NGINX:
sudo systemctl restart nginx
-
Add the certificate specified in the
ssl_certificate
directive to the browser's trusted root certificate store (instructions for Mozilla Firefox). -
In your browser, go to
https://<VM_public_IP_address>
. -
Enter the username and password for the
admin
user.
Note
When using the OpenSearch Dashboards API:
- To send requests, use
443
for port instead of the standard5601
. - Add the SSL certificate path to your application's configuration to use the API.
Before you connect from a Docker container
To connect to a Managed Service for OpenSearch cluster from a Docker container, add the following lines to the Dockerfile:
RUN apt-get update && \
apt-get install curl --yes
RUN apt-get update && \
apt-get install wget curl --yes && \
mkdir --parents ~/.opensearch && \
wget "https://storage.yandexcloud.net/cloud-certs/CA.pem" \
--output-document ~/.opensearch/root.crt && \
chmod 0600 ~/.opensearch/root.crt
Examples of connection strings
Before connecting, prepare a certificate.
To connect, enter admin
for the username and the password you set when creating the cluster.
To see code examples with the host FQDN filled in, open the cluster page in the management console
Bash
curl \
--user admin:<password> \
--cacert ~/.opensearch/root.crt \
--request GET 'https://<FQDN_of_the_OpenSearch_host_with_the_DATA_role>:9200/'
To learn how to get a host FQDN, see this guide.
Go
Before connecting, install the dependencies:
go mod init opensearch-example && \
go get github.com/opensearch-project/opensearch-go
-
Code example:
connect.go
package main import ( "crypto/tls" "crypto/x509" "crypto/x509" "github.com/opensearch-project/opensearch-go" "io/ioutil" "log" "net/http" ) var hosts = []string{ "<FQDN_of_host_1_with_DATA_role>:9200", ..., "<FQDN_of_host_N_with_DATA_role>:9200" } var CA = "/home/<home_directory>/.opensearch/root.crt" var password = "<password>" func main() { caCert, err := ioutil.ReadFile(CA) if err != nil { log.Fatal(err) } caCertPool := x509.NewCertPool() caCertPool.AppendCertsFromPEM(caCert) cfg := opensearch.Config{ Addresses: hosts, Transport: &http.Transport{ TLSClientConfig: &tls.Config{ RootCAs: caCertPool, }, }, Username: "admin", Password: password, } es, err := opensearch.NewClient(cfg) if err != nil { log.Printf("Error creating the client: %s", err) } else { log.Println(es.Info()) } }
Unlike other connection methods, in this example, you need to use the full path to the
CA.pem
certificate for OpenSearch in theCA
variable. -
Connecting:
go run connect.go
To learn how to get a host FQDN, see this guide.
PowerShell
curl `
-Certificate <absolute_path_to_certificate_file> `
-Uri https://<FQDN_of_OpenSearch_DATA_host>:9200 `
-Credential admin
To learn how to get a host FQDN, see this guide.
Python
Before connecting, install the dependencies:
sudo apt update && sudo apt install --yes python3 python3-pip && \
pip3 install opensearch-py
-
Code example:
connect.py
from opensearchpy import OpenSearch CA = '~/.opensearch/root.crt' PASS = '<password>' HOSTS = [ "<FQDN_of_DATA_host_1>", ..., "<FQDN_of_DATA_host_N>" ] conn = OpenSearch( HOSTS, http_auth=('admin', PASS), use_ssl=True, verify_certs=True, ca_certs=CA) print(conn.info())
-
Connecting:
python3 connect.py
To learn how to get a host FQDN, see this guide.