Filebeat OSS

Filebeat OSS is a plugin that helps you collect and forward logs to the OpenSearch ecosystem. Filebeat is installed in a Managed Service for Kubernetes cluster; it collects cluster and pod logs and forwards them to Managed Service for OpenSearch.

Filebeat version: 7.12.1.

1. Make sure that both your Managed Service for Kubernetes cluster and Managed Service for OpenSearch cluster are in the same cloud network.

2. Enable the compatibility mode to support the Filebeat OSS client in OpenSearch. For this, run the following query:

   wget "https://storage.yandexcloud.net/cloud-certs/CA.pem" && \
   curl --user <username>:<password> --cacert CA.pem \
   --request PUT https://<host URL>:9200/_cluster/settings \
   -H "Content-Type: application/json" -d \
   '{
   "persistent": {
       "compatibility": {
       "override_main_response_version": true
       }
   }
   }'
   

   Where:

   • <username>: OpenSearch username.
   • <password>: OpenSearch password.
   • <host URL>: URL of the OpenSearch host with the DATA role, e.g., rc1a-6khpaeo31lacqo21.mdb.yandexcloud.net.

   Example of a successful response:

   {"acknowledged":true,"persistent":{"compatibility":{"override_main_response_version":"true"}},"transient":{}}
   

3. Configure the application:

   • Namespace: Select a namespace or create a new one.
   • Application name: Enter a name for the application, e.g., filebeat-oss.
   • OpenSearch username: Enter the account name that Filebeat OSS will use to connect to the Managed Service for OpenSearch cluster.
   • OpenSearch connection password: Enter the password for your account in the Managed Service for OpenSearch cluster.
   • OpenSearch FQDN: Enter the URL and port for the Managed Service for OpenSearch cluster with the DATA role, e.g., https://rc1a-6khpaeo31lacqo21.mdb.yandexcloud.net. For more information about connecting to a cluster, see the service documentation.

4. Click Install.

5. Wait for the application to change its status to Deployed.

6. Make sure that OpenSearch is receiving logs from the Kubernetes cluster:

   1. Connect to the OpenSearch Dashboards web interface.
   2. In the management console, under OpenSearch Plugins, select Index Management.
   3. Go to Indexes.
   4. Make sure the list has the Kubernetes cluster logs.

• Collecting and forwarding Kubernetes cluster logs to OpenSearch
• Analyzing logs in OpenSearch

Yandex Cloud technical support is available 24/7. The types of requests you can submit and the appropriate response time depend on your pricing plan. You can switch to the paid support plan in the management console. You can learn more about the technical support terms here.