Fluent Bit with a Cloud Logging plugin

Fluent Bit is an open-source multi-platform tool for collecting and processing logs. Fluent Bit collects, parses, and filters messages from a variety of inputs and saves them to storage. Once messages are saved, they are forwarded from storage to a router that determines which destination to route them to. Plugins are used to work with various inputs and outputs.

The Cloud Logging plugin lets you export logs generated in Kubernetes clusters, including at the pod level, to Yandex Cloud Logging.

1. Create a Kubernetes cluster and a node group.

2. Create a service account, which is required for Fluent Bit.

3. Assign the service account the logging.writer role.

4. Create an authorized key for the service account and save it to a file named sa-key.json:

   yc iam key create \
      --service-account-name <service account name> \
      --output sa-key.json
   

5. Create a log group.

6. If you want to save logs to Yandex Object Storage:

   1. Create a bucket.

   2. Assign the service account the storage.uploader role.

   3. Create a static access key for the service account and save it in JSON format:

      {
          "access_key": {
          "id": "xxxxxx",
          "created_at": "xxxxxx",
          "key_id": "xxxxxx"
          },
          "secret": "xxxxxx"
      }
      

7. Set the Fluent Bit settings:

   • Namespace: Select a namespace or create a new one.
   • Application name: Enter a name for the application.
   • Target log group ID: Specify an ID of the log group to which logs will be transmitted.
   • Service account key for Cloud Logging: Paste the contents of the sa-key.json file.
   • Save logs to Object Storage: Select this option if you want to save logs to Object Storage.
   • Object Storage bucket name: Specify the name of the bucket where logs will be stored. Leave the field blank if you do not want to save logs.
   • Static access key for Object Storage: Paste the static access key in JSON format. Leave the field blank if you do not want to save logs.

8. Click Install.

9. Wait for the application to change its status to Deployed.

• Exporting Kubernetes cluster logs to Yandex Cloud Logging.

Yandex Cloud technical support is available 24/7. The types of requests you can submit and the appropriate response time depend on your pricing plan. You can switch to the paid support plan in the management console. You can learn more about the technical support terms here.