Data transmission via Fluent Bit

Fluent Bit is an agent that collects, processes, and exports logs and metrics. You can use Fluent Bit to send telemetry data to Monium in OpenTelemetry (OTLP) format.

Fluent Bit is your optimal choice in the following cases:

• Logs come in many different formats, so you need flexible parsers.
• Your app operates in a Kubernetes cluster.
• You need to collect logs centrally from a single host (files, Docker, system logs).
• Logs are already delivered via files or standard app outputs.

Otherwise, we recommend using OTel Collector.

Version requirementsVersion requirements

We recommend using Fluent Bit version 4.0 or higher with opentelemetry output.

Metric transmission limitationsMetric transmission limitations

Fluent Bit cuts the startTimestampNanos field off the metrics. Do not use Fluent Bit to transmit metrics if your application is configured for delta temporality (OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE="delta") because delta metrics without the period start time are processed incorrectly.

Configuring telemetry transmissionConfiguring telemetry transmission

1. Install Fluent Bit next to the telemetry source, e.g., on a server, in a container, or in a Kubernetes cluster.

2. Create a configuration file, e.g., fluent-bit.yaml.

   Below is an example of the minimum configuration you need to send logs, metrics, and traces to Monium. Configure inputs for your data source.

   Via gRPC

   pipeline:
     inputs:
       - name: opentelemetry
         listen: 127.0.0.1
         port: 4318
   
     outputs:
       - name: opentelemetry
         match: "*"
         host: ingest.monium.yandex.cloud
         port: 443
         tls: on
         compress: zstd
         grpc: on
         header:
           - Authorization Api-Key ${MONIUM_API_KEY}
           - x-monium-project ${MONIUM_PROJECT}
   

   Via HTTP

   pipeline:
     inputs:
       - name: opentelemetry
         listen: 127.0.0.1
         port: 4318
   
     outputs:
       - name: opentelemetry
         match: "*"
         host: ingest.monium.yandex.cloud
         port: 443
         tls: on
         compress: zstd
         logs_uri: /otlp/v1/logs
         traces_uri: /otlp/v1/traces
         metrics_uri: /otlp/v1/metrics
         header:
           - Authorization Api-Key ${MONIUM_API_KEY}
           - x-monium-project ${MONIUM_PROJECT}
   

3. Set these environment variables:

   • MONIUM_PROJECT: Monium project ID.
   • MONIUM_API_KEY: API key with permission to write telemetry data.

4. Run Fluent Bit stating the configuration:

5. Check that logs are fed into Monium.

For a simple scenario of using Fluent Bit to send all types of telemetry from a Java application to Monium, see Example for a Java demo application with Fluent Bit.

For detailed configuration examples (Docker, Kubernetes, parsers), see Sending logs via Fluent Bit.