Delivering metrics from HAProxy and other third-party applications

Yandex Unified Agent supports metric collection in the Prometheus format and metric conversion to the Yandex Monitoring format. Unified Agent enables you to pull metrics from applications that deliver them in the Prometheus format.

To deliver metrics from third-party applications to Yandex Monitoring, use the metrics_pull input that regularly polls a third-party application directly (if the application supports metrics in Prometheus format) or polls a special export application that integrates with Prometheus.

As an example, let's consider the delivery of HAProxy metrics to Yandex Monitoring.

Example of delivering HAProxy metricsExample of delivering HAProxy metrics

This procedure can also be used to send metrics from any third-party applications integrated with Prometheus.

1. Install HAProxy.

   1. To install HAProxy, run this command:

      sudo apt install haproxy
      

   2. To implement Prometheus integration, configure the exporter. To do this, add a section named frontend to the haproxy.cfg configuration file:

      frontend stats
          bind *:8404
          http-request use-service prometheus-exporter if { path /metrics }
          stats enable
          stats uri /stats
          stats refresh 10s
      

   3. Restart the service:

      sudo service haproxy restart
      

   4. Make sure the exporter is running and exposes metrics. To do this, run the curl http://localhost:8404/metrics command. Command output example:

      # HELP haproxy_process_nbthread Number of started threads (global.nbthread)
      # TYPE haproxy_process_nbthread gauge
      haproxy_process_nbthread 2
      # HELP haproxy_process_nbproc Number of started worker processes (historical, always 1)
      # TYPE haproxy_process_nbproc gauge
      haproxy_process_nbproc 1
      # HELP haproxy_process_relative_process_id Relative worker process number (1)
      # TYPE haproxy_process_relative_process_id gauge
      haproxy_process_relative_process_id 1
      # HELP haproxy_process_uptime_seconds How long ago this worker process was started (seconds)
      # TYPE haproxy_process_uptime_seconds gauge
      haproxy_process_uptime_seconds 5
      # HELP haproxy_process_pool_failures_total Number of failed pool allocations since this worker was started
      # TYPE haproxy_process_pool_failures_total counter
      haproxy_process_pool_failures_total 0
      

2. Set up a service account under which metrics will be written to Yandex Monitoring.

   1. Create a service account in the folder you want to write metrics to and assign it the monitoring.editor role.

   2. Link your service account to a virtual machine with Unified Agent installed.

3. Install and configure Yandex Unified Agent:

   1. Create a file named config.yml in your home folder:

       status:
         port: "16241"
      
       storages:
         - name: main
           plugin: fs
           config:
             directory: /var/lib/yandex/unified_agent/main
             max_partition_size: 100mb
             max_segment_size: 10mb
      
       channels:
         - name: cloud_monitoring
           channel:
             pipe:
               - storage_ref:
                   name: main
             output:
               plugin: yc_metrics
               config:
                 folder_id: "$FOLDER_ID"
                 iam:
                   cloud_meta: {}
      
       routes:
         - input:
             plugin: metrics_pull
             config:
               url: http://localhost:8404/metrics
               format:
                 prometheus: {}
               namespace: haproxy
           channel:
             channel_ref:
               name: cloud_monitoring
      
         - input:
             plugin: agent_metrics
             config:
               namespace: ua
           channel:
             pipe:
               - filter:
                   plugin: filter_metrics
                   config:
                     match: "{scope=health}"
             channel_ref:
               name: cloud_monitoring
      
       import:
         - /etc/yandex/unified_agent/conf.d/*.yml
      

      Where $FOLDER_ID is the ID of the folder you want to write metrics to.

   2. Install Unified Agent on your VM by running the following command in the home folder:

      docker run \
      -p 16241:16241 -it --detach --uts=host \
      --name=ua \
      -v /proc:/ua_proc \
      -v `pwd`/config.yml:/etc/yandex/unified_agent/config.yml \
      -e PROC_DIRECTORY=/ua_proc \
      -e FOLDER_ID=a1bs81qpemb4******** \
      cr.yandex/yc/unified-agent
      

      You can find more ways to install the agent in Installing and running Yandex Unified Agent.

4. Make sure the metrics are delivered to Yandex Monitoring:

   1. On the Yandex Monitoring home page, go to Metric Explorer.

   2. In the query block, select:

   • Folder where metrics are collected.
   • service=custom label value.
   • Metric name starting with the haproxy prefix.

What's nextWhat's next

• Read about Unified Agent concepts.
• Learn more about configuring Unified Agent.
• Read the Unified Agent operating guidelines.