Delivering metrics from HAProxy and other third-party applications

Yandex Unified Agent supports metric collection in Prometheus format and metric conversion to Yandex Monitoring format. Unified Agent enables you to pull metrics from applications that provide them in Prometheus format.

To deliver metrics from third-party applications to Yandex Monitoring, use the metrics_pull input. It periodically polls the third-party application directly (if the application supports metrics in Prometheus format) or polls a special export application that manages the integration with Prometheus.

As an example, let's consider the delivery of HAProxy metrics to Yandex Monitoring.

Example of delivering HAProxy metricsExample of delivering HAProxy metrics

You can also use this method to send metrics from any third-party applications integrated with Prometheus.

1. Install HAProxy.

   1. To install HAProxy, run this command:

      sudo apt install haproxy
      

   2. To enable Prometheus integration, configure the exporter by adding a section named frontend to the haproxy.cfg configuration file:

      frontend stats
          bind *:8404
          http-request use-service prometheus-exporter if { path /metrics }
          stats enable
          stats uri /stats
          stats refresh 10s
      

   3. Restart HAProxy:

      sudo service haproxy restart
      

   4. Make sure the exporter is running and exposing metrics. To do this, run curl http://localhost:8404/metrics. Command output example:

      # HELP haproxy_process_nbthread Number of started threads (global.nbthread)
      # TYPE haproxy_process_nbthread gauge
      haproxy_process_nbthread 2
      # HELP haproxy_process_nbproc Number of started worker processes (historical, always 1)
      # TYPE haproxy_process_nbproc gauge
      haproxy_process_nbproc 1
      # HELP haproxy_process_relative_process_id Relative worker process number (1)
      # TYPE haproxy_process_relative_process_id gauge
      haproxy_process_relative_process_id 1
      # HELP haproxy_process_uptime_seconds How long ago this worker process was started (seconds)
      # TYPE haproxy_process_uptime_seconds gauge
      haproxy_process_uptime_seconds 5
      # HELP haproxy_process_pool_failures_total Number of failed pool allocations since this worker was started
      # TYPE haproxy_process_pool_failures_total counter
      haproxy_process_pool_failures_total 0
      

2. Set up a service account to use for writing metrics to Yandex Monitoring.

   1. Create a service account in the folder for metric storage and assign it the monitoring.editor role.

   2. Link your service account to a virtual machine with Unified Agent installed.

3. Install and configure Yandex Unified Agent:

   1. Create a file named config.yml in your home directory:

       status:
         port: "16241"
      
       storages:
         - name: main
           plugin: fs
           config:
             directory: /var/lib/yandex/unified_agent/main
             max_partition_size: 100mb
             max_segment_size: 10mb
      
       channels:
         - name: cloud_monitoring
           channel:
             pipe:
               - storage_ref:
                   name: main
             output:
               plugin: yc_metrics
               config:
                 folder_id: "$FOLDER_ID"
                 iam:
                   cloud_meta: {}
      
       routes:
         - input:
             plugin: metrics_pull
             config:
               url: http://localhost:8404/metrics
               format:
                 prometheus: {}
               namespace: haproxy
           channel:
             channel_ref:
               name: cloud_monitoring
      
         - input:
             plugin: agent_metrics
             config:
               namespace: ua
           channel:
             pipe:
               - filter:
                   plugin: filter_metrics
                   config:
                     match: "{scope=health}"
             channel_ref:
               name: cloud_monitoring
      
       import:
         - /etc/yandex/unified_agent/conf.d/*.yml
      

      Where $FOLDER_ID is the ID of the folder to which you want to write metrics.

   2. Install Unified Agent on your VM by running the following command in your home directory:

      docker run \
      -p 16241:16241 -it --detach --uts=host \
      --name=ua \
      -v /proc:/ua_proc \
      -v `pwd`/config.yml:/etc/yandex/unified_agent/config.yml \
      -e PROC_DIRECTORY=/ua_proc \
      -e FOLDER_ID=a1bs81qpemb4******** \
      cr.yandex/yc/unified-agent
      

      You can find more ways to install the agent in Installing and updating Yandex Unified Agent.

4. Make sure Yandex Monitoring receives the metrics:

   1. On the Yandex Monitoring home page, go to Metrics.

   2. In the query string, specify the following:

   • Folder for storing collected metrics.
   • service=custom label value.
   • Metric name prefixed with haproxy.

What's nextWhat's next

• Read about Unified Agent concepts
• Learn more about configuring Unified Agent
• Read best practices for using Unified Agent