Delivering custom application metrics

Yandex Unified Agent collects metrics in Prometheus format and converts them to Yandex Monitoring format. Unified Agent enables you to pull metrics from applications that provide them in Prometheus format.

To deliver custom application metrics to Yandex Monitoring, use the metrics_pull input that regularly polls the app over HTTP, expecting to receive metrics in Prometheus format.

As an example, let's consider delivery of metrics from a test app written in Python. Your test app and Unified Agent can run either on different VMs or on the same one. If the VMs are different, make sure their security groups allow incoming and outgoing traffic on port 8000 via TCP.

Example of delivering custom application metricsExample of delivering custom application metrics

You can also use this method to deliver metrics from any custom applications that use Prometheus client libraries.

1. Set up a service account to use for writing metrics to Yandex Monitoring.

   1. Create a service account in the folder for metric storage and assign it the monitoring.editor role.

   2. Link your service account to the VM where Unified Agent will be deployed.

2. Run a test Python app that provides metrics in Prometheus format.

   1. Install the prometheus_client Python library by running the following commands:

      sudo apt install python3-pip
      pip3 install prometheus_client
      

   2. Create a test Python app by writing the following code to the example.py file:

      example.py:

      from prometheus_client import start_http_server, Summary
      import random
      import time
      
      # Create a metric to monitor the number of requests and their processing time.
      REQUEST_TIME = Summary('request_processing_seconds', 'Time spent processing request')
      
      # Use a decorator to add the metric to the request handler function.
      @REQUEST_TIME.time()
      def process_request(t):
          """A dummy function that takes some time."""
          time.sleep(t)
      
      if __name__ == '__main__':
          # Start the HTTP server to expose metrics from the application.
          start_http_server(8000)
          # Generate random requests.
          while True:
              process_request(random.random())
      

   3. Run your test Python app with this command:

      python3 example.py
      

      To successfully deliver metrics to Unified Agent, keep your test app running.

   4. Check that the application delivers metrics by running the following command, specifying the public IP address of your VM running the app:

      curl http://<VM_public_address>:8000
      

      Result:

      # HELP python_gc_objects_collected_total Objects collected during gc
      # TYPE python_gc_objects_collected_total counter
      python_gc_objects_collected_total{generation="0"} 362.0
      python_gc_objects_collected_total{generation="1"} 0.0
      python_gc_objects_collected_total{generation="2"} 0.0
      # HELP python_gc_objects_uncollectable_total Uncollectable object found during GC
      # TYPE python_gc_objects_uncollectable_total counter
      

3. Install and configure Yandex Unified Agent:

   1. Install Docker, if required:

      sudo apt-get install docker.io
      

   2. Create a file named config.yml in your home directory.

      config.yml:

       status:
         port: "16241"
      
       storages:
         - name: main
           plugin: fs
           config:
             directory: /var/lib/yandex/unified_agent/main
             max_partition_size: 100mb
             max_segment_size: 10mb
      
       channels:
         - name: cloud_monitoring
           channel:
             pipe:
               - storage_ref:
                   name: main
             output:
               plugin: yc_metrics
               config:
                 folder_id: "<folder_ID>"
                 iam:
                   cloud_meta: {}
      
       routes:
         - input:
             plugin: metrics_pull
             config:
               url: http://<VM_public_address>:8000/metrics
               format:
                 prometheus: {}
               namespace: app
           channel:
             channel_ref:
               name: cloud_monitoring
      
         - input:
             plugin: agent_metrics
             config:
               namespace: ua
           channel:
             pipe:
               - filter:
                   plugin: filter_metrics
                   config:
                     match: "{scope=health}"
             channel_ref:
               name: cloud_monitoring
      
       import:
         - /etc/yandex/unified_agent/conf.d/*.yml
      

      Where:

      • folder_id: ID of the folder to which you want to write metrics.
      • url: Public address of the VM hosting the test app that delivers metrics.

   3. Install Unified Agent by running the following command in your home directory:

      docker run \
      -p 16241:16241 -it --detach --uts=host \
      --name=ua \
      -v /proc:/ua_proc \
      -v `pwd`/config.yml:/etc/yandex/unified_agent/config.yml \
      -e PROC_DIRECTORY=/ua_proc \
      -e FOLDER_ID=<folder_ID> \
      cr.yandex/yc/unified-agent
      

      Where FOLDER_ID is the ID of the folder to which you want to write metrics.

      You can find more ways to install the agent in Installing and updating Yandex Unified Agent.

4. Make sure Yandex Monitoring receives the metrics:

   1. On the Yandex Monitoring home page, go to Metrics.

   2. In the query string, specify the following:

   • Folder for storing collected metrics.
   • Custom Metrics label value.
   • Metric name prefixed with app.

What's nextWhat's next

• Read about Unified Agent concepts
• Learn more about configuring Unified Agent
• Read best practices for using Unified Agent