Delivering custom application metrics
Yandex Unified Agent collects metrics in Prometheus format and converts them to Yandex Monitoring format. Unified Agent enables you to pull metrics from applications that provide them in Prometheus format.
To deliver custom application metrics to Yandex Monitoring, use the metrics_pull input that regularly polls the app over HTTP, expecting to receive metrics in Prometheus format.
As an example, let's consider delivery of metrics from a test app written in Python. Your test app and Unified Agent can run either on different VMs or on the same one. If the VMs are different, make sure their security groups allow incoming and outgoing traffic on port 8000 via TCP.
Example of delivering custom application metrics
You can also use this method to deliver metrics from any custom applications that use Prometheus client libraries
-
Set up a service account to use for writing metrics to Yandex Monitoring.
-
Create a service account in the folder for metric storage and assign it the
monitoring.editorrole. -
Link your service account to the VM where Unified Agent will be deployed.
-
-
Run a test Python app that provides metrics in Prometheus format.
-
Install the prometheus_client
Python library by running the following commands:sudo apt install python3-pip pip3 install prometheus_client -
Create a test Python app by writing the following code to the example.py file:
example.py:
from prometheus_client import start_http_server, Summary import random import time # Create a metric to monitor the number of requests and their processing time. REQUEST_TIME = Summary('request_processing_seconds', 'Time spent processing request') # Use a decorator to add the metric to the request handler function. @REQUEST_TIME.time() def process_request(t): """A dummy function that takes some time.""" time.sleep(t) if __name__ == '__main__': # Start the HTTP server to expose metrics from the application. start_http_server(8000) # Generate random requests. while True: process_request(random.random()) -
Run your test Python app with this command:
python3 example.pyTo successfully deliver metrics to Unified Agent, keep your test app running.
-
Check that the application delivers metrics by running the following command, specifying the public IP address of your VM running the app:
curl http://<VM_public_address>:8000Result:
# HELP python_gc_objects_collected_total Objects collected during gc # TYPE python_gc_objects_collected_total counter python_gc_objects_collected_total{generation="0"} 362.0 python_gc_objects_collected_total{generation="1"} 0.0 python_gc_objects_collected_total{generation="2"} 0.0 # HELP python_gc_objects_uncollectable_total Uncollectable object found during GC # TYPE python_gc_objects_uncollectable_total counter
-
-
Install and configure Yandex Unified Agent:
-
Install Docker, if required:
sudo apt-get install docker.io -
Create a file named config.yml in your home directory.
config.yml:
status: port: "16241" storages: - name: main plugin: fs config: directory: /var/lib/yandex/unified_agent/main max_partition_size: 100mb max_segment_size: 10mb channels: - name: cloud_monitoring channel: pipe: - storage_ref: name: main output: plugin: yc_metrics config: folder_id: "<folder_ID>" iam: cloud_meta: {} routes: - input: plugin: metrics_pull config: url: http://<VM_public_address>:8000/metrics metric_name_label: my_name # optional, allows you to rename your application's name label, as this name is reserved by the agent. format: prometheus: {} namespace: app channel: channel_ref: name: cloud_monitoring - input: plugin: agent_metrics config: namespace: ua channel: pipe: - filter: plugin: filter_metrics config: match: "{scope=health}" channel_ref: name: cloud_monitoringWhere:
-
folder_id: ID of the folder to which you want to write metrics. -
url: Public address of the VM hosting the test app that delivers metrics. -
metric_name_label: Determines which label the agent writes the metric name for the Prometheus data to. By default, thenamelabel is used, which may cause a conflict if your app already uses this label. In which case you get this error when writing metrics:label name 'name' is reserved
To avoid an error, specify any other unique name.
-
-
Install Unified Agent by running the following command in your home directory:
docker run \ -p 16241:16241 -it --detach --uts=host \ --name=ua \ -v /proc:/ua_proc \ -v $(pwd)/config.yml:/etc/yandex/unified_agent/conf.d/config.yml \ -e PROC_DIRECTORY=/ua_proc \ -e FOLDER_ID=<folder_ID> \ cr.yandex/yc/unified-agentWhere
FOLDER_IDis the ID of the folder to which you want to write metrics.You can find more ways to install the agent in Installing and updating Yandex Unified Agent.
-
-
Make sure Yandex Monitoring receives the metrics:
-
On the Yandex Monitoring home page
, go to Metrics. -
In the query string, specify the following:
- Folder for storing collected metrics.
Custom Metricslabel value.- Metric name prefixed with
app.
-