Alerting rules
Note
Alerting rules are at the Technical Preview stage. To access the feature, contact support
With Yandex Managed Service for Prometheus® alerting, you can add alert evaluation rules and send notifications when they trigger. To configure alerting in Prometheus, you need to create alerting rules and set up the alert manager to process and deliver notifications.
Requirements for alerting rules
In Prometheus, you can use your current PromQL-based alerting rule
It supports all YAML file specification$value
and $labels
variables. Iterations and functions are not supported.
This section describes some aspects related to alerting rules and the alert manager configuration. For how to upload and work with files, see Recording rules.
Alert manager
Alert manager
Rule evaluation highlights:
- Notification channels from the configuration file are mapped against the Yandex Monitoring notification channels specified in the workspace folder.
- Only email
and Telegram channels are currently supported. All other channels will be ignored without any error notifications. Future versions will include support for all notification channels available in Yandex Monitoring. - The channel is selected according to the specified type; dynamic routing
is currently not supported but is planned for upcoming releases. - The system will not accept the file if the configuration lacks channels matching those in the folder.
You can use alerting rules without loading a configuration file. In this case, the alerting rules will be evaluated and will create the ALERTS
and ALERTS_FOR_STATE
metrics, but no alert notifications will be sent.
You can manage rule files via the management console
Pre-configuration for using the API
The API consists of REST resources available at https://monitoring.api.cloud.yandex.net/prometheus/workspaces/<workspace_ID>/extensions/v1/rules
.
To start running requests:
- Install cURL
. - Authenticate with the API.
- Create a workspace and copy its ID to then use it in the request address.
Adding or replacing an alerting rule file
- On the Monitoring
page, select Prometheus on the left. - Select or create a workspace.
- Navigate to the Recording rules tab.
- If you have not uploaded any files yet, click Add file and select a
.yml
file with rules. - To add another file, click Add file.
- To replace the existing file, click
> Replace file to its right.
-
Encode the file contents to Base64
as defined in RFC 4648 :cat alerting-rule.yaml # groups: # - name: example # rules: # - alert: HighRequestLatency # expr: job:request_latency_seconds:mean5m{job="myjob"} > 0.5 # for: 10m # labels: # severity: page # annotations: # summary: High request latency base64 -i alerting-rule.yaml # Z3JvdXBzOgotIG5hbWU6IGV4YW1wbGUKICBydWxlczoKICAtIGFsZXJ0OiBIaW...CBsYXRlbmN5Cg==
-
Save the result as a JSON file:
body.json
{ "name": "alerting-rules", "content" : "Z3JvdXBzOgotIG5hbWU6IGV4YW1wbGUKICBydWxlczoKICAtIGFsZXJ0OiBIaW...CBsYXRlbmN5Cg==" }
-
Create or replace an alerting rule file:
export IAM_TOKEN=<IAM_token> curl -X PUT \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${IAM_TOKEN}" \ -d "@body.json" \ "https://monitoring.api.cloud.yandex.net/prometheus/workspaces/<workspace_ID>/extensions/v1/rules"
Adding or replacing an alert manager configuration file
- On the Monitoring
page, select Prometheus on the left. - Select or create a workspace.
- Navigate to the Alert manager configuration tab.
- If you have no uploaded configuration files yet, click Upload a configuration file and select a
.yml
file. - To download a file, click Download.
- To replace a file, click Replace file.
-
Encode the file contents to Base64 as defined in RFC 4648:
cat alert-manager.yaml # receivers: # - name: 'email' # email_configs: # - to: 'alerts@monitoring.org' # - name: 'telegram' # telegram_configs: # - api_url: https://api.telegram.org base64 -i alert-manager.yaml # cmVjZWl2ZXJzOgogIC0gbmFtZTogJ2VtYWlsJwogICA...sOiBodHRwczovL2FwaS50ZWxlZ3JhbS5vcmcKCg==
-
Save the result as a JSON file:
body.json
{ "content" : "cmVjZWl2ZXJzOgogIC0gbmFtZTogJ2VtYWlsJwogICA...sOiBodHRwczovL2FwaS50ZWxlZ3JhbS5vcmcKCg==" }
-
Create or replace a configuration file:
export IAM_TOKEN=<IAM_token> curl -X PUT \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${IAM_TOKEN}" \ -d "@body.json" \ "https://monitoring.api.cloud.yandex.net/prometheus/workspaces/<workspace_ID>/extensions/v1/alertmanager"
If the request is successful, you will get the 204
HTTP code, if not, the error text. A file without a single match with current notification channels in the folder will not be accepted.