Alert settings in Yandex Monitoring

In this tutorial, you will learn how to track trail status using dashboards and Yandex Monitoring alerts.

This guide assumes that you already have deployed your infrastructure:

• Created Yandex Cloud resources to collect security events for.
• Created a trail in Audit Trails to collect events.
• Configured the target to store and manage events: a bucket, datastream, or log group.

To start tracking the status of trails:

• Set up alerts.
• Set up the dashboard.

If you no longer need the resources, delete them.

Set up alertsSet up alerts

Create a notification channelCreate a notification channel

To get notifications about a triggered alert:

1. In the management console, select the folder where you want to create a notification channel.
2. Select Monitoring.
3. In the left-hand panel, select Notification channels.
4. In the top-right corner, click Create channel.
5. Specify the channel settings:
   • In the Name field, specify alerts-channel.
   • In the Method field, specify the notification method.
   • In the Recipients field, list notification recipients.
6. Click Create.

The channel will appear in the list.

Add alertsAdd alerts

You can set up one or more alerts.

For more information about how to create alerts and about alert parameters, see the Yandex Monitoring documentation.

Deactivating a trailDeactivating a trail

The alert will send a notification that the trail is being deactivated.

1. In the management console, select the folder where you want to create an alert.
2. Select Monitoring.
3. In the left-hand panel, select Alerts.
4. In the top-right corner, click Create alert.
5. In the Name field, specify deactivating-trail-alert.
6. Under Metrics, click to the right of the folder name and specify:
   1. service = Audit Trails.
   2. name = trail.status.
   3. status != ACTIVE.
   4. trail = <trail_name>.
7. Under Alert parameters, specify:
   1. Condition: Not equals to.
   2. Alarm: 0.
8. Under Notification channels, click Add channel and select the previously created notification channel.
9. Click Create alert.

The alert is created.

Stopping delivery of audit logs to destination objectStopping delivery of audit logs to destination object

The alert will send notification that the trail has stopped uploading audit logs to its destination object, for example, due to a lack of free space in the bucket.

The Evaluation window parameter depends on the specific trail. The type and number of resources within the audit trail logging section will define the frequency for uploading audit logs to the destination object.

1. In the management console, select the folder where you want to create an alert.
2. Select Monitoring.
3. In the left-hand panel, select Alerts.
4. In the top-right corner, click Create alert.
5. In the Name field, specify stopping-logs-alert.
6. Under Metrics, click to the right of the folder name and specify:
   1. service = Audit Trails.
   2. name = trail.delivered_events_count.
   3. trail = <trail_name>.
7. Under Alert parameters, specify:
   1. Condition: Equals to.
   2. Alarm: 0.
   3. Evaluation window: <trail_value>.
8. Under Notification channels, click Add channel and select the previously created notification channel.
9. Click Create alert.

The alert is created.

Modifying the number of trailsModifying the number of trails

The alert will send a notification that the number of trails in a cloud has changed.

1. In the management console, select the folder where you want to create an alert.
2. Select Monitoring.
3. In the left-hand panel, select Alerts.
4. In the top-right corner, click Create alert.
5. In the Name field, specify number-trails-alert.
6. Under Metrics, click to the right of the folder name and specify:
   1. service = Audit Trails.
   2. name = quota.trails_count.usage.
7. Under Alert parameters, specify:
   1. Condition: Not equals to.
   2. Alarm: <number_of_trails>.
8. Under Notification channels, click Add channel and select the previously created notification channel.
9. Click Create alert.

Nearing cloud trail quotaNearing cloud trail quota

The alert will send a notification that the number of trails used per cloud consumed over 80% of the quota.

1. In the management console, select the folder where you want to create an alert.
2. Select Monitoring.
3. In the left-hand panel, select Alerts.
4. In the top-right corner, click Create alert.
5. In the Name field, specify trail-quota-alert.
6. Under Metrics, click to the right of the folder name and specify:
   1. service = Audit Trails.
   2. name = quota.trails_count.usage.
7. Under Alert parameters, specify:
   1. Condition: Greater than.
   2. Alarm: <number_equal_to_80%_of_quota>.
8. Under Notification channels, click Add channel and select the previously created notification channel.
9. Click Create alert.

Unauthorized access attemptsUnauthorized access attempts

The alert will send a notification that an unauthorized request has been sent to one of the trail resources.

1. In the management console, select the folder where you want to create an alert.
2. Select Monitoring.
3. In the left-hand panel, select Alerts.
4. In the top-right corner, click Create alert.
5. In the Name field, specify unauthorized-access-alert.
6. Under Metrics, click to the right of the folder name and specify:
   1. service = Audit Trails.
   2. name = trail.unauthorized_events_count.
7. Under Alert parameters, specify:
   1. Condition: Greater than.
   2. Alarm: 0.
8. Under Notification channels, click Add channel and select the previously created notification channel.
9. Click Create alert.

Set up a dashboardSet up a dashboard

Create a dashboardCreate a dashboard

Create a chart for missed event monitoringCreate a chart for missed event monitoring

You can monitor missed events using Audit Trails metrics:

• trail.processed_events_count: Rate at which the events are accepted for processing.
• trail.delivered_events_count: Event delivery rate to the destination object.

Copy a ready-to-use chart for missed event monitoring to your dashboard:

1. Open the trail dashboard in Monitoring:

   Management console

   1. In the management console, select Audit Trails.

   2. In the left-hand panel, select Trails.

   3. Select the trail you need.

   4. Go to the Monitoring panel for the selected trail.

   5. Click Open in Monitoring at the top right.

      This will take you to the Monitoring interface.

2. Copy the chart:

   Monitoring UI

   1. Find the Processed versus delivered events chart.

   2. To the right of the chart name, click → Copy to another dashboard.

   3. Specify a name, e.g., Processed versus delivered events — <trail_name>.

   4. Select a cloud and folder, then specify the dashboard you created earlier.

   5. Click Copy and edit.

      This will open your dashboard with a new chart.

If you need to, follow the same steps to add charts from other trails to your dashboard.

Review the chart for missed event monitoringReview the chart for missed event monitoring

Note that there may be a lag between Delivered events and Processed events. Normally, you may encounter short-term lags that are compensated soon. If you have observed a persistent delivery lag of one hour or longer, check the trail status and diagnostics logs.

Get the trail statusGet the trail status

The Active status during a persistent lag between Delivered events and Processed events means that the trail operates normally, but there are some other reasons why data is delivered to the destination object with delays. In this case, check the destination object status and logs:

• Getting bucket information and statistics
• Getting information about a log group
• Step-by-step guides for Data Streams

The Error status indicates a trail performance error. In which case, review the trail diagnostics log.

Review the trail diagnostics logReview the trail diagnostics log

Create a chart for the destination objectCreate a chart for the destination object

Apart from the chart for missed event monitoring, you can add a chart for the destination object:

• Object Storage

  Events can be missed if there is not enough space for storing logs, e.g, if a trail sends logs to a bucket of limited size. To monitor the available bucket space, create a chart for the space_usage metric and add the max-size metric as a threshold.

  The max-size metric will not be available if the maximum bucket size is not specified. If so, you need to track the storage space per cloud quota usage on your own.

• Cloud Logging

  Add a chart for the group.service.ingested_records_per_second metric to the dashboard to display the actual rate of log ingestion into the log group. Comparing this value with the Maximum write speed quota helps determine whether the log stream is hitting its limit. The additional group.service.ingest_requests_per_second chart filtered by ERROR status enables detecting write errors promptly.

• Data Streams: See the Yandex Managed Service for YDB tutorials:

  • Yandex Monitoring metric reference
  • Quotas and limits in Managed Service for YDB

How to delete the resources you createdHow to delete the resources you created

• Delete the alerts
• Delete the dashboard