Configuring alerts for the service

Alerting allows you to track the changes in metrics and send notifications when indicators reach critical values. The changes in metrics are tracked with the help of requests sent from time to time, called alerts. When threshold values are reached, the system sends a notification via the specified communication channel. For example, by email or messenger.

Smart Web Security offers these inbound request rate metrics:

• load_balancer.smart_web_security.requests_per_second: Number of requests to the protected resource per second. This value matches the number of requests processed by the security profile.
• load_balancer.smart_web_security.arl_requests_per_second: Number of requests to the protected resource per second processed by Advanced Rate Limiter.

You can configure alerts for metrics reaching the specified rate thresholds. You can also use labels to calculate metrics by individual events.

To configure alerting:

1. Create a notification channel.
2. Select metrics and labels for monitoring.
3. Create an alert.

Let's take a look at how to create some alerts you can use for anomaly detection:

• Alert of increasing share of blocked requests.
• Alert of high rate of legitimate requests.

Alert of increasing share of blocked requestsAlert of increasing share of blocked requests

The alert will be triggered when the number of blocked requests reaches 50% or more of the total number of requests. This alert can help detect different types of attacks, e.g., DDoS or bot-based hack attempts.

Let's use letters to represent the number of incoming requests per second:

• A: Total number of requests.
• B: Number of requests blocked by security profile.

Let's set up our alerts:

• B is 30% of A: Warning (Warning).
• B is 50% of A: Critical level (Alarm).

Creating a notification channelCreating a notification channel

1. In the management console, select the folder on the left.

2. Select Monitoring.

3. Go to the Notification channels section.

4. Click Create channel.

5. Specify the channel name, notification method, and recipients.

   Note

   To get notifications, the user must:

   • Have the monitoring.viewer role for the folder with the configured alert.
   • In the management console settings:
     • Enable Monitoring.
     • Specify an email address, phone number, and Telegram account or group.

6. Click Create.

Selecting metrics for monitoringSelecting metrics for monitoring

• We will get the A value using the load_balancer.smart_web_security.requests_per_second metric.
• We will get the B value using the load_balancer.smart_web_security.requests_per_second metric filtered by label antirobot_verdict = deny.
• We will calculate the B to A ratio (in percent) using the formula 100 * B / A and save it as C.

Creating an alertCreating an alert

1. On the Monitoring homepage, click Create alert.

2. Name your alert, e.g., exceeding-blocked-requests.

3. Describe your request to get the A value under Alerts config.

   1. Click next to and add these parameters:
   • service = Application Load Balancer
   • name = load_balancer.smart_web_security.requests_per_second
   • antirobot_verdict = allow
   • load_balancer = <load_balancer_name>.
   1. Click next to the icon.
   2. Specify the data processing function replace_nan(0): Replaces the lacking data with 0 for a continuous chart.

4. Click Add query.

5. Describe your request to get the B value:

   1. Specify data to collect:
   • service = Application Load Balancer
   • name = load_balancer.smart_web_security.requests_per_second
   • antirobot_verdict = deny
   • load_balancer = <load_balancer_name>.
   1. Specify the replace_nan(0) function.

6. Click Add query.

7. Describe you request for C to get the B to A ratio in percent:

   1. Click to switch to text mode to edit the request.
   2. Enter 100 * B / A in the request string.

8. Under Alert parameters, specify:

   • Test request: C
   • Aggregation function: All values
   • Warning: 30 (warning)
   • Alarm: 50 (critical level)
   • Evaluation window: 30 seconds
   • Evaluation delay: 15 seconds

9. Leave the default values under No data policy.

10. Optionally, under Annotations, add the information that must be written when the alert is triggered.

11. Under Notifications, add the notification channel.

12. Click Create.

Alert of high rate of legitimate requestsAlert of high rate of legitimate requests

Let's use letters to represent the number of incoming requests per second:

• A: Allowed by the security profile.
• B: Allowed the ARL profile.

Normally A must be slightly greater or equal to B. In other words, the number of requests allowed by the security profile slightly exceeds the limit set in the ARL profile. Which means the majority of requests allowed by the security profile will reach the virtual host. In which case the system load is within the norm.

A significant difference in the number of requests means the system is unable to cope with the load and blocks potentially legitimate requests. This can be down to insufficient capacity or an attack. If this situation persists, we recommend reviewing your security profile settings or rate limits.

Let's set up alerts to detect such exceeded rate limits:

• A is 30% grater than B: Warning (Warning).
• A is 50% grater than B: Critical level (Alarm).

Creating a notification channelCreating a notification channel

1. In the management console, select the folder on the left.

2. Select Monitoring.

3. Go to the Notification channels section.

4. Click Create channel.

5. Specify the channel name, notification method, and recipients.

   Note

   To get notifications, the user must:

   • Have the monitoring.viewer role for the folder with the configured alert.
   • In the management console settings:
     • Enable Monitoring.
     • Specify an email address, phone number, and Telegram account or group.

6. Click Create.

Selecting metrics for monitoringSelecting metrics for monitoring

• We will get the A value using the load_balancer.smart_web_security.requests_per_second metric filtered by label antirobot_verdict = allow.
• We will get the B value using the load_balancer.smart_web_security.arl_requests_per_second metric filtered by label arl_verdict = allow.
• We will calculate the excess of A over B (in percent) using the formula 100 * abs(A - B) / B and save it as C.

Creating an alertCreating an alert

1. On the Monitoring homepage, click Create alert.

2. Name your alert, e.g., inbound-over-ARL.

3. Describe your request to get the A value under Alerts config.

   • Click next to and add these parameters:
     • service = Application Load Balancer
     • name = load_balancer.smart_web_security.requests_per_second
     • antirobot_verdict = allow
     • load_balancer = <load_balancer_name>.
   • Click next to and set the data processing functions:
     • series_sum(): Sum of metrics with different labels for a particular point. In our example, there is one label: antirobot_verdict, but this function will be useful for other metrics.
     • replace_nan(0): Replaces the lacking data with 0 for a continuous chart.

4. Click Add query.

5. Describe your request to get the B value:

   • Specify data to collect:
     • service = Application Load Balancer
     • name = load_balancer.smart_web_security.arl_requests_per_second
     • arl_verdict = allow
     • load_balancer = <load_balancer_name>.
   • Specify the series_sum() and replace_nan(0) functions.

6. Click Add query.

7. Describe your request for C to get the difference between A and B in percent:

   1. Click to switch to text mode to edit the request.
   2. Enter 100 * abs(A - B) / B in the request string.

8. Under Alert parameters, specify:

   • Test request: C
   • Aggregation function: All values
   • Warning: 30 (warning)
   • Alarm: 50 (critical level)
   • Evaluation window: 30 seconds
   • Evaluation delay: 15 seconds

9. Leave the default values under No data policy.

10. Optionally, under Annotations, add the information that must be written when the alert is triggered.

11. Under Notifications, add the notification channel.

12. Click Create.

See alsoSee also

• Yandex Monitoring metric reference
• Query string