Creating a composite alert
A composite alert figures out its status based on the statuses of other alerts. It helps you set the overall alert logic for several related checks. For example, you can count an issue only if several alerts get the Alarm status at the same time.
Note
To view alerts, you need the viewer role or higher for the relevant cloud. Learn more about roles in Access management in Monium.
-
On the Monium
home page, select Alerts and SLOs → Alerts on the left. -
In the top-right corner, click Create alert → Composite.
-
Specify the main alert parameters:
-
Name: Alert name.
-
ID: Unique alert ID with the project's prefix. If not set, it is generated automatically. Once the alert is created, you cannot change its ID.
-
Description: Alert description or comment.
-
Severity: Alert criticality level. Select it based on the event effect:
Unspecified: Draft or a test without a level.Disaster: Service unavailability or data loss risk.Critical: Service degradation or lack of resources.Info: Scheduled operation or a test alert.
-
-
Under Alert selector, specify labels to search for source alerts.
For example,
severity = "Critical"(groups critical alerts),status = "Alarm"(groups triggered alerts), ornotification_channel.id = "<channel_ID>"(groups alerts from a specific notification channel). -
Under Trigger conditions, select a composition strategy (method used to determine the final alert status):
- By share: This status will be assigned to the alert as soon as a target percentage of selected alerts get the specified statuses. Use this strategy for similar types of checks, e.g., based on the share of unavailable hosts in the cluster.
- By count: This status will be assigned to the alert as soon as the target number of selected alerts get the specified statuses. Use the fixed quorum strategy, e.g., when it is enough to have three triggered alerts across the availability zones.
-
Configure the conditions for the Warning and Alarm statuses:
- Set a threshold: percentage or quantity. For Warning, set an early warning threshold, e.g.,
20% of problematic instances. For Alarm, set a threshold that requires a response from the operator on duty. - Select the statuses of the source alerts to consider. If no statuses are selected, alerts with the
WarningandAlarmstatuses are counted.
When the threshold is reached, the composite alert will get the appropriate status. For example, for Alarm, the selected statuses are
AlarmandError, with the threshold of3. In this case, the composite will goAlarmas soon as a total of at least three source alerts get theAlarmandErrorstatuses. - Set a threshold: percentage or quantity. For Warning, set an early warning threshold, e.g.,
-
Optionally, configure additional conditions:
- Force the composite status to No Data: The composite alert gets the
No Datastatus if at least one source alert is in theNo Datastatus. Use this setting if a data loss for a single critical source is more important than the rest of the conditions. - Force the composite status to Error: The composite alert gets the
Errorstatus if at least one source alert is in theErrorstatus. This prevents concealing the source alert evaluation error. - Treat muted alerts as Ok: Muted alerts are counted as the
Okstatus. Use this setting for scheduled maintenance so that muting one source alert does not elevate the status of the composite.
- Force the composite status to No Data: The composite alert gets the
-
Optionally, specify annotations for the alert.
You can add new annotations or select those specified in the source alerts.
-
Optionally, to sort alerts and search for alerts, add labels in
key=valueformat.You can add multiple labels. Do it by entering a
key=valuepair and clicking Enter. To use labels in another alert or the search bar, click Copy all labels. -
Optionally, under Links, add links to related resources.
-
Configure notifications. If you do not have a notification channel, create one.
To avoid receiving notifications every time the status changes, enable Noise suppression. Use this setting if the source alerts change their status frequently near the composite threshold. For more on noise suppression, see Noise suppression.
-
Check the alert evaluation result.
In the top right, click Check result. Monium will reevaluate the alert based on the current settings and display the resulting status, chart, and variable values. The check makes it possible to evaluate data availability and the correctness of alert settings.
-
Click Create. The alert will appear in the list.