Cluster, host, and shard state monitoring in Yandex Managed Service for Valkey™
Data on the cluster and host state is available in the management console
Diagnostic information about cluster states is presented as graphs.
Chart update rate:
- Standard hosts and hosts with an increased RAM to vCPU ratio (
memory-optimized): 15 seconds. - Hosts with a guaranteed vCPU share under 100% (
burstable): 150 seconds.
Note
The most appropriate multiple units (MB, GB, and more) are automatically used in charts.
You can configure alerts in Yandex Monitoring to receive notifications about cluster failures. In Yandex Monitoring, there are two alert thresholds: Warning and Alarm. If the specified threshold is exceeded, you will receive alerts via the configured notification channels.
Cluster health state monitoring
To view detailed information on the health state of a Yandex Managed Service for Valkey™ cluster:
-
In the management console
, select the folder containing your cluster. -
Navigate to the Yandex Managed Service for Valkey™ service.
-
Click the name of your cluster and select the Monitoring tab.
The page that opens will display performance charts for the cluster.
To get started with Yandex Monitoring metrics, dashboards, or alerts, click Open in Monitoring in the top panel.
The following charts are displayed for a cluster:
- Hosts available for read: Cluster host read availability.
- Hosts available for write: Cluster host write availability.
Under Traffic:
-
Connected Clients: Number of open connections to services on cluster hosts.
If the cluster is sharded or uses replication, some of the connections will be used for inter-host communication within the cluster.
If you encounter errors when connecting to the cluster, inactive applications may be keeping connections open for too long. If this is the case, update the Valkey™ settings by editing the Timeout value. -
Commands Processed: Rate of command processing on each cluster host.
-
Commandstats: Rate of command calls on each cluster host.
Under Latency:
- Latencystats, p50: Median command execution time.
- Latencystats, p99: Command execution time in the 99th percentile.
- Latencystats, p99.9: Command execution time in the 99.9th percentile.
Under DB Metrics:
- DB keys: Number of keys on each cluster host.
- Evicted keys: Number of keys deleted from memory when inserting new data. Information is displayed for each cluster host.
- Cache Hit Rate: Percentage of requests for which data was retrieved from the cache. Information is displayed for each cluster host.
- Replication Lag: Replica's lag behind the master, in seconds.
- IO threads active: Number of active threads on each cluster host.
- Valkey-server OOM kills (for last hour): Number of
Out of Memoryerrors on each cluster host.
Under Resources → Memory:
- Outer memory limit: Limit and RAM usage by processes on each cluster host.
- Inner Memory limit: Limit and RAM usage on each cluster host.
- Memory fragmentation ratio: Ratio of allocated memory to actually used memory on each cluster host.
- Valkey Used Memory on Masters: RAM usage on master hosts of the cluster.
- Valkey Used Memory on Replicas: RAM usage in replicated cluster hosts.
- Replication buffer size: Size of used and available memory for the replication buffer on each cluster host.
- Client recent max input buffer size: Maximum buffer size for handling incoming client connections on each cluster host.
- Client recent max output buffer size: Maximum buffer size for handling outgoing client connections on each cluster host.
- Copy-on-write allocations: Amount of memory allocated by the COW mechanism when creating child processes. Information is displayed for each cluster host.
Under Resources → CPU:
- CPU usage main thread: CPU time, system and user, consumed by the main thread on each cluster host.
- CPU usage main thread on Masters: CPU time, system and user, consumed by the main thread on the cluster's master hosts.
- CPU usage main thread on Replicas: CPU time, system and user, consumed by the main thread on the cluster's replica hosts.
Under Resources → Network:
- Network usage: Amount of incoming and outgoing network traffic on each cluster host.
- Network usage on Masters: Amount of incoming and outgoing network traffic on the cluster's master hosts.
- Network usage on Replicas: Amount of incoming and outgoing network traffic on the cluster's replica hosts.
- Replication network usage: Amount of incoming and outgoing network traffic for replication on each cluster host.
- Replication network usage on Masters: Amount of incoming and outgoing network traffic for replication on the cluster's master hosts.
- Replication network usage on Replicas: Amount of incoming and outgoing network traffic for replication on the cluster's replica hosts.
Under Resources → Data:
- Disk IO: Rate of disk operations on each cluster host.
- Disk space usage percentage: Percentage of disk space used on each cluster host.
- Disk space usage: Amount of disk space used on each cluster host.
Under Errors:
- Errorstats: Rate of failed queries on each cluster host.
- Sentinel error replies: Number of Valkey™ Sentinel's error responses on each cluster host.
- ACL errors: Rate of access errors on each cluster host.
Under Slowlog:
- Slowlog top operations on Master: Number of records for each command in the Slowlog on the cluster's master hosts.
- Slowlog top operations on Replicas: Number of records for each command in the Slowlog on the cluster's replica hosts.
- Slowlog new records: Number of new records in the Slowlog on each cluster host.
Host state monitoring
To view detailed information on the state of individual Yandex Managed Service for Valkey™ hosts:
- In the management console
, select the folder containing your cluster. - Navigate to the Yandex Managed Service for Valkey™ service.
- Click the name of your cluster and select Hosts → Monitoring.
- Select the host from the drop-down list.
This page displays the charts showing workloads of individual cluster hosts:
- CPU usage: Processor core workload. As the workload goes up, the
idlevalue goes down. - Disk read/write bytes: Speed of disk operations, in bytes per second.
- Disk IOPS: Number of disk operations per second.
- Disk space usage: Amount of used and total disk space.
- Memory usage: Amount of RAM used, in bytes. At high workloads, the
Freevalue goes down, while the other values go up. - Network bytes: Network data exchange rate, in bytes per second.
- Network packets: Network packet exchange rate, in packets per second.
The Disk read/write bytes and the Disk IOPS charts show the increase of the Read value during database read activity, and in Write, during database write activity.
For Replica hosts, the Received value is normally greater than Sent on the Network bytes and Network packets charts.
Shard state monitoring
To view detailed information on the health state of Yandex Managed Service for Valkey™ shards:
-
In the management console
, select the folder containing your cluster. -
Navigate to the Yandex Managed Service for Valkey™ service.
-
Click the name of your cluster and select the Monitoring tab.
-
Navigate to the Shards tab and select a shard.
The page that opens will display health state charts for the selected shard and its hosts.
To get started with Yandex Monitoring metrics, dashboards, or alerts, click Open in Monitoring in the top panel.
The following charts are displayed for shards:
- Hosts available for read: Shard host read availability.
- Hosts available for write: Shard host write availability.
Under Traffic:
-
Connected Clients: Number of open connections to services on shard hosts.
If the cluster is sharded or uses replication, some of the connections will be used for inter-host communication within the cluster.
If you encounter errors when connecting to the cluster, inactive applications may be keeping connections open for too long. If this is the case, update the Valkey™ settings by editing the Timeout value. -
Commands Processed: Rate of command processing on each shard host.
-
Commandstats: Rate of command calls on each shard host.
Under Latency:
- Latencystats, p50: Median command execution time on the shard.
- Latencystats, p99: Command execution time in the 99th percentile on the shard.
- Latencystats, p99.9: Command execution time in the 99.9th percentile on the shard.
Under DB Metrics:
- DB keys: Number of keys on each shard host.
- Evicted keys: Number of keys deleted from memory when inserting new data. Information is displayed for each shard host.
- Cache Hit Rate: Percentage of requests for which data was retrieved from the cache. Information is displayed for each shard host.
- Replication Lag: Replica's lag behind the master, in seconds.
- IO threads active: Number of active threads on each shard host.
- Valkey-server OOM kills (for last hour): Number of
Out of Memoryerrors on each shard host.
Under Resources → Memory:
- Outer memory limit: Limit and RAM usage by processes on each shard host.
- Inner Memory limit: Limit and RAM usage on each shard host.
- Memory fragmentation ratio: Ratio of allocated memory to actually used memory on each shard host.
- Valkey Used Memory on Masters: RAM usage on the shard's master hosts.
- Valkey Used Memory on Replicas: RAM usage in replicated shard hosts.
- Replication buffer size: Size of used and available memory for the replication buffer on each shard host.
- Client recent max input buffer size: Maximum buffer size for handling incoming client connections on each shard host.
- Client recent max output buffer size: Maximum buffer size for handling outgoing client connections on each shard host.
- Copy-on-write allocations: Amount of memory allocated by the COW mechanism when creating child processes. Information is displayed for each shard host.
Under Resources → CPU:
- CPU usage main thread: CPU time, system and user, consumed by the main thread on each shard host.
- CPU usage main thread on Masters: CPU time, system and user, consumed by the main thread on the shard's master hosts.
- CPU usage main thread on Replicas: CPU time, system and user, used by the main thread on the shard's replica hosts.
Under Resources → Network:
- Network usage: Amount of incoming and outgoing network traffic on each shard host.
- Network usage on Masters: Amount of incoming and outgoing network traffic on the shard's master hosts.
- Network usage on Replicas: Amount of incoming and outgoing network traffic on the shard's replica hosts.
- Replication network usage: Amount of incoming and outgoing network traffic for replication on each shard host.
- Replication network usage on Masters: Amount of incoming and outgoing network traffic for replication on the shard's master hosts.
- Replication network usage on Replicas: Amount of incoming and outgoing network traffic for replication on the shard's replica hosts.
Under Resources → Data:
- Disk IO: Rate of disk operations on each shard host.
- Disk space usage percentage: Percentage of disk space used on each shard host.
- Disk space usage: Amount of disk space used on each shard host.
Under Errors:
- Errorstats: Rate of failed queries on each shard host.
- Sentinel error replies: Number of Valkey™ Sentinel's error responses on each shard host.
- ACL errors: Rate of access errors on each shard host.
Under Slowlog:
- Slowlog top operations on Master: Number of records for each command in the Slowlog on the shard's master hosts.
- Slowlog top operations on Replicas: Number of records for each command in the Slowlog on the shard's replica hosts.
- Slowlog new records: Number of new records in the Slowlog on each shard host.
Setting up alerts in Yandex Monitoring
To configure state indicator alerts for a cluster and hosts:
- In the management console
, select the folder containing the cluster where you want to set up alerts. - Go to
Monitoring. - Under Service dashboards, select:
- Yandex Managed Service for Valkey™ — Cluster Overview to set up cluster alerts.
- Yandex Managed Service for Valkey™ — Host Overview to set up host alerts.
- In the chart you need, click
and select Create alert. - If the chart shows multiple metrics, select the data query to generate a metric and click Continue. You can learn more about the query language in this Yandex Monitoring article.
- Set the
AlarmandWarningalert thresholds. - Click Create alert.
To have other cluster health indicators monitored automatically:
- Create an alert.
- Add a status metric.
- In the alert parameters, set the alert thresholds.
Recommended threshold values for selected metrics:
| Metric | Internal metric name | Alarm |
Warning |
|---|---|---|---|
| Database write availability | can_write |
Equal to 0 |
— |
| Number of out-of-memory errors, per hour | redis_oom_count |
More than 2 |
More than 0 |
| RAM utilization (only for the noeviction policy) | redis_used_memory |
90% of RAM | 75% of RAM |
| Storage space used | disk.used_bytes |
90% of the storage size | 80% of the storage size |
For the disk.used_bytes metric, the Alarm and Warning thresholds are only set in bytes. For example, the recommended values for a 100 GB disk are as follows:
Alarm:96636764160bytes (90%).Warning:85899345920bytes (80%).
You can view the current RAM of the hosts in the detailed information about the cluster. For a complete list of supported metrics, see this Monitoring guide.
Cluster health and status
The State of a cluster shows the health of its hosts, while the Status shows whether the cluster is started, stopped, or is at an intermediate stage.
To check the cluster state and status:
- In the management console
, select the folder containing your cluster. - Navigate to the Yandex Managed Service for Valkey™ service.
- In the cluster row, hover over the indicator in the Availability column.
Cluster health states
| State | Description | Suggested actions |
|---|---|---|
| ALIVE | Cluster is operating normally. | No action is required. |
| DEGRADED | Cluster is not running at its full capacity: the state of at least one of the hosts is other than ALIVE. |
Run the diagnostics:
|
| DEAD | The cluster is down: none of its hosts are running. | Make a support request
|
| UNKNOWN | Cluster state is unknown. | Make a support request
|
Cluster statuses
| Status | Description | Suggested actions |
|---|---|---|
| CREATING | Preparing for the first start | Wait a while and get started. The time it takes to create a cluster depends on the host class. |
| RUNNING | The cluster is operating normally | No action is required. |
| STOPPING | The cluster is stopping | After a while, the cluster status will switch to STOPPED and the cluster will be disabled. No action is required. |
| STOPPED | The cluster is stopped | Start the cluster to get it running again. |
| STARTING | Starting the cluster that was stopped earlier | After a while, the cluster status will switch to RUNNING. Wait a while and get started. |
| UPDATING | Updating the cluster's configuration | Once the update is complete, the cluster will get the status it had prior to the update: RUNNING or STOPPED. |
| ERROR | Error when performing an operation with the cluster or during a maintenance window | If the cluster remains in this status for a long time, contact support |
| STATUS_UNKNOWN | The cluster is unable to determine its status | If the cluster remains in this status for a long time, contact support |