Contact UsTry it for free
© 2026 Direct Cursus Technology L.L.C.

Apache Kafka® cluster and host state monitoring

Written by
Updated at January 22, 2026

Data on the cluster and host state is available in the management console. You can view them on the Monitoring tab of the cluster management page or in Yandex Monitoring.

Diagnostic information about cluster states is presented as graphs.

Chart update rate:

  • Standard hosts and hosts with an increased RAM to vCPU ratio (memory-optimized): 15 seconds.
  • Hosts with a guaranteed vCPU share under 100% (burstable): 150 seconds.

Note

The most appropriate multiple units (MB, GB, and more) are automatically used in charts.

You can configure alerts in Yandex Monitoring to receive notifications about cluster failures. In Yandex Monitoring, there are two alert thresholds: Warning and Alarm. If the specified threshold is exceeded, you will receive alerts via the configured notification channels.

Cluster state monitoring

To view detailed information on the state of a Managed Service for Apache Kafka® cluster:

  1. In the management console, navigate to the relevant folder.

  2. Go to Managed Service for Kafka.

  3. Click the name of your cluster and select the Monitoring tab.

  4. To get started with Yandex Monitoring metrics, dashboards, or alerts, click Open in Monitoring in the top panel.

The page displays the following charts:

  • Alive brokers: Number of functional brokers for each host with the KAFKA role.
  • Offline partitions: Displays the parameter values below:
    • OfflineReplicaCount: Number of partitions with no leader broker. These partitions do not support message writes or reads.
    • Underreplicated partitions: Number of partitions with ISR count below the replication factor.
    • Under min ISR partitions: Number of partitions with ISR count below the minimum value specified in the settings.
  • Errors: Number of failed requests by error type.
  • Free space: Free disk space for each host with the KAFKA role, in bytes.
  • Messages in: Message write rate, in messages per second.
  • Bytes In/Out: Message write and read rate for each host with the KAFKA role, in bytes per second.
  • Top 5 topics by size: Amount of data for each of the five largest topics, in bytes.
  • Replicated bytes: Replication data streaming rate for each host with the KAFKA role, in bytes per second.
  • Partitions: Total number of partition replicas.
  • Replication lag [messages]: Greatest replication lag for each host with the KAFKA role, in messages.
  • Request time (0.95 quantile): Request processing time in the 0.95 quantile by request type.
  • Requests: Request rate by request type, in requests per second.

Host state monitoring

To view detailed information on the state of individual Managed Service for Apache Kafka® hosts:

  1. In the management console, navigate to the relevant folder.
  2. Go to Managed Service for Kafka.
  3. Click the name of your cluster and select Hosts → Monitoring.
  4. Select the host from the drop-down list.

This page displays the charts showing workloads of individual cluster hosts:

  • CPU usage: Processor core workload. As the workload goes up, the Idle value goes down.
  • Memory usage: Amount of RAM used, in bytes. At high workloads, the Free value goes down, while the other values go up.
  • Disk read/write bytes: Speed of disk operations, in bytes per second.
  • Disk IOPS: Number of disk operations per second.
  • Network bytes: Network data exchange rate, in bytes per second.
  • Network packets: Network packet rate, in packets per second.
  • Free space: Free disk space, in bytes.

Setting up alerts in Yandex Monitoring

To configure state indicator alerts for a cluster and hosts:

  1. In the management console, select the folder containing the cluster for which you want to set up alerts.
  2. Go to Monitoring.
  3. Under Service dashboards, select Managed Service for Apache Kafka® — Cluster Overview.
  4. In the chart you need, click and select Create alert.
  5. If the chart shows multiple metrics, select the data query to generate a metric and click Continue. You can learn more about the query language in this Yandex Monitoring article.
  6. Set the Alarm and Warning threshold values to trigger the alert.
  7. Click Create alert.

To have other cluster health indicators monitored automatically:

  1. Create an alert.
  2. Add a status metric.
  3. In the alert parameters, set the alert thresholds.

The recommended thresholds for some metrics are as follows:

Metric Designation Alarm Warning
Number of healthy hosts kafka_is_alive <number_of_hosts> - 2 <number_of_hosts> - 1
Partition replication state kafka_server_ReplicaManager_UnderReplicatedPartitions — More than 0
Number of lagging replicas kafka_server_ReplicaManager_UnderMinIsrPartitionCount More than 0 —
Storage space used disk.used_bytes 90% of the storage size 80% of the storage size

For the disk.used_bytes metric, the Alarm and Warning thresholds are only set in bytes. For example, the recommended values for a 100 GB disk are as follows:

  • Alarm: 96,636,764,160 bytes (90%)
  • Warning: 85,899,345,920 bytes (80%)

You can check the current storage size in the cluster details. For a complete list of supported metrics, see this Monitoring guide.

Cluster state and status

The State of a cluster shows the health of its hosts, while the Status shows whether the cluster is started, stopped, or is at an intermediate stage.

To check the cluster's state and status:

  1. In the management console, navigate to the relevant folder.
  2. Go to Managed Service for Kafka.
  3. In the cluster row, hover over the indicator in the Availability column.

Cluster states

State Description Suggested actions
ALIVE Cluster is operating normally. No action is required.
DEGRADED Cluster is not running at its full capacity: the state of at least one of the hosts is other than ALIVE. Run the diagnostics:
  • Go to the Hosts tab and see which hosts are not working.
  • Go to the Operations tab and make sure all operations are completed.
  • Make sure the cluster is not under maintenance.
If you cannot find the cause yourself, contact support.
DEAD The cluster is down: none of its hosts are running. Make a support request stating the following:
  • Cluster ID.
  • IDs of the last operations performed on it.
  • Time the cluster entered the DEAD state according to the availability charts.
UNKNOWN Cluster state is unknown. Make a support request stating the following:
  • Cluster ID.
  • IDs of the last operations performed on it.
  • Time the cluster entered the UNKNOWN state according to the availability charts.

Cluster statuses

Status Description Suggested actions
CREATING Preparing for the first start Wait a while and get started. The time it takes to create a cluster depends on the host class.
RUNNING The cluster is operating normally No action is required.
STOPPING The cluster is stopping After a while, the cluster status will switch to STOPPED and the cluster will be disabled. No action is required.
STOPPED The cluster is stopped Start the cluster to get it running again.
STARTING Starting the cluster that was stopped earlier After a while, the cluster status will switch to RUNNING. Wait a while and get started.
UPDATING Updating the cluster's configuration Once the update is complete, the cluster will get the status it had prior to the update: RUNNING or STOPPED.
ERROR Error when performing an operation with the cluster or during a maintenance window If the cluster remains in this status for a long time, contact support. You can see whether a cluster is available by its status.
STATUS_UNKNOWN The cluster is unable to determine its status If the cluster remains in this status for a long time, contact support.
Previous
Viewing cluster logs
Next
All tutorials