Monitoring and logs

• What metrics and processes can be monitored?

• How is log storage billed?

• How longs are logs retained?

• How do I monitor free storage space on ZooKeeper hosts?

• How do I monitor space taken up by data in hybrid storage?

• How do I set up an alert to signal when my disk space is a certain percentage full?

What metrics and processes can be monitored?What metrics and processes can be monitored?

For all DBMS types, you can monitor:

• CPU, memory, network, and disk utilization, in absolute terms.
• Memory, network, and disk utilization as a percentage of the set limits for the cluster’s host class.
• Total cluster data size and available storage space remaining.

For all database hosts, you can monitor metrics specific to their DBMS type. For example, for PostgreSQL, you can monitor:

• Average query execution time.
• Number of queries per second.
• Number of errors in logs, and more.

You can monitor metrics with a minimum granularity of five seconds.

How is log storage billed?How is log storage billed?

Logs of any level are written to a disk's system partition (with 20 GB allocated) and are not charged separately. The size of the logs only affects the frequency of their rotation.

What is the retention period for logs?What is the retention period for logs?

Cluster logs are stored for 45 days.

How do I monitor free storage space on ZooKeeper hosts?How do I monitor free storage space on ZooKeeper hosts?

Follow the steps in this guide to monitor the host state or set up alerts.

How do I monitor space taken up by data in hybrid storage?How do I monitor space taken up by data in hybrid storage?

Use the ch_s3_disk_parts_size metric in Yandex Monitoring. It shows the amount of space used by MergeTree table parts in Object Storage. The metric is only available for Managed Service for ClickHouse® clusters with hybrid storage configured. For more on how to run a query with a metric, see this tutorial.

How can I configure an alert for when disk usage reaches a certain percentage?How can I configure an alert for when disk usage reaches a certain percentage?

Create an alert for the disk.used_bytes metric in Yandex Monitoring. This metric shows the disk space usage in the Managed Service for ClickHouse® cluster.

For disk.used_bytes, use notification thresholds. Their recommended values are as follows:

• Alarm: 95% of disk space
• Warning: 80% of disk space

Thresholds are set in bytes only. For example, the recommended values for a 100 GB disk are as follows:

• Alarm: 102,005,473,280 bytes (95%)
• Warning: 85,899,345,920 bytes (80%)