Monitoring and logs

• What metrics and processes can be monitored?

• How is log storage billed?

• How do I monitor free storage space on ZooKeeper hosts?

• How do I monitor space taken up by data in hybrid storage?

• How do I set up an alert that triggers as soon as a certain percentage of disk space is used up?

What metrics and processes can be monitored?What metrics and processes can be monitored?

For all DBMS types, you can monitor:

• CPU, memory, network, or disk usage, in absolute terms.
• Memory, network, and disk usage as a percentage of the set limits for the relevant cluster host class.
• Amount of data in a database cluster and the remaining free space in your data storage.

For all database hosts, you can monitor metrics specific to their DBMS type. For example, for PostgreSQL, you can monitor:

• Average query execution time.
• Number of queries per second.
• Number of errors in logs, and more.

You can monitor metrics with a minimum granularity of five seconds.

How is log storage billed?How is log storage billed?

Logs of any level are written to a disk's system partition (with 20 GB allocated) and are not charged separately. The size of the logs only affects the frequency of their rotation.

What is the retention period for logs?What is the retention period for logs?

Cluster logs are stored for 30 days.

How do I monitor free storage space on ZooKeeper hosts?How do I monitor free storage space on ZooKeeper hosts?

Follow the steps in this guide to monitor the host state or set up alerts.

How do I monitor space taken up by data in hybrid storage?How do I monitor space taken up by data in hybrid storage?

Use the ch_s3_disk_parts_size metric in Yandex Monitoring. It shows the amount of space used by MergeTree table parts in Object Storage. The metric is only available for Managed Service for ClickHouse® clusters with hybrid storage configured. To learn more on how to run a query with a metric, see this tutorial.

How do I set up an alert that triggers as soon as a certain percentage of disk space is used up?How do I set up an alert that triggers as soon as a certain percentage of disk space is used up?

Create an alert for the disk.used_bytes metric in Yandex Monitoring. This metric shows the disk space usage in the Managed Service for ClickHouse® cluster.

For disk.used_bytes, use notification thresholds. Their recommended values are as follows:

• Alarm: 95% of disk space
• Warning: 80% of disk space

Thresholds are set in bytes only. For example, the recommended values for a 100 GB disk are as follows:

• Alarm: 102,005,473,280 bytes (95%)
• Warning: 85,899,345,920 bytes (80%)