Monitoring and logs

• What metrics and processes can be tracked using monitoring?

• How is log storage billed?

• How do I track free storage space on ZooKeeper hosts?

• How do I monitor space taken up by data in hybrid storage?

• How do I set up an alert that triggers as soon as a certain percentage of disk space has been used up?

What metrics and processes can be tracked using monitoring?What metrics and processes can be tracked using monitoring?

For all DBMS types, you can track:

• CPU, memory, network, or disk usage, in absolute terms.
• Memory, network, or disk usage as a percentage of the set limits for the corresponding cluster host class.
• The amount of data in the DB cluster and the remaining free space in data storage.

For DB hosts, you can track metrics specific to the corresponding type of DBMS. For example, for PostgreSQL, you can track:

• Average query execution time.
• Number of requests per second.
• Number of errors in logs.

Monitoring can be performed with a minimum granularity of 5 seconds.

How is log storage charged?How is log storage charged?

Logs of any level are written to a disk's system partition with 20 GB allocated, so you are not charged for them separately. The size of the logs created only affects log rotation frequency.

What is the retention period for logs?What is the retention period for logs?

Cluster logs are stored for 30 days.

How do I track the amount of free storage space on ZooKeeper hosts?How do I track the amount of free storage space on ZooKeeper hosts?

Follow the steps in Monitoring the state of clusters and hosts to track the host state or set up alerts.

How do I monitor space used by data in hybrid storage?How do I monitor space used by data in hybrid storage?

Use the ch_s3_disk_parts_size metric in Yandex Monitoring. It shows the amount of space used by MergeTree table parts in Object Storage. The metric is only available for Managed Service for ClickHouse® clusters with the hybrid storage configured. You can learn how to make a request using the metric in this guide.

How do I set up an alert that triggers as soon as a certain percentage of disk space has been used up?How do I set up an alert that triggers as soon as a certain percentage of disk space has been used up?

Create an alert with the disk.used_bytes metric in Yandex Monitoring. This metric shows the disk space usage in the Managed Service for ClickHouse® cluster.

For disk.used_bytes, use notification thresholds. The recommended values are as follows:

• Alarm: 95% of the disk space
• Warning: 80% of the disk space

Thresholds are set in bytes only. For example, the recommended values for a 100 GB disk are as follows:

• Alarm: 102,005,473,280 bytes (95%)
• Warning: 85,899,345,920 bytes (80%)