Yandex MPP Analytics for PostgreSQL cluster and host state monitoring

1. Open the folder dashboard.

2. Navigate to Yandex MPP Analytics for PostgreSQL.

3. Click the name of your cluster and select the Monitoring tab.

   This opens a page displaying a dashboard with your Yandex MPP Analytics for PostgreSQL cluster charts and signals.

4. To get started with Yandex Monitoring metrics, dashboards, or alerts, click Open in Monium in the top panel.

The dashboard consists of the following sections:

• Signals:

  • Cluster: General cluster status.
  • Pooler: Connection pooler status.
  • Connections: Connection status.
  • Resources: Status of host resources.
  • Queries: Read/write test query status.

• Cluster:

  • Read availability: Cluster read availability over the last five minutes.
  • Write availability: Cluster write availability over the last five minutes.
  • Master switch history: Master failover history (current master FQDN).
  • XID utilization: Transaction counter utilization percentage.
  • Segments down: Number of segments currently in the down state.
  • Segments not in preferred role: Number of segments that are not in their preferred role.
  • Segments not synchronised: Number of unsynchronized segments.
  • Log events: Number of WARNING, ERROR, and FATAL events in logs.

• Pooler:

  • Availability: Pooler availability over the last five minutes.
  • CPU usage: CPU utilization by the pooler process.
  • Memory usage: Memory utilization by the pooler process.
  • Network usage: Amount of sent and received data.
  • Server connections: Active and available server connections.
  • Client connections: Active client connections.
  • Query timings: Distribution of query durations by percentiles.

• Connections:

  • Total master connections utilization: Total utilization of master connections.
  • Idle in transaction master connections utilization: Utilization of connections that are idle in transaction.
  • Master connections: Detailed master node connection breakdown by state: active, idle, idle in transaction, waiting, aborted, total (total number for all states), and max (limit).
  • Total segments connections utilization: Connection utilization by segments.

• Resources:

  • This section is split into the Master hosts and Segment hosts subgroups.
  • Load Average: Average load over 15 minutes.
  • CPU utilization: CPU utilization.
  • Virtual memory consumption: Total virtual memory used.
  • Physical memory consumption: Total physical memory used.
  • Network packets: Network packets sent and received per second.
  • Network packets in queues: Number of packets in network interface queues.
  • Network traffic: Amount of outgoing and incoming traffic.
  • Disk space utilization: Percentage of used disk space.
  • Disk IO: Disk read and write throughput in bytes per second.
  • Interconnect retransmits ratio (segments only): Ratio of resent Interconnect packets to total sent packets.

• Queries:

  • Read test query duration: Test read query duration.
  • Write test query duration: Test write query duration.
  • Read duration from each segment: Test read query duration by segments.
  • Queries sent to the cluster: Number of new and failed queries.
  • Queries execution time distribution: Histogram of historical query durations.
  • Running queries execution time distribution: Distribution histogram of active query durations.
  • Spill files count: Total and per-host count of temporary files.
  • Spill files size: Total and per-host size of temporary files.

• Resource groups:

  • Charts in this section display information by process groups:
    • admin_group: Administrative group.
    • default_group: Default group.
  • Connections utilization by resource group: Connection utilization.
  • CPU usage by resource group: CPU utilization.
  • Memory utilization by resource group: Memory utilization percentage.
  • Memory usage by resource group: Absolute value of used and available memory.
  • Running transactions by resource group: Number of running transactions.
  • Executed transactions by resource group: Transaction frequency.
  • Queueing transactions by resource group: Number of transactions in the queue.

• Background activities:

  • Vacuum: catalog:
    • Catalog size: System catalog size per database.
    • Catalog percent of dead tuples: Percentage of dead tuples within the catalog per database.
  • Vacuum: pg_attribute:
    • pg_attribute size: Size of the pg_attribute table per database.
    • pg_attribute percent of dead tuples: Percentage of dead tuples in the pg_attribute table per database.
  • Vacuum/analyze: user tables:
    • Tables vacuum age: Distribution of tables by number of days since the last vacuuming (VACUUM).
    • Tables analyze age: Distribution of tables by number of days since the last statistics collection (ANALYZE).

• Hybrid storage:

  • CPU usage per host: CPU utilization by yproxy per cluster host.
  • Memory usage per host: Memory utilization by yproxy per cluster host.
  • Storage requests: Number of storage requests (total, successful, and failed).
  • Storage objects count: Total number of objects in storage.
  • Storage objects size: Total size of objects in storage.

1. Open the folder dashboard.

2. Navigate to Yandex MPP Analytics for PostgreSQL.

3. Click the cluster name and select the Monitoring → Hosts tab.

4. Select the host you need.

   The page that opens will display state charts for the cluster host.

5. To get started with Yandex Monitoring metrics, dashboards, or alerts, click Open in Monium in the top panel.

The dashboard consists of the following sections:

• Signals:

  • CPU: Processor load.
  • Memory: Virtual memory usage and availability.
  • Disk: Used data disk space.

• CPU:

  • Load average 15 min: 15-minute average CPU load.
  • Load average 5 min: Five-minute average CPU load.
  • Load average 1 min: One-minute average CPU load.
  • CPU utilization: Percentage of CPU core utilization.
  • CPU pressure time: Time in microseconds during which processes were waiting for CPU access.
  • CPU: CPU time breakdown across states (user, system, idle, iowait, and so on).
  • Process statuses: Number of processes in various states: running, idle, interruptible sleep, uninterruptible sleep, and zombie.

• Memory:

  • Virtual memory utilization: Percentage of virtual memory used.
  • Virtual memory usage: Amount of virtual memory used, in bytes.
  • Out of memory count: Number of out-of-memory events.
  • Memory utilization: Percentage of physical RAM used.
  • Memory usage: Amount of physical memory used, in bytes.
  • Memory pressure time: Time during which processes were waiting for memory access.

• Disk:

  • Disk usage percent: Percentage of data disk space used.
  • Disk usage: Amount of data stored on disk, in bytes.
  • Disk read/write operations: Number of read and write operations per second.
  • Disk read/write bytes: Volume of data read from and written to disk per second, in bytes.
  • Disk read and write time: Average time to complete read and write operations.

• Disk Metrics Details:

  • Disk read latency (percentiles): Disk read operation latencies, percentiles.
  • Disk write latency (percentiles): Disk write operation latencies, percentiles.
  • Disk read throttler latency (percentiles): Throttler-introduced disk read operation delay, percentiles.
  • Disk write throttler latency (percentiles): Throttler-introduced disk write operation delay, percentiles.
  • Disk read operations: Average and maximum number of disk read operations per second.
  • Disk write operations: Average and maximum number of disk write operations per second.
  • Disk used quota: Percentage of disk operation quota usage.
  • Disk read bytes: Average and maximum number of bytes read from disk.
  • Disk write bytes: Average and maximum number of bytes written to disk.

• Network:

  • Client network reliability: Network interface reliability indicator.
  • Client network packets sent: Number of packets sent on the interface per second.
  • Client network packets received: Number of packets received on the interface per second.
  • Client network errors: Number of errors sending and receiving packets.
  • Client network drops: Number of dropped packets.
  • Client network interface errors/drops: Detailed breakdown of errors and drops per interface.
  • Client network bytes: Network traffic volume on the interface, in bytes per second.

1. Open the folder dashboard.

2. Navigate to Yandex MPP Analytics for PostgreSQL.

3. Click the cluster name and select the Monitoring → PXF tab.

   The page that opens will display PXF state charts.

4. To get started with Yandex Monitoring metrics, dashboards, or alerts, click Open in Monium in the top panel.

The page displays the following charts:

• Signals:

  • Availability: PXF service availability over the last five minutes.
  • Memory Utilization: Indicates RAM usage by the PXF service.
  • CPU Utilization: Indicates CPU usage by the PXF service.

• Overview: General metrics of the PXF service:

  • Unavailable instances: Unavailable PXF instances.
  • Max Memory Usage: Maximum RAM usage across all PXF instances.
  • Total OOM Events: Total number of out-of-memory events on all PXF instances over a period of time.
  • Max CPU Usage: Maximum CPU usage across all PXF instances.
  • Max PXF Threads Usage: Maximum number of busy and running PXF threads across all instances.
  • Total Log Events By Level: Total number of warn, error, and fatal log events across all PXF instances.
  • Total Sent And Received Data: Total size of data exchanged by the PXF data sources and the database per second.
  • Total Sent And Received Records: Total number of records exchanged by the PXF data sources and the database per second.
  • Total Read And Write Requests: Total number of read and write requests from the database to the data source per second.
  • Max Read And Write Requests Latency: Maximum latency of read and write requests.

• By host statistics: Metrics broken down by hosts:

  • Top 10 Hosts By Memory Usage: Top ten hosts by PXF service RAM usage.
  • OOM Events By Hosts: Number of out-of-memory events on each PXF instance for a period of time.
  • Top 10 Hosts By CPU Usage: Top ten hosts by PXF service CPU usage.
  • Top 10 Hosts By PXF Current Threads Usage: Top ten PXF instances by number of running threads.
  • Top 10 Hosts By Busy PXF Threads Usage: Top ten PXF instances by number of busy threads.
  • Top 10 Hosts By Fatal Log Events: Top ten hosts by number of fatal log events in PXF logs.
  • Top 10 Hosts By Error Log Events: Top ten hosts by number of error log events in PXF logs.
  • Top 10 Hosts By Warn Log Events: Top ten hosts by number of warn log events in PXF logs.
  • Top 10 Hosts By Sent Data: Top ten hosts by amount of data sent from PXF data sources to the database per second.
  • Top 10 Hosts By Received Data: Top ten hosts by amount of data received by PXF data sources from the database per second.
  • Top 10 Hosts By Sent Records: Top ten hosts by number of records sent from the PXF data source to the database per second.
  • Top 10 Hosts By Received Records: Top ten hosts by number of records received by PXF data sources from the database per second.
  • Top 10 Hosts By Read Requests: Top ten hosts by number of read requests from the database to the PXF data source per second.
  • Top 10 Hosts By Write Records: Top ten hosts by number of write requests from the database to the PXF data source per second.
  • Top 10 Hosts By Read Requests Latency: Top ten hosts by latency of read requests from the database to the PXF data source.
  • Top 10 Hosts By Write Requests Latency: Top ten hosts by latency of write requests from the database to the PXF data source.

• By server statistics: Metrics broken down by PXF data sources:

  • Top 10 Servers By Sent Data: Top ten PXF data sources by amount of data sent to the database per second.
  • Top 10 Servers By Received Data: Top ten PXF data sources by amount of data received from the database per second.
  • Top 10 Servers By Sent Records: Top ten PXF data sources by number of records sent to the database per second.
  • Top 10 Servers By Received Records: Top ten PXF data sources by number of records received from the database per second.
  • Top 10 Servers By Read Requests: Top ten PXF data sources by number of read requests to the database per second.
  • Top 10 Servers By Write Requests: Top ten PXF data sources by number of write requests to the database per second.
  • Top 10 Servers By Read Requests Latency: Top ten PXF data sources by database read request latency.
  • Top 10 Servers By Write Requests Latency: Top ten PXF data sources by database write request latency.

• By user statistics: Metrics broken down by users:

  • Top 10 Users By Sent Data:Top ten users by amount of data sent from PXF data sources to the database per second.
  • Top 10 Users By Received Data: Top ten users by amount of data received by PXF data sources from the database per second.
  • Top 10 Users By Sent Records: Top ten users by number of records sent from the PXF data source to the database per second.
  • Top 10 Users By Received Records: Top ten users by number of records received by PXF data sources from the database per second.
  • Top 10 Users By Read Requests: Top ten users by number of read requests from the database to the PXF data source per second.
  • Top 10 Users By Write Records: Top ten users by number of write requests from the database to the PXF data source per second.
  • Top 10 users By Read Requests Latency: Top ten users by latency of read requests from the database to the PXF data source.
  • Top 10 Users By Write Requests Latency: Top ten users by latency of write requests from the database to the PXF data source.

1. Open the folder dashboard.

2. Navigate to Yandex MPP Analytics for PostgreSQL.

3. Click the cluster name and select Monitoring → Resource groups.

   The page that opens will display performance charts for resource groups.

4. To get started with Yandex Monitoring metrics, dashboards, or alerts, click Open in Monium in the top panel.

The page displays the following charts:

• CPU Usage: Total CPU usage across all hosts in a resource group.
• Memory Usage: Total memory usage across all hosts in a resource group.
• Running queries: Current number of active transactions in a resource group.
• Queueing queries: Current number of queueing transactions for this resource group.
• Executed queries: Total number of transactions executed in a resource group since the most recent cluster startup (excluding Running queries).
• Queued queries: Total number of transactions enqueued for this resource group since the most recent cluster startup (excluding Queueing queries).

1. Open the folder dashboard.

2. Navigate to Yandex MPP Analytics for PostgreSQL.

3. Click the cluster name and select Monitoring → Resource groups per host.

   The page that opens will display performance charts for resource groups by host.

4. To get started with Yandex Monitoring metrics, dashboards, or alerts, click Open in Monium in the top panel.

The page displays the following charts:

• CPU Usage: Total CPU usage in a resource group within a host.
• Memory Usage: Total memory usage in a resource group within a host.

1. In the management console, select the folder with the cluster for which you want to set up alerts.
2. Navigate to  Monitoring.
3. Under Service dashboards, select:
   • Yandex MPP Analytics for PostgreSQL — Cluster Overview to configure cluster alerts.
   • Yandex MPP Analytics for PostgreSQL — Host Overview to configure host alerts.
4. On the indicator chart of interest, click and select Create alert.
5. If the chart shows multiple metrics, select the data query to generate the metric and click Continue. For more on the query language, see this Yandex Monitoring guide.
6. Set the Alarm and Warning thresholds for notifications.
7. Click Create alert.
8. For signals, we recommend configuring alerts for when the status switches to Crit (0) or Warn (1).

1. Create an alert.
2. Add a status metric.
3. In the alert parameters, set the alert thresholds.