Resource groups
By using resource groups, users with the mdb_admin role can set quotas for the following resources allocated for query processing:
- CPU computing resources
- Amount of RAM
- Number of concurrent transactions
A resource group may include several roles that will share their quotas.
Yandex MPP Analytics for PostgreSQL gets a transaction query from the user and matches its resource requirements with the unused quotas in the resource group. If the quotas are sufficient, the query execution begins immediately. If not, the query waits for other queries to be completed and to free up the resources. Queries are queued based on the first in, first out (FIFO) principle.
Clusters with Apache Cloudberry™ and Greenplum® DBMS’s have different resource group parameters, default resource groups, and RAM allocation.
Learn how to manage resource groups in this section.
Resource group parameters
|
Name |
Description |
|
|
Maximum number of concurrent transactions a resource group can process. Both active and pending transactions are counted. The minimum value is |
|
|
Maximum CPU percentage a resource group can use on a segment. The minimum value is For more information, see Specifics of CPU resource allocation. |
|
|
Resource group priority for CPU allocation. The minimum value is |
|
|
Numbers of logical CPU cores allocated to a resource group. It is set for the master and segments as separate values or ranges. The default value is
Warning For stable cluster operation, do not allocate core |
|
|
Maximum read and write throughput, as well as the maximum number of IOPS for a resource group. Set separately for each table space:
For each
|
|
|
Amount of RAM, in MB, a resource group can use on a segment. The default value is For more information, see Specifics of RAM allocation. |
|
|
Minimum cost of query plan sufficient to assign a query to a resource group. The default value is |
|
Name |
Description |
|
|
Memory auditor used by a resource group. Yandex MPP Analytics for PostgreSQL uses |
|
|
Maximum number of concurrent transactions a resource group can process. Both active and pending transactions are counted. The default value is |
|
|
CPU percentage a resource group can use on a segment. The minimum value is |
|
|
Numbers of allocated CPU cores. The numbers or intervals are enclosed in quotes, e.g., Warning For stable cluster operation, do not allocate core |
|
|
RAM percentage a resource group can use on a segment. The minimum value is |
|
|
Percentage of For more information, see Specifics of RAM allocation. |
|
|
RAM usage threshold for transactions as a percentage of |
Default resource groups
When you create a cluster with an Apache Cloudberry™ DBMS, the following resource groups are created:
-
admin_group: Assigned to the roles with the SUPERUSER attribute. -
system_group: Used by Apache Cloudberry™ system processes. You cannot assign this group manually. -
mdb_admin_group: Group for administrators. By default, it is assigned to the administrator whose username and password were specified when creating the cluster.If you reach the resource limit for this group, the admin will not be able to connect to the cluster and terminate resource-hungry processes. This is why we recommend assigning
mdb_admin_grouponly to users with themdb_adminrole. -
default_group: Assigned to other roles if no resource group was set when creating them.
Resource group parameter values:
| Parameter | admin_group |
system_group |
mdb_admin_group |
default_group |
|---|---|---|---|---|
CONCURRENCY |
20 |
0 |
20 |
20 |
CPU_MAX_PERCENT |
-1 |
10 |
1 |
20 |
CPU_WEIGHT |
100 |
100 |
100 |
100 |
CPUSET |
-1 |
-1 |
-1 |
-1 |
IO_LIMIT |
-1 |
-1 |
-1 |
-1 |
MEMORY_QUOTA |
-1 |
-1 |
-1 |
-1 |
MIN_COST |
0 |
500 |
0 |
500 |
You cannot change parameter values for admin_group.
For more information about managing roles, see Users and roles and this Apache Cloudberry™ guide
When you create a cluster with a Greenplum® DBMS DBMS, the following resource groups are created:
-
admin_group: Service group assigned to themonitorandgpadminservice roles. -
mdb_admin_group: Group for administrators. By default, it is assigned to the administrator whose username and password were specified when creating the cluster.If the administrator and other users belong to the same resource group, and all its resources are used up, the administrator will not be able to connect to the database and interrupt resource-intensive processes manually. This is why we recommend assigning this resource group only to the users with the
mdb_adminrole. -
default_group: Shared group. It is assigned to a newly created role unless it is assigned another explicit resource group.
Resource group parameter values:
| Parameter | admin_group |
mdb_admin_group |
default_group |
|---|---|---|---|
MEMORY_AUDITOR |
vmtracker |
vmtracker |
vmtracker |
CONCURRENCY |
10 |
20 |
20 |
CPU_RATE_LIMIT |
10 |
1 |
30 |
CPUSET |
-1 |
-1 |
-1 |
MEMORY_LIMIT |
10 |
0 |
0 |
MEMORY_SHARED_QUOTA |
80 |
80 |
80 |
MEMORY_SPILL_RATIO |
0 |
0 |
0 |
You cannot change parameter values for admin_group.
For more information about managing roles, see Users and roles and this Greenplum® guide
Specifics of RAM allocation
-
If there is a limit on RAM per resource group (the
MEMORY_QUOTAparameter), RAM management takes place at the segment level.RAM per query is decided by the
gp_resgroup_memory_query_fixed_memsetting. If it is not set, RAM per query is calculated using this formula:MEMORY_QUOTA ÷ CONCURRENCY.Read more about the
gp_resgroup_memory_query_fixed_memsetting in this Greenplum® guide .For more information on additional criteria affecting RAM per query, see this Apache Cloudberry™ guide.
-
If there is no limit on RAM per resource group, RAM management takes place at the query level. RAM per query is decided by the statement_mem setting.
-
You can allocate a segment's RAM share for each resource group using the
MEMORY_LIMITparameter. This percentage is split into the shared quota and guaranteed quota:-
The shared quota is set in the
MEMORY_SHARED_QUOTAparameter and can be used by all the resource group transactions. Yandex MPP Analytics for PostgreSQL distributes the shared quota to transactions based on the first in, first out principle. -
The guaranteed quota is allocated to each transaction and calculated by the following formula:
(MEMORY_LIMIT − MEMORY_SHARED_QUOTA) ÷ CONCURRENCY
-
-
If the total of
MEMORY_LIMITvalues of all resource groups is below100, Yandex MPP Analytics for PostgreSQL distributes unallocated RAM to any transactions from any resource groups based on the first in, first out principle.The transaction uses non-allocated memory when both conditions are met:
- The guaranteed memory share for the transaction is used up.
- The shared quota for the resource group is used up.
For your cluster to run in a stable and efficient manner, leave 10% to 20% of memory unallocated.
Specifics of CPU resource allocation
You can allocate CPU resources using one of the following methods:
-
By setting a CPU percentage per resource group on a segment. In Apache Cloudberry™, the parameter used to set CPU percentage is
CPU_MAX_PERCENT, while in Greenplum®, it isCPU_RATE_LIMIT.If a resource group is idle, its resources get re-allocated towards highly loaded groups. In Apache Cloudberry™, resource group priority (the
CPU_WEIGHTparameter) is another factor affecting CPU resource allocation. As soon as the resource group stops idling, it gets its re-allocated CPU resources back. -
By specifying the numbers of logical CPU cores you are allocating to the resource group (in the
CPUSETparameter). These cores will be unavailable to other resource groups even if the resource group they are allocated to is idle. Therefore, we recommend minimizing the number of resource groups with allocated cores.Warning
For stable cluster operation, do not allocate core
0.
CPU allocation methods may vary from one resource group to another.
For more on resource groups, see:
Greenplum® and Greenplum Database® are registered trademarks or trademarks of Broadcom Inc. in the United States and/or other countries.
Apache® and Apache Cloudberry™ are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.