Managing resource groups in Managed Service for Trino

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

To get the description of resource groups and selector rules in the Managed Service for Trino cluster, run this command:

yc managed-trino cluster get-resource-groups <cluster_name_or_ID>

You can get the cluster ID and name with the list of clusters in the folder.

1. Get an IAM token for API authentication and put it into an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Call the Cluster.Get method, e.g., via the following cURL request:

   curl \
       --request GET \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --url 'https://trino.api.cloud.yandex.net/managed-trino/v1/clusters/<cluster_ID>'
   

   You can request the cluster ID with the list of clusters in the folder.

3. The description of resource groups and selector rules in the Managed Service for Trino cluster is in the resourceGroups field of the server response.

1. Get an IAM token for API authentication and put it into an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume that the repository contents reside in the ~/cloudapi/ directory.

3. Call the ClusterService.Get method, e.g., via the following gRPCurl request:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/trino/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
               "cluster_id": "<cluster_ID>"
           }' \
       trino.api.cloud.yandex.net:443 \
       yandex.cloud.trino.v1.ClusterService.Get
   

   You can request the cluster ID with the list of clusters in the folder.

4. The description of resource groups and selector rules in the Managed Service for Trino cluster is in the resource_groups field of the server response.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

To specify a resource group configuration:

1. Create a file with a description of resource groups and selector rules in JSON or YAML format. Below is an example of the resource-groups.yaml file.

   rootGroups:
     # Root resource group 1
     - name: "<resource_group_name>"
       softMemoryLimit: "<memory_usage_limit>"
       softConcurrencyLimit: <soft_limit_on_the_number_of_running_queries>
       hardConcurrencyLimit: <hard_limit_on_the_number_of_running_queries>
       softCpuLimit: <soft_CPU_time_limit>
       hardCpuLimit: <hard_CPU_time_limit>
       maxQueued: <limit_on_the_number_of_queued_requests>
       schedulingPolicy: "<query_scheduling_policy>"
       schedulingWeight: <group_weight>
       subGroups:
         # Resource sub-group 1
         - name: "<resource_group_name>"
           ...
           subGroups: [<list_of_sub-groups_of_the_following_levels>]
         # Resource sub-group 2
         - name: "<resource_group_name>"
           ...
           subGroups: [<list_of_sub-groups_of_the_following_levels>]
         ...
         # Resource sub-group N
         - name: "<resource_group_name>"
           ...
           subGroups: [<list_of_sub-groups_of_the_following_levels>]
     # Root resource group 2
     - name: "<resource_group_name_2>"
       ...
     # Root resource group N
     - name: "<resource_group_name_N>"
       ...
   selectors:
     # Selector rule 1
     - user: "<regular_expression_for_usernames>"
       userGroup: "<regular_expression_for_user_groups>"
       queryType: "<request_type>"
       source: "<regular_expression_for_query_sources>"
       clientTags: ["<list_of_tags>"]
       group: "<resource_group_name>"
     # Selector rule 2
     - ...
     ...
     # Selector rule N
     - ...
   cpuQuotaPeriod: "<CPU_quota_calculation_period>"
   

   Where:

   • rootGroups: Resource group tree description. The name, hardConcurrencyLimit, and maxQueued settings are required.

     • name: Group name.

       A group name can be a template that uses variable placeholders in ${<variable_name>} format. A template name may contain the following types of variables:

       • Built-in variables. USER for username, SOURCE for query source.
       • User-defined variables. These are declared in selector rule properties: user, userGroup, and source.

       When routing queries, Managed Service for Trino will create group instances with dynamically generated names.

     • softMemoryLimit: Maximum amount of memory available to the group. Once this limit is reached, new queries will be queued. You can set softMemoryLimit in one of the following ways:

       • As a percentage of the total available memory, e.g., 10%.
       • As an absolute memory value, e.g., 1GB. You can use one of these suffixes: B, kB, MB, GB, TB, or PB.

     • softConcurrencyLimit: Soft limit on the number of running queries. Once the limit is reached, the group will run new queries only if sibling groups are unable to accept queries or are above their soft limits. Otherwise, new queries will be queued.

     • hardConcurrencyLimit: Hard limit on the number of running queries. Once this limit is reached, new queries will be queued.

     • softCpuLimit: Soft limit on CPU time. Once this limit is reached, hardConcurrencyLimit will be reduced until the end of the current CPU quota calculation period. softCpuLimit cannot be greater than hardCpuLimit.

     • hardCpuLimit: Hard limit on CPU time. Once this limit is reached, new queries will be queued.

     Note

     If at least one of the two (softCpuLimit or hardCpuLimit) is specified, make sure to specify cpuQuotaPeriod as well.

     • maxQueued: Maximum number of queries per queue. Once this limit is reached, new queries will be rejected with an error.

     • schedulingPolicy: Queue scheduling policy. The possible values are:

       • FAIR
       • WEIGHTED
       • WEIGHTED_FAIR
       • QUERY_PRIORITY

       If not specified, the FAIR policy will apply.

     • schedulingWeight: Group weight used in the WEIGHTED and WEIGHTED_FAIR scheduling policies.

       If not specified, the group weight is 1.

     • subGroups: Descriptions of sub-groups.

   • selectors: Selector rule description. The group property is required.

     • user: Name of the user who submitted the query or a Java regular expression. The rule applies if the user name matches the regular expression. If not specified, the rule applies to queries from all users.

     • userGroup: Name of the user group that submitted the query or a Java regular expression for user groups. The rule applies if the user group name matches the regular expression. If not specified, the rule applies to queries from users of any groups.

     • queryType: Type of the request. The possible values are:

       • SELECT: SELECT queries.
       • EXPLAIN: EXPLAIN queries, except EXPLAIN ANALYZE.
       • DESCRIBE: DESCRIBE, DESCRIBE INPUT, and DESCRIBE OUTPUT queries, as well as SHOW queries, e.g., SHOW CATALOGS or SHOW SCHEMAS.
       • INSERT: INSERT, CREATE TABLE AS, and REFRESH MATERIALIZED VIEW queries.
       • MERGE: MERGE queries.
       • DELETE: DELETE queries.
       • ANALYZE: ANALYZE queries.
       • DATA_DEFINITION: CREATE, ALTER, and DROP queries for schemas, tables, views, and materialized views, as well as queries that manage prepared statements, access permissions, sessions, and transactions.
       • ALTER_TABLE_EXECUTE: ALTER TABLE ... EXECUTE queries.

     • source: Query source name or a Java regular expression. The rule applies if the query source name matches the regular expression. If not specified, the rule applies to queries from all sources.

     • clientTags: Tag list. The rule applies only if the query contains all the specified tags. If not specified, the rule applies to queries with any tags.

     • group: Full name of the target resource group. It includes the names of all its parent groups, separated by dots, starting from the root group. A group name can use variable placeholders in ${<variable_name>} format. A target group cannot have sub-groups.

     Warning

     A query that does not satisfy any of the specified rules is rejected with an error. To avoid this, add to the end of the description a rule without any restrictive conditions, containing only the group property.

   • cpuQuotaPeriod: CPU quota calculation period for resource groups. It takes an integer value in seconds, minutes, or hours, e.g., 140s, 30m, or 1h.

   For more information about the group tree and selector rules, see Resource groups in Managed Service for Trino.

2. View the description of the CLI command for creating a cluster:

   yc managed-trino cluster create --help
   

3. Run this command:

   yc managed-trino cluster create \
     ...
     --resource-groups-from-file resource-groups.yaml
   

   For available cluster parameters and their descriptions, see this guide.

1. Create a Terraform configuration file describing your infrastructure.

2. Add the resource_groups section to the cluster description:

   resource_groups = jsonencode(
     {
       "rootGroups" : [
         # Root resource group 1
         {
           "name" : "<resource_group_name>",
           "softMemoryLimit" : "<memory_usage_limit>",
           "softConcurrencyLimit" : <soft_limit_on_the_number_of_running_queries>,
           "hardConcurrencyLimit" : <hard_limit_on_the_number_of_running_queries>,
           "softCpuLimit" : <soft_CPU_time_limit>,
           "hardCpuLimit" : <hard_CPU_time_limit>,
           "maxQueued" : <limit_on_the_number_of_queued_requests>,
           "schedulingPolicy" : "<query_scheduling_policy>",
           "schedulingWeight" : <group_weight>,
           "subGroups" : [
             # Resource sub-group 1
             {
               ...
               "subGroups" : [<list_of_next-level_sub-groups>]
             },
             # Resource sub-group 2
             {
               ...
               "subGroups" : [<list_of_next-level_sub-groups>]
             },
             ...
             # Resource sub-group N
             {
               ...
               "subGroups" : [<list_of_next-level_sub-groups>]
             }
           ]
         },
         # Root resource group 2
         {
           ... 
         },
         ...
         # Root resource group N
         {
           ... 
         }
       ],
       "selectors" : [
         # Selector rule 1
         {
           "user" : "<regular_expression_for_usernames>",
           "userGroup" : "<regular_expression_for_user_groups>",
           "queryType" : "<request_type>",
           "source" : "<regular_expression_for_query_sources>",
           "clientTags" : ["<list_of_tags>"],
           "group" : "<resource_group_name>"
         },
         # Selector rule 2
         {
           ... 
         },
         ...
         # Selector rule N
         {
           ... 
         }
       ],
       "cpuQuotaPeriod" : "<CPU_quota_calculation_period>"
     }
   )
   

   Where:

   • rootGroups: Resource group tree description. The name, hardConcurrencyLimit, and maxQueued settings are required.

     • name: Group name.

       A group name can be a template that uses variable placeholders in ${<variable_name>} format. A template name may contain the following types of variables:

       • Built-in variables. USER for username, SOURCE for query source.
       • User-defined variables. These are declared in selector rule properties: user, userGroup, and source.

       When routing queries, Managed Service for Trino will create group instances with dynamically generated names.

     • softMemoryLimit: Maximum amount of memory available to the group. Once this limit is reached, new queries will be queued. You can set softMemoryLimit in one of the following ways:

       • As a percentage of the total available memory, e.g., 10%.
       • As an absolute memory value, e.g., 1GB. You can use one of these suffixes: B, kB, MB, GB, TB, or PB.

     • softConcurrencyLimit: Soft limit on the number of running queries. Once the limit is reached, the group will run new queries only if sibling groups are unable to accept queries or are above their soft limits. Otherwise, new queries will be queued.

     • hardConcurrencyLimit: Hard limit on the number of running queries. Once this limit is reached, new queries will be queued.

     • softCpuLimit: Soft limit on CPU time. Once this limit is reached, hardConcurrencyLimit will be reduced until the end of the current CPU quota calculation period. softCpuLimit cannot be greater than hardCpuLimit.

     • hardCpuLimit: Hard limit on CPU time. Once this limit is reached, new queries will be queued.

     Note

     If at least one of the two (softCpuLimit or hardCpuLimit) is specified, make sure to specify cpuQuotaPeriod as well.

     • maxQueued: Maximum number of queries per queue. Once this limit is reached, new queries will be rejected with an error.

     • schedulingPolicy: Queue scheduling policy. The possible values are:

       • FAIR
       • WEIGHTED
       • WEIGHTED_FAIR
       • QUERY_PRIORITY

       If not specified, the FAIR policy will apply.

     • schedulingWeight: Group weight used in the WEIGHTED and WEIGHTED_FAIR scheduling policies.

       If not specified, the group weight is 1.

     • subGroups: Descriptions of sub-groups.

   • selectors: Selector rule description. The group property is required.

     • user: Name of the user who submitted the query or a Java regular expression. The rule applies if the user name matches the regular expression. If not specified, the rule applies to queries from all users.

     • userGroup: Name of the user group that submitted the query or a Java regular expression for user groups. The rule applies if the user group name matches the regular expression. If not specified, the rule applies to queries from users of any groups.

     • queryType: Type of the request. The possible values are:

       • SELECT: SELECT queries.
       • EXPLAIN: EXPLAIN queries, except EXPLAIN ANALYZE.
       • DESCRIBE: DESCRIBE, DESCRIBE INPUT, and DESCRIBE OUTPUT queries, as well as SHOW queries, e.g., SHOW CATALOGS or SHOW SCHEMAS.
       • INSERT: INSERT, CREATE TABLE AS, and REFRESH MATERIALIZED VIEW queries.
       • MERGE: MERGE queries.
       • DELETE: DELETE queries.
       • ANALYZE: ANALYZE queries.
       • DATA_DEFINITION: CREATE, ALTER, and DROP queries for schemas, tables, views, and materialized views, as well as queries that manage prepared statements, access permissions, sessions, and transactions.
       • ALTER_TABLE_EXECUTE: ALTER TABLE ... EXECUTE queries.

     • source: Query source name or a Java regular expression. The rule applies if the query source name matches the regular expression. If not specified, the rule applies to queries from all sources.

     • clientTags: Tag list. The rule applies only if the query contains all the specified tags. If not specified, the rule applies to queries with any tags.

     • group: Full name of the target resource group. It includes the names of all its parent groups, separated by dots, starting from the root group. A group name can use variable placeholders in ${<variable_name>} format. A target group cannot have sub-groups.

     Warning

     A query that does not satisfy any of the specified rules is rejected with an error. To avoid this, add to the end of the description a rule without any restrictive conditions, containing only the group property.

   • cpuQuotaPeriod: CPU quota calculation period for resource groups. It takes an integer value in seconds, minutes, or hours, e.g., 140s, 30m, or 1h.

   For more information about the group tree and selector rules, see Resource groups in Managed Service for Trino.

   Note

   You can also create a separate JSON file describing resource groups and selector rules and provide it in the resource_groups parameter as follows:

   resource_groups = file("resource-groups.json")
   

3. Make sure the settings are correct.

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

4. Confirm updating the resources.

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

For more information, see this Terraform provider guide.

1. Get an IAM token for API authentication and put it into an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Create a file named body.json and paste the following code into it:

   {
     <cluster_parameters>
     ...
     "trino": {
       ...
       "resourceManagement": {
         "resourceGroups": {
           "rootGroups": [
             {
               "name" : "<resource_group_name>",
               "softMemoryLimit" : "<memory_usage_limit>",
               "softConcurrencyLimit" : "<soft_limit_on_the_number_of_running_queries>",
               "hardConcurrencyLimit" : "<hard_limit_on_the_number_of_running_queries>",
               "softCpuLimit" : "<soft_CPU_time_limit>",
               "hardCpuLimit" : "<hard_CPU_time_limit>",
               "maxQueued" : "<limit_on_the_number_of_queued_requests>",
               "schedulingPolicy" : "<query_scheduling_policy>",
               "schedulingWeight" : "<group_weight>",
               "subGroups": [
                 {
                   <resource_sub-group_1>
                   ...
                   "subGroups" : [<list_of_next-level_sub-groups>]
                 },
                 {
                   <resource_sub-group_2>
                   ...
                   "subGroups" : [<list_of_next-level_sub-groups>]
                 },
                 ...
                 {
                   <resource_sub-group_N>
                   ...
                   "subGroups" : [<list_of_next-level_sub-groups>]
                 }
               ]
             },
             {
               <root_resource_group_2>
             },
             ...
             {
               <root_resource_group_N>
             }
           ],
           "selectors": [
             {
               "user": "<regular_expression_for_usernames>",
               "userGroup": "<regular_expression_for_user_groups>",
               "queryType": "<request_type>",
               "source": "<regular_expression_for_query_sources>",
               "clientTags": ["<list_of_tags>"],
               "group": "<resource_group_name>"
             },
             {
               <selector_rule_2> 
             },
             ...
             {
               <selector_rule_N>
             }
           ],
           "cpuQuotaPeriod": "<CPU_quota_calculation_period>"
         }
         ...
       },
     }
   }
   

   Where:

   • rootGroups: Resource group tree description. The name, hardConcurrencyLimit, and maxQueued settings are required.

     • name: Group name.

       A group name can be a template that uses variable placeholders in ${<variable_name>} format. A template name may contain the following types of variables:

       • Built-in variables. USER for username, SOURCE for query source.
       • User-defined variables. These are declared in selector rule properties: user, userGroup, and source.

       When routing queries, Managed Service for Trino will create group instances with dynamically generated names.

     • softMemoryLimit: Maximum amount of memory available to the group. Once this limit is reached, new queries will be queued. You can set softMemoryLimit in one of the following ways:

       • As a percentage of the total available memory, e.g., 10%.
       • As an absolute memory value, e.g., 1GB. You can use one of these suffixes: B, kB, MB, GB, TB, or PB.

     • softConcurrencyLimit: Soft limit on the number of running queries. Once the limit is reached, the group will run new queries only if sibling groups are unable to accept queries or are above their soft limits. Otherwise, new queries will be queued.

     • hardConcurrencyLimit: Hard limit on the number of running queries. Once this limit is reached, new queries will be queued.

     • softCpuLimit: Soft limit on CPU time. Once this limit is reached, hardConcurrencyLimit will be reduced until the end of the current CPU quota calculation period. softCpuLimit cannot be greater than hardCpuLimit.

     • hardCpuLimit: Hard limit on CPU time. Once this limit is reached, new queries will be queued.

     Note

     If at least one of the two (softCpuLimit or hardCpuLimit) is specified, make sure to specify cpuQuotaPeriod as well.

     • maxQueued: Maximum number of queries per queue. Once this limit is reached, new queries will be rejected with an error.

     • schedulingPolicy: Queue scheduling policy. The possible values are:

       • FAIR
       • WEIGHTED
       • WEIGHTED_FAIR
       • QUERY_PRIORITY

       If not specified, the FAIR policy will apply.

     • schedulingWeight: Group weight used in the WEIGHTED and WEIGHTED_FAIR scheduling policies.

       If not specified, the group weight is 1.

     • subGroups: Descriptions of sub-groups.

   • selectors: Selector rule description. The group property is required.

     • user: Name of the user who submitted the query or a Java regular expression. The rule applies if the user name matches the regular expression. If not specified, the rule applies to queries from all users.

     • userGroup: Name of the user group that submitted the query or a Java regular expression for user groups. The rule applies if the user group name matches the regular expression. If not specified, the rule applies to queries from users of any groups.

     • queryType: Type of the request. The possible values are:

       • SELECT: SELECT queries.
       • EXPLAIN: EXPLAIN queries, except EXPLAIN ANALYZE.
       • DESCRIBE: DESCRIBE, DESCRIBE INPUT, and DESCRIBE OUTPUT queries, as well as SHOW queries, e.g., SHOW CATALOGS or SHOW SCHEMAS.
       • INSERT: INSERT, CREATE TABLE AS, and REFRESH MATERIALIZED VIEW queries.
       • MERGE: MERGE queries.
       • DELETE: DELETE queries.
       • ANALYZE: ANALYZE queries.
       • DATA_DEFINITION: CREATE, ALTER, and DROP queries for schemas, tables, views, and materialized views, as well as queries that manage prepared statements, access permissions, sessions, and transactions.
       • ALTER_TABLE_EXECUTE: ALTER TABLE ... EXECUTE queries.

     • source: Query source name or a Java regular expression. The rule applies if the query source name matches the regular expression. If not specified, the rule applies to queries from all sources.

     • clientTags: Tag list. The rule applies only if the query contains all the specified tags. If not specified, the rule applies to queries with any tags.

     • group: Full name of the target resource group. It includes the names of all its parent groups, separated by dots, starting from the root group. A group name can use variable placeholders in ${<variable_name>} format. A target group cannot have sub-groups.

     Warning

     A query that does not satisfy any of the specified rules is rejected with an error. To avoid this, add to the end of the description a rule without any restrictive conditions, containing only the group property.

   • cpuQuotaPeriod: CPU quota calculation period for resource groups. It takes an integer value in seconds, minutes, or hours, e.g., 140s, 30m, or 1h.

   For more information about the group tree and selector rules, see Resource groups in Managed Service for Trino.

3. Call the Cluster.Create method, e.g., via the following cURL request:

   curl \
       --request POST \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --url 'https://trino.api.cloud.yandex.net/managed-trino/v1/clusters'
       --data '@body.json'
   

4. View the server response to make sure your request was successful.

1. Get an IAM token for API authentication and place it in an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume that the repository contents reside in the ~/cloudapi/ directory.

3. Create a file named body.json and paste the following code into it:

   {
     <cluster_parameters>
     ...
     "trino": {
       ...
       "resource_management": {
         "resource_groups": {
           "root_groups": [
             {
               "name" : "<resource_group_name>",
               "soft_memory_limit" : "<memory_usage_limit>",
               "soft_concurrency_limit" : "<soft_limit_on_the_number_of_running_queries>",
               "hard_concurrency_limit" : "<hard_limit_on_the_number_of_running_queries>",
               "soft_cpu_limit" : "<soft_CPU_time_limit>",
               "hard_cpu_limit" : "<hard_CPU_time_limit>",
               "max_queued" : "<limit_on_the_number_of_queued_requests>",
               "scheduling_policy" : "<query_scheduling_policy>",
               "scheduling_weight" : "<group_weight>",
               "sub_groups": [
                 {
                   <resource_sub-group_1>
                   ...
                   "sub_groups" : [<list_of_next-level_sub-groups>]
                 },
                 {
                   <resource_sub-group_2>
                   ...
                   "sub_groups" : [<list_of_next-level_sub-groups>]
                 },
                 ...
                 {
                   <resource_sub-group_N>
                   ...
                   "sub_groups" : [<list_of_next-level_sub-groups>]
                 }
               ]
             },
             {
               <root_resource_group_2>
             },
             ...
             {
               <root_resource_group_N>
             }
           ],
           "selectors": [
             {
               "user": "<regular_expression_for_usernames>",
               "user_group": "<regular_expression_for_user_groups>",
               "query_type": "<request_type>",
               "source": "<regular_expression_for_query_sources>",
               "client_tags": ["<list_of_tags>"],
               "group": "<resource_group_name>"
             },
             {
               <selector_rule_2> 
             },
             ...
             {
               <selector_rule_N>
             }
           ],
           "cpu_quota_period": "<CPU_quota_calculation_period>"
         }
         ...
       },
     }
   }
   

   Where:

   • root_groups: Resource group tree description. The name, hard_concurrency_limit, and max_queued settings are required.

     • name: Group name.

       A group name can be a template that uses variable placeholders in ${<variable_name>} format. A template name may contain the following types of variables:

       • Built-in variables. USER for username, SOURCE for query source.
       • User-defined variables. These are declared in selector rule properties: user, user_group, and source.

       When routing queries, Managed Service for Trino will create group instances with dynamically generated names.

     • soft_memory_limit: Maximum amount of memory available to the group. Once this limit is reached, new queries will be queued. You can set soft_memory_limit in one of the following ways:

       • As a percentage of the total available memory, e.g., 10%.
       • As an absolute memory value, e.g., 1GB. You can use one of these suffixes: B, kB, MB, GB, TB, or PB.

     • soft_concurrency_limit: Soft limit on the number of running queries. Once the limit is reached, the group will run new queries only if sibling groups are unable to accept queries or are above their soft limits. Otherwise, new queries will be queued.

     • hard_concurrency_limit: Hard limit on the number of running queries. Once this limit is reached, new queries will be queued.

     • soft_cpu_limit: Soft limit on CPU time. Once this limit is reached, hard_concurrency_limit will be reduced until the end of the current CPU quota calculation period. soft_cpu_limit cannot be greater than hard_cpu_limit.

     • hard_cpu_limit: Hard limit on CPU time. Once this limit is reached, new queries will be queued.

     Note

     If at least one of the two (soft_cpu_limit or hard_cpu_limit) is specified, make sure to specify cpu_quota_period as well.

     • max_queued: Maximum number of queries per queue. Once this limit is reached, new queries will be rejected with an error.

     • scheduling_policy: Queue scheduling policy. The possible values are:

       • FAIR
       • WEIGHTED
       • WEIGHTED_FAIR
       • QUERY_PRIORITY

       If not specified, the FAIR policy will apply.

     • scheduling_weight: Group weight used in the WEIGHTED and WEIGHTED_FAIR scheduling policies.

       If not specified, the group weight is 1.

     • sub_groups: Descriptions of sub-groups.

   • selectors: Selector rule description. The group property is required.

     • user: Name of the user who submitted the query or a Java regular expression. The rule applies if the user name matches the regular expression. If not specified, the rule applies to queries from all users.

     • user_group: Name of the user group that submitted the query or a Java regular expression for user groups. The rule applies if the user group name matches the regular expression. If not specified, the rule applies to queries from users of any groups.

     • query_type: Type of the request. The possible values are:

       • SELECT: SELECT queries.
       • EXPLAIN: EXPLAIN queries, except EXPLAIN ANALYZE.
       • DESCRIBE: DESCRIBE, DESCRIBE INPUT, and DESCRIBE OUTPUT queries, as well as SHOW queries, e.g., SHOW CATALOGS or SHOW SCHEMAS.
       • INSERT: INSERT, CREATE TABLE AS, and REFRESH MATERIALIZED VIEW queries.
       • MERGE: MERGE queries.
       • DELETE: DELETE queries.
       • ANALYZE: ANALYZE queries.
       • DATA_DEFINITION: CREATE, ALTER, and DROP queries for schemas, tables, views, and materialized views, as well as queries that manage prepared statements, access permissions, sessions, and transactions.
       • ALTER_TABLE_EXECUTE: ALTER TABLE ... EXECUTE queries.

     • source: Query source name or a Java regular expression. The rule applies if the query source name matches the regular expression. If not specified, the rule applies to queries from all sources.

     • client_tags: Tag list. The rule applies only if the query contains all the specified tags. If not specified, the rule applies to queries with any tags.

     • group: Full name of the target resource group. It includes the names of all its parent groups, separated by dots, starting from the root group. A group name can use variable placeholders in ${<variable_name>} format. A target group cannot have sub-groups.

     Warning

     A query that does not satisfy any of the specified rules is rejected with an error. To avoid this, add to the end of the description a rule without any restrictive conditions, containing only the group property.

   • cpu_quota_period: CPU quota calculation period for resource groups. It takes an integer value in seconds, minutes, or hours, e.g., 140s, 30m, or 1h.

   For more information about the group tree and selector rules, see Resource groups in Managed Service for Trino.

   For available cluster parameters and their descriptions, see this guide.

4. Call the ClusterService.Create method, e.g., via the following gRPCurl request:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/trino/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d @ \
       trino.api.cloud.yandex.net:443 \
       yandex.cloud.trino.v1.ClusterService.Create \
       < body.json
   

5. Check the server response to make sure your request was successful.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

To specify a resource group configuration:

1. If no description of resource groups and selector rules in the Managed Service for Trino cluster has been given yet, create a description file in JSON or YAML format. Below is an example of the resource-groups.yaml file.

   rootGroups:
     # Root resource group 1
     - name: "<resource_group_name>"
       softMemoryLimit: "<memory_usage_limit>"
       softConcurrencyLimit: <soft_limit_on_the_number_of_running_queries>
       hardConcurrencyLimit: <hard_limit_on_the_number_of_running_queries>
       softCpuLimit: <soft_CPU_time_limit>
       hardCpuLimit: <hard_CPU_time_limit>
       maxQueued: <limit_on_the_number_of_queued_requests>
       schedulingPolicy: "<query_scheduling_policy>"
       schedulingWeight: <group_weight>
       subGroups:
         # Resource sub-group 1
         - name: "<resource_group_name>"
           ...
           subGroups: [<list_of_sub-groups_of_the_following_levels>]
         # Resource sub-group 2
         - name: "<resource_group_name>"
           ...
           subGroups: [<list_of_sub-groups_of_the_following_levels>]
         ...
         # Resource sub-group N
         - name: "<resource_group_name>"
           ...
           subGroups: [<list_of_sub-groups_of_the_following_levels>]
     # Root resource group 2
     - name: "<resource_group_name_2>"
       ...
     # Root resource group N
     - name: "<resource_group_name_N>"
       ...
   selectors:
     # Selector rule 1
     - user: "<regular_expression_for_usernames>"
       userGroup: "<regular_expression_for_user_groups>"
       queryType: "<request_type>"
       source: "<regular_expression_for_query_sources>"
       clientTags: ["<list_of_tags>"]
       group: "<resource_group_name>"
     # Selector rule 2
     - ...
     ...
     # Selector rule N
     - ...
   cpuQuotaPeriod: "<CPU_quota_calculation_period>"
   

   Where:

   • rootGroups: Resource group tree description. The name, hardConcurrencyLimit, and maxQueued settings are required.

     • name: Group name.

       A group name can be a template that uses variable placeholders in ${<variable_name>} format. A template name may contain the following types of variables:

       • Built-in variables. USER for username, SOURCE for query source.
       • User-defined variables. These are declared in selector rule properties: user, userGroup, and source.

       When routing queries, Managed Service for Trino will create group instances with dynamically generated names.

     • softMemoryLimit: Maximum amount of memory available to the group. Once this limit is reached, new queries will be queued. You can set softMemoryLimit in one of the following ways:

       • As a percentage of the total available memory, e.g., 10%.
       • As an absolute memory value, e.g., 1GB. You can use one of these suffixes: B, kB, MB, GB, TB, or PB.

     • softConcurrencyLimit: Soft limit on the number of running queries. Once the limit is reached, the group will run new queries only if sibling groups are unable to accept queries or are above their soft limits. Otherwise, new queries will be queued.

     • hardConcurrencyLimit: Hard limit on the number of running queries. Once this limit is reached, new queries will be queued.

     • softCpuLimit: Soft limit on CPU time. Once this limit is reached, hardConcurrencyLimit will be reduced until the end of the current CPU quota calculation period. softCpuLimit cannot be greater than hardCpuLimit.

     • hardCpuLimit: Hard limit on CPU time. Once this limit is reached, new queries will be queued.

     Note

     If at least one of the two (softCpuLimit or hardCpuLimit) is specified, make sure to specify cpuQuotaPeriod as well.

     • maxQueued: Maximum number of queries per queue. Once this limit is reached, new queries will be rejected with an error.

     • schedulingPolicy: Queue scheduling policy. The possible values are:

       • FAIR
       • WEIGHTED
       • WEIGHTED_FAIR
       • QUERY_PRIORITY

       If not specified, the FAIR policy will apply.

     • schedulingWeight: Group weight used in the WEIGHTED and WEIGHTED_FAIR scheduling policies.

       If not specified, the group weight is 1.

     • subGroups: Descriptions of sub-groups.

   • selectors: Selector rule description. The group property is required.

     • user: Name of the user who submitted the query or a Java regular expression. The rule applies if the user name matches the regular expression. If not specified, the rule applies to queries from all users.

     • userGroup: Name of the user group that submitted the query or a Java regular expression for user groups. The rule applies if the user group name matches the regular expression. If not specified, the rule applies to queries from users of any groups.

     • queryType: Type of the request. The possible values are:

       • SELECT: SELECT queries.
       • EXPLAIN: EXPLAIN queries, except EXPLAIN ANALYZE.
       • DESCRIBE: DESCRIBE, DESCRIBE INPUT, and DESCRIBE OUTPUT queries, as well as SHOW queries, e.g., SHOW CATALOGS or SHOW SCHEMAS.
       • INSERT: INSERT, CREATE TABLE AS, and REFRESH MATERIALIZED VIEW queries.
       • MERGE: MERGE queries.
       • DELETE: DELETE queries.
       • ANALYZE: ANALYZE queries.
       • DATA_DEFINITION: CREATE, ALTER, and DROP queries for schemas, tables, views, and materialized views, as well as queries that manage prepared statements, access permissions, sessions, and transactions.
       • ALTER_TABLE_EXECUTE: ALTER TABLE ... EXECUTE queries.

     • source: Query source name or a Java regular expression. The rule applies if the query source name matches the regular expression. If not specified, the rule applies to queries from all sources.

     • clientTags: Tag list. The rule applies only if the query contains all the specified tags. If not specified, the rule applies to queries with any tags.

     • group: Full name of the target resource group. It includes the names of all its parent groups, separated by dots, starting from the root group. A group name can use variable placeholders in ${<variable_name>} format. A target group cannot have sub-groups.

     Warning

     A query that does not satisfy any of the specified rules is rejected with an error. To avoid this, add to the end of the description a rule without any restrictive conditions, containing only the group property.

   • cpuQuotaPeriod: CPU quota calculation period for resource groups. It takes an integer value in seconds, minutes, or hours, e.g., 140s, 30m, or 1h.

   For more information about the group tree and selector rules, see Resource groups in Managed Service for Trino.

2. If a description of resource groups and selector rules has already been given, open resource-groups.yaml and edit it as needed. You can:

   • Add new resource groups and selector rules.
   • Update the parameters of existing groups and rules.
   • Delete the groups and rules you no longer need.

3. Run this command:

yc managed-trino cluster set-resource-groups <cluster_name_or_ID> \
   --from-file resource-groups.yaml

You can get the cluster name and ID with the list of clusters in the folder.

1. Open the current Terraform configuration file describing your infrastructure.

   For more on how to create this file, see Creating a cluster.

2. If no description of resource groups and selector rules in the Managed Service for Trino cluster has been given yet, add the resource_groups section to the cluster description:

   resource_groups = jsonencode(
     {
       "rootGroups" : [
         # Root resource group 1
         {
           "name" : "<resource_group_name>",
           "softMemoryLimit" : "<memory_usage_limit>",
           "softConcurrencyLimit" : <soft_limit_on_the_number_of_running_queries>,
           "hardConcurrencyLimit" : <hard_limit_on_the_number_of_running_queries>,
           "softCpuLimit" : <soft_CPU_time_limit>,
           "hardCpuLimit" : <hard_CPU_time_limit>,
           "maxQueued" : <limit_on_the_number_of_queued_requests>,
           "schedulingPolicy" : "<query_scheduling_policy>",
           "schedulingWeight" : <group_weight>,
           "subGroups" : [
             # Resource sub-group 1
             {
               ...
               "subGroups" : [<list_of_next-level_sub-groups>]
             },
             # Resource sub-group 2
             {
               ...
               "subGroups" : [<list_of_next-level_sub-groups>]
             },
             ...
             # Resource sub-group N
             {
               ...
               "subGroups" : [<list_of_next-level_sub-groups>]
             }
           ]
         },
         # Root resource group 2
         {
           ... 
         },
         ...
         # Root resource group N
         {
           ... 
         }
       ],
       "selectors" : [
         # Selector rule 1
         {
           "user" : "<regular_expression_for_usernames>",
           "userGroup" : "<regular_expression_for_user_groups>",
           "queryType" : "<request_type>",
           "source" : "<regular_expression_for_query_sources>",
           "clientTags" : ["<list_of_tags>"],
           "group" : "<resource_group_name>"
         },
         # Selector rule 2
         {
           ... 
         },
         ...
         # Selector rule N
         {
           ... 
         }
       ],
       "cpuQuotaPeriod" : "<CPU_quota_calculation_period>"
     }
   )
   

   Where:

   • rootGroups: Resource group tree description. The name, hardConcurrencyLimit, and maxQueued settings are required.

     • name: Group name.

       A group name can be a template that uses variable placeholders in ${<variable_name>} format. A template name may contain the following types of variables:

       • Built-in variables. USER for username, SOURCE for query source.
       • User-defined variables. These are declared in selector rule properties: user, userGroup, and source.

       When routing queries, Managed Service for Trino will create group instances with dynamically generated names.

     • softMemoryLimit: Maximum amount of memory available to the group. Once this limit is reached, new queries will be queued. You can set softMemoryLimit in one of the following ways:

       • As a percentage of the total available memory, e.g., 10%.
       • As an absolute memory value, e.g., 1GB. You can use one of these suffixes: B, kB, MB, GB, TB, or PB.

     • softConcurrencyLimit: Soft limit on the number of running queries. Once the limit is reached, the group will run new queries only if sibling groups are unable to accept queries or are above their soft limits. Otherwise, new queries will be queued.

     • hardConcurrencyLimit: Hard limit on the number of running queries. Once this limit is reached, new queries will be queued.

     • softCpuLimit: Soft limit on CPU time. Once this limit is reached, hardConcurrencyLimit will be reduced until the end of the current CPU quota calculation period. softCpuLimit cannot be greater than hardCpuLimit.

     • hardCpuLimit: Hard limit on CPU time. Once this limit is reached, new queries will be queued.

     Note

     If at least one of the two (softCpuLimit or hardCpuLimit) is specified, make sure to specify cpuQuotaPeriod as well.

     • maxQueued: Maximum number of queries per queue. Once this limit is reached, new queries will be rejected with an error.

     • schedulingPolicy: Queue scheduling policy. The possible values are:

       • FAIR
       • WEIGHTED
       • WEIGHTED_FAIR
       • QUERY_PRIORITY

       If not specified, the FAIR policy will apply.

     • schedulingWeight: Group weight used in the WEIGHTED and WEIGHTED_FAIR scheduling policies.

       If not specified, the group weight is 1.

     • subGroups: Descriptions of sub-groups.

   • selectors: Selector rule description. The group property is required.

     • user: Name of the user who submitted the query or a Java regular expression. The rule applies if the user name matches the regular expression. If not specified, the rule applies to queries from all users.

     • userGroup: Name of the user group that submitted the query or a Java regular expression for user groups. The rule applies if the user group name matches the regular expression. If not specified, the rule applies to queries from users of any groups.

     • queryType: Type of the request. The possible values are:

       • SELECT: SELECT queries.
       • EXPLAIN: EXPLAIN queries, except EXPLAIN ANALYZE.
       • DESCRIBE: DESCRIBE, DESCRIBE INPUT, and DESCRIBE OUTPUT queries, as well as SHOW queries, e.g., SHOW CATALOGS or SHOW SCHEMAS.
       • INSERT: INSERT, CREATE TABLE AS, and REFRESH MATERIALIZED VIEW queries.
       • MERGE: MERGE queries.
       • DELETE: DELETE queries.
       • ANALYZE: ANALYZE queries.
       • DATA_DEFINITION: CREATE, ALTER, and DROP queries for schemas, tables, views, and materialized views, as well as queries that manage prepared statements, access permissions, sessions, and transactions.
       • ALTER_TABLE_EXECUTE: ALTER TABLE ... EXECUTE queries.

     • source: Query source name or a Java regular expression. The rule applies if the query source name matches the regular expression. If not specified, the rule applies to queries from all sources.

     • clientTags: Tag list. The rule applies only if the query contains all the specified tags. If not specified, the rule applies to queries with any tags.

     • group: Full name of the target resource group. It includes the names of all its parent groups, separated by dots, starting from the root group. A group name can use variable placeholders in ${<variable_name>} format. A target group cannot have sub-groups.

     Warning

     A query that does not satisfy any of the specified rules is rejected with an error. To avoid this, add to the end of the description a rule without any restrictive conditions, containing only the group property.

   • cpuQuotaPeriod: CPU quota calculation period for resource groups. It takes an integer value in seconds, minutes, or hours, e.g., 140s, 30m, or 1h.

   For more information about the group tree and selector rules, see Resource groups in Managed Service for Trino.

   Note

   You can also create a separate JSON file describing resource groups and selector rules and provide it in the resource_groups parameter as follows:

   resource_groups = file("resource-groups.json")
   

3. If a description of resource groups and selector rules has already been given, edit the resource_groups section or JSON file as needed. You can:

   • Add new resource groups and selector rules.
   • Update the parameters of existing groups and rules.
   • Delete the groups and rules you no longer need.

4. Make sure the settings are correct.

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

5. Confirm updating the resources.

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

For more information, see this Terraform provider guide.

1. Get an IAM token for API authentication and put it into an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Create a file named body.json and paste the following code into it:

   {
     "updateMask": "<list_of_settings_to_update>",
     "trino": {
       "resourceManagement": {
         "resourceGroups": {
           "rootGroups": [
             {
               "name" : "<resource_group_name>",
               "softMemoryLimit" : "<memory_usage_limit>",
               "softConcurrencyLimit" : "<soft_limit_on_the_number_of_running_queries>",
               "hardConcurrencyLimit" : "<hard_limit_on_the_number_of_running_queries>",
               "softCpuLimit" : "<soft_CPU_time_limit>",
               "hardCpuLimit" : "<hard_CPU_time_limit>",
               "maxQueued" : "<limit_on_the_number_of_queued_requests>",
               "schedulingPolicy" : "<query_scheduling_policy>",
               "schedulingWeight" : "<group_weight>",
               "subGroups": [
                 {
                   <resource_sub-group_1>
                   ...
                   "subGroups" : [<list_of_next-level_sub-groups>]
                 },
                 {
                   <resource_sub-group_2>
                   ...
                   "subGroups" : [<list_of_next-level_sub-groups>]
                 },
                 ...
                 {
                   <resource_sub-group_N>
                   ...
                   "subGroups" : [<list_of_next-level_sub-groups>]
                 }
               ]
             },
             {
               <root_resource_group_2>
             },
             ...
             {
               <root_resource_group_N>
             }
           ],
           "selectors": [
             {
               "user": "<regular_expression_for_usernames>",
               "userGroup": "<regular_expression_for_user_groups>",
               "queryType": "<request_type>",
               "source": "<regular_expression_for_query_sources>",
               "clientTags": ["<list_of_tags>"],
               "group": "<resource_group_name>"
             },
             {
               <selector_rule_2> 
             },
             ...
             {
               <selector_rule_N>
             }
           ],
           "cpuQuotaPeriod": "<CPU_quota_calculation_period>"
         }
         ...
       },
     }
   }
   

   Where:

   • updateMask: Comma-separated list of parameters to update.

     Warning

     When you update a cluster, all parameters of the object you are modifying will be reset to their defaults unless explicitly provided in the request. To avoid this, list the settings you want to change in the updateMask parameter.

   • rootGroups: Resource group tree description. The name, hardConcurrencyLimit, and maxQueued settings are required.

     • name: Group name.

       A group name can be a template that uses variable placeholders in ${<variable_name>} format. A template name may contain the following types of variables:

       • Built-in variables. USER for username, SOURCE for query source.
       • User-defined variables. These are declared in selector rule properties: user, userGroup, and source.

       When routing queries, Managed Service for Trino will create group instances with dynamically generated names.

     • softMemoryLimit: Maximum amount of memory available to the group. Once this limit is reached, new queries will be queued. You can set softMemoryLimit in one of the following ways:

       • As a percentage of the total available memory, e.g., 10%.
       • As an absolute memory value, e.g., 1GB. You can use one of these suffixes: B, kB, MB, GB, TB, or PB.

     • softConcurrencyLimit: Soft limit on the number of running queries. Once the limit is reached, the group will run new queries only if sibling groups are unable to accept queries or are above their soft limits. Otherwise, new queries will be queued.

     • hardConcurrencyLimit: Hard limit on the number of running queries. Once this limit is reached, new queries will be queued.

     • softCpuLimit: Soft limit on CPU time. Once this limit is reached, hardConcurrencyLimit will be reduced until the end of the current CPU quota calculation period. softCpuLimit cannot be greater than hardCpuLimit.

     • hardCpuLimit: Hard limit on CPU time. Once this limit is reached, new queries will be queued.

     Note

     If at least one of the two (softCpuLimit or hardCpuLimit) is specified, make sure to specify cpuQuotaPeriod as well.

     • maxQueued: Maximum number of queries per queue. Once this limit is reached, new queries will be rejected with an error.

     • schedulingPolicy: Queue scheduling policy. The possible values are:

       • FAIR
       • WEIGHTED
       • WEIGHTED_FAIR
       • QUERY_PRIORITY

       If not specified, the FAIR policy will apply.

     • schedulingWeight: Group weight used in the WEIGHTED and WEIGHTED_FAIR scheduling policies.

       If not specified, the group weight is 1.

     • subGroups: Descriptions of sub-groups.

   • selectors: Selector rule description. The group property is required.

     • user: Name of the user who submitted the query or a Java regular expression. The rule applies if the user name matches the regular expression. If not specified, the rule applies to queries from all users.

     • userGroup: Name of the user group that submitted the query or a Java regular expression for user groups. The rule applies if the user group name matches the regular expression. If not specified, the rule applies to queries from users of any groups.

     • queryType: Type of the request. The possible values are:

       • SELECT: SELECT queries.
       • EXPLAIN: EXPLAIN queries, except EXPLAIN ANALYZE.
       • DESCRIBE: DESCRIBE, DESCRIBE INPUT, and DESCRIBE OUTPUT queries, as well as SHOW queries, e.g., SHOW CATALOGS or SHOW SCHEMAS.
       • INSERT: INSERT, CREATE TABLE AS, and REFRESH MATERIALIZED VIEW queries.
       • MERGE: MERGE queries.
       • DELETE: DELETE queries.
       • ANALYZE: ANALYZE queries.
       • DATA_DEFINITION: CREATE, ALTER, and DROP queries for schemas, tables, views, and materialized views, as well as queries that manage prepared statements, access permissions, sessions, and transactions.
       • ALTER_TABLE_EXECUTE: ALTER TABLE ... EXECUTE queries.

     • source: Query source name or a Java regular expression. The rule applies if the query source name matches the regular expression. If not specified, the rule applies to queries from all sources.

     • clientTags: Tag list. The rule applies only if the query contains all the specified tags. If not specified, the rule applies to queries with any tags.

     • group: Full name of the target resource group. It includes the names of all its parent groups, separated by dots, starting from the root group. A group name can use variable placeholders in ${<variable_name>} format. A target group cannot have sub-groups.

     Warning

     A query that does not satisfy any of the specified rules is rejected with an error. To avoid this, add to the end of the description a rule without any restrictive conditions, containing only the group property.

   • cpuQuotaPeriod: CPU quota calculation period for resource groups. It takes an integer value in seconds, minutes, or hours, e.g., 140s, 30m, or 1h.

   For more information about the group tree and selector rules, see Resource groups in Managed Service for Trino.

3. Call the Cluster.Update method, e.g., via the following cURL request:

   curl \
     --request PATCH \
     --header "Authorization: Bearer $IAM_TOKEN" \
     --url 'https://trino.api.cloud.yandex.net/managed-trino/v1/clusters/<cluster_ID>'
     --data '@body.json'
   

   You can get the cluster ID with the list of clusters in the folder.

4. Check the server response to make sure your request was successful.

1. Get an IAM token for API authentication and place it in an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume that the repository contents reside in the ~/cloudapi/ directory.

3. Create a file named body.json and paste the following code into it:

   {
     "cluster_id": "<cluster_ID>",
       "update_mask": {
         "paths": [
           <list_of_settings_to_update>
         ]
       },
     "trino": {
       "resource_management": {
         "resource_groups": {
           "root_groups": [
             {
               "name" : "<resource_group_name>",
               "soft_memory_limit" : "<memory_usage_limit>",
               "soft_concurrency_limit" : "<soft_limit_on_the_number_of_running_queries>",
               "hard_concurrency_limit" : "<hard_limit_on_the_number_of_running_queries>",
               "soft_cpu_limit" : "<soft_CPU_time_limit>",
               "hard_cpu_limit" : "<hard_CPU_time_limit>",
               "max_queued" : "<limit_on_the_number_of_queued_requests>",
               "scheduling_policy" : "<query_scheduling_policy>",
               "scheduling_weight" : "<group_weight>",
               "sub_groups": [
                 {
                   <resource_sub-group_1>
                   ...
                   "sub_groups" : [<list_of_next-level_sub-groups>]
                 },
                 {
                   <resource_sub-group_2>
                   ...
                   "sub_groups" : [<list_of_next-level_sub-groups>]
                 },
                 ...
                 {
                   <resource_sub-group_N>
                   ...
                   "sub_groups" : [<list_of_next-level_sub-groups>]
                 }
               ]
             },
             {
               <root_resource_group_2>
             },
             ...
             {
               <root_resource_group_N>
             }
           ],
           "selectors": [
             {
               "user": "<regular_expression_for_usernames>",
               "user_group": "<regular_expression_for_user_groups>",
               "query_type": "<request_type>",
               "source": "<regular_expression_for_query_sources>",
               "client_tags": ["<list_of_tags>"],
               "group": "<resource_group_name>"
             },
             {
               <selector_rule_2> 
             },
             ...
             {
               <selector_rule_N>
             }
           ],
           "cpu_quota_period": "<CPU_quota_calculation_period>"
         }
         ...
       },
     }
   }
   

   Where:

   • cluster_id: Cluster ID.

     You can get the cluster ID with the list of clusters in the folder.

   • update_mask: List of parameters to update as an array of strings (paths[]).

     Format for listing settings

     "update_mask": {
       "paths": [
         "<setting_1>",
         "<setting_2>",
         ...
         "<setting_N>"
       ]
     }
     

     Warning

     When you update a cluster, all parameters of the object you are modifying will be reset to their defaults unless explicitly provided in the request. To avoid this, list the settings you want to change in the update_mask parameter.

   • root_groups: Resource group tree description. The name, hard_concurrency_limit, and max_queued settings are required.

     • name: Group name.

       A group name can be a template that uses variable placeholders in ${<variable_name>} format. A template name may contain the following types of variables:

       • Built-in variables. USER for username, SOURCE for query source.
       • User-defined variables. These are declared in selector rule properties: user, user_group, and source.

       When routing queries, Managed Service for Trino will create group instances with dynamically generated names.

     • soft_memory_limit: Maximum amount of memory available to the group. Once this limit is reached, new queries will be queued. You can set soft_memory_limit in one of the following ways:

       • As a percentage of the total available memory, e.g., 10%.
       • As an absolute memory value, e.g., 1GB. You can use one of these suffixes: B, kB, MB, GB, TB, or PB.

     • soft_concurrency_limit: Soft limit on the number of running queries. Once the limit is reached, the group will run new queries only if sibling groups are unable to accept queries or are above their soft limits. Otherwise, new queries will be queued.

     • hard_concurrency_limit: Hard limit on the number of running queries. Once this limit is reached, new queries will be queued.

     • soft_cpu_limit: Soft limit on CPU time. Once this limit is reached, hard_concurrency_limit will be reduced until the end of the current CPU quota calculation period. soft_cpu_limit cannot be greater than hard_cpu_limit.

     • hard_cpu_limit: Hard limit on CPU time. Once this limit is reached, new queries will be queued.

     Note

     If at least one of the two (soft_cpu_limit or hard_cpu_limit) is specified, make sure to specify cpu_quota_period as well.

     • max_queued: Maximum number of queries per queue. Once this limit is reached, new queries will be rejected with an error.

     • scheduling_policy: Queue scheduling policy. The possible values are:

       • FAIR
       • WEIGHTED
       • WEIGHTED_FAIR
       • QUERY_PRIORITY

       If not specified, the FAIR policy will apply.

     • scheduling_weight: Group weight used in the WEIGHTED and WEIGHTED_FAIR scheduling policies.

       If not specified, the group weight is 1.

     • sub_groups: Descriptions of sub-groups.

   • selectors: Selector rule description. The group property is required.

     • user: Name of the user who submitted the query or a Java regular expression. The rule applies if the user name matches the regular expression. If not specified, the rule applies to queries from all users.

     • user_group: Name of the user group that submitted the query or a Java regular expression for user groups. The rule applies if the user group name matches the regular expression. If not specified, the rule applies to queries from users of any groups.

     • query_type: Type of the request. The possible values are:

       • SELECT: SELECT queries.
       • EXPLAIN: EXPLAIN queries, except EXPLAIN ANALYZE.
       • DESCRIBE: DESCRIBE, DESCRIBE INPUT, and DESCRIBE OUTPUT queries, as well as SHOW queries, e.g., SHOW CATALOGS or SHOW SCHEMAS.
       • INSERT: INSERT, CREATE TABLE AS, and REFRESH MATERIALIZED VIEW queries.
       • MERGE: MERGE queries.
       • DELETE: DELETE queries.
       • ANALYZE: ANALYZE queries.
       • DATA_DEFINITION: CREATE, ALTER, and DROP queries for schemas, tables, views, and materialized views, as well as queries that manage prepared statements, access permissions, sessions, and transactions.
       • ALTER_TABLE_EXECUTE: ALTER TABLE ... EXECUTE queries.

     • source: Query source name or a Java regular expression. The rule applies if the query source name matches the regular expression. If not specified, the rule applies to queries from all sources.

     • client_tags: Tag list. The rule applies only if the query contains all the specified tags. If not specified, the rule applies to queries with any tags.

     • group: Full name of the target resource group. It includes the names of all its parent groups, separated by dots, starting from the root group. A group name can use variable placeholders in ${<variable_name>} format. A target group cannot have sub-groups.

     Warning

     A query that does not satisfy any of the specified rules is rejected with an error. To avoid this, add to the end of the description a rule without any restrictive conditions, containing only the group property.

   • cpu_quota_period: CPU quota calculation period for resource groups. It takes an integer value in seconds, minutes, or hours, e.g., 140s, 30m, or 1h.

   For more information about the group tree and selector rules, see Resource groups in Managed Service for Trino.

4. Call the ClusterService.Update method, e.g., via the following gRPCurl request:

   grpcurl \
     -format json \
     -import-path ~/cloudapi/ \
     -import-path ~/cloudapi/third_party/googleapis/ \
     -proto ~/cloudapi/yandex/cloud/trino/v1/cluster_service.proto \
     -rpc-header "Authorization: Bearer $IAM_TOKEN" \
     -d @ \
     trino.api.cloud.yandex.net:443 \
     yandex.cloud.trino.v1.ClusterService.Update \
     < body.json
   

5. Check the server response to make sure your request was successful.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

To delete all resource groups and selector rules, run this command:

yc managed-trino cluster remove-resource-groups <cluster_name_or_ID>

You can get the cluster ID and name with the list of clusters in the folder.

1. Open the current Terraform configuration file describing your infrastructure.

   For more on how to create this file, see Creating a cluster.

2. To delete all resource groups and selector rules, delete the resource_groups parameter from the cluster description.

3. Make sure the settings are correct.

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

4. Confirm updating the resources.

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

1. Get an IAM token for API authentication and put it into an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Create a file named body.json and paste the following code into it:

   {
     "updateMask": "trino.resourceManagement.resourceGroups",
     "trino": {
       "resourceManagement": {
         "resourceGroups": {}
       }
     }
   }
   

   Where:

   • updateMask: Comma-separated list of parameters to update.

     Warning

     When you update a cluster, all parameters of the object you are modifying will be reset to their defaults unless explicitly provided in the request. To avoid this, list the settings you want to change in the updateMask parameter.

   • resourceGroups: Description of resource groups and selector rules.

3. Call the Cluster.Update method, e.g., via the following cURL request:

   curl \
     --request PATCH \
     --header "Authorization: Bearer $IAM_TOKEN" \
     --url 'https://trino.api.cloud.yandex.net/managed-trino/v1/clusters/<cluster_ID>'
     --data '@body.json'
   

   You can get the cluster ID with the list of clusters in the folder.

4. Check the server response to make sure your request was successful.

1. Get an IAM token for API authentication and place it in an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume that the repository contents reside in the ~/cloudapi/ directory.

3. Create a file named body.json and paste the following code into it:

   {
     "cluster_id": "<cluster_ID>",
     "update_mask": {
       "paths": [
         "trino.resource_management.resource_groups"
       ]
     },
     "trino": {
       "resource_management": {
         "resource_groups": {}
       }
     }
   }
   

   Where:

   • cluster_id: Cluster ID.

     You can get the cluster ID with the list of clusters in the folder.

   • update_mask: List of parameters to update as an array of strings (paths[]).

     Format for listing settings

     "update_mask": {
       "paths": [
         "<setting_1>",
         "<setting_2>",
         ...
         "<setting_N>"
       ]
     }
     

     Warning

     When you update a cluster, all parameters of the object you are modifying will be reset to their defaults unless explicitly provided in the request. To avoid this, list the settings you want to change in the update_mask parameter.

   • resource_groups: Description of resource groups and selector rules.

4. Call the ClusterService.Update method, e.g., via the following gRPCurl request:

   grpcurl \
     -format json \
     -import-path ~/cloudapi/ \
     -import-path ~/cloudapi/third_party/googleapis/ \
     -proto ~/cloudapi/yandex/cloud/trino/v1/cluster_service.proto \
     -rpc-header "Authorization: Bearer $IAM_TOKEN" \
     -d @ \
     trino.api.cloud.yandex.net:443 \
     yandex.cloud.trino.v1.ClusterService.Update \
     < body.json
   

5. Check the server response to make sure your request was successful.

resource-groups.json for this example:

{
  "rootGroups": [
    {
      "name": "global",
      "hardConcurrencyLimit": 20,
      "maxQueued": 200,
      "schedulingPolicy": "weighted_fair",
      "subGroups": [
        {
          "name": "adm",
          "hardConcurrencyLimit": 5,
          "maxQueued": 50,
          "schedulingWeight": 5
        },
        {
          "name": "bi",
          "hardConcurrencyLimit": 10,
          "softMemoryLimit": "20%",
          "maxQueued": 100,
          "schedulingWeight": 3,
          "subGroups": [
            {
              "name": "${toolname}",
              "hardConcurrencyLimit": 5,
              "maxQueued": 50,
              "subGroups": [
                {
                  "name": "${USER}",
                  "hardConcurrencyLimit": 2,
                  "maxQueued": 20
                }
              ]
            }
          ]
        },
        {
          "name": "etl",
          "hardConcurrencyLimit": 5,
          "softCpuLimit": "20s",
          "hardCpuLimit": "40s",
          "maxQueued": 50,
          "schedulingWeight": 1
        },
        {
          "name": "adhoc",
          "hardConcurrencyLimit": 5,
          "maxQueued": 100,
          "subGroups": [
            {
              "name": "${USER}",
              "hardConcurrencyLimit": 2,
              "maxQueued": 20
            }
          ]
        }
      ]
    }
  ],
  "selectors": [
    {
      "userGroup": "admins",
      "group": "global.adm"
    },
    {
      "clientTags": ["etl_job"],
      "group": "global.etl"
    },
    {
      "queryType": "DATA_DEFINITION",
      "group": "global.adm"
    },
    {
      "source": "jdbc#(?<toolname>[^#]+)",
      "queryType": "SELECT",
      "group": "global.bi.${toolname}.${USER}"
    },
    {
      "group": "global.adhoc.${USER}"
    }
  ],
  "cpuQuotaPeriod": "1m"
}

body.json for this example:

{
  "updateMask": "trino.resourceManagement.resourceGroups",
  "trino": {
    "resourceManagement": {
      "resourceGroups": {
        "rootGroups": [
          {
            "name": "global",
            "hardConcurrencyLimit": "20",
            "maxQueued": "200",
            "schedulingPolicy": "weighted_fair",
            "subGroups": [
              {
                "name": "adm",
                "hardConcurrencyLimit": "5",
                "maxQueued": "50",
                "schedulingWeight": "5"
              },
              {
                "name": "bi",
                "hardConcurrencyLimit": "10",
                "softMemoryLimit": "20%",
                "maxQueued": "100",
                "schedulingWeight": "3",
                "subGroups": [
                  {
                    "name": "${toolname}",
                    "hardConcurrencyLimit": "5",
                    "maxQueued": "50",
                    "subGroups": [
                      {
                        "name": "${USER}",
                        "hardConcurrencyLimit": "2",
                        "maxQueued": "20"
                      }
                    ]
                  }
                ]
              },
              {
                "name": "etl",
                "hardConcurrencyLimit": "5",
                "softCpuLimit": "20s",
                "hardCpuLimit": "40s",
                "maxQueued": "50",
                "schedulingWeight": "1"
              },
              {
                "name": "adhoc",
                "hardConcurrencyLimit": "5",
                "maxQueued": "100",
                "subGroups": [
                  {
                    "name": "${USER}",
                    "hardConcurrencyLimit": "2",
                    "maxQueued": "20"
                  }
                ]
              }
            ]
          }
        ],
        "selectors": [
          {
            "userGroup": "admins",
            "group": "global.adm"
          },
          {
            "clientTags": ["etl_job"],
            "group": "global.etl"
          },
          {
            "queryType": "DATA_DEFINITION",
            "group": "global.adm"
          },
          {
            "source": "jdbc#(?<toolname>[^#]+)",
            "queryType": "SELECT",
            "group": "global.bi.${toolname}.${USER}"
          },
          {
            "group": "global.adhoc.${USER}"
          }
        ],
        "cpuQuotaPeriod": "1m"
      }
    }
  }
}

body.json for this example:

{
  "cluster_id": "<cluster_ID>",
  "update_mask": {
    "paths": [
      "trino.resource_management.resource_groups"
    ]
  },
  "trino": {
    "resource_management": {
      "resource_groups": {
        "root_groups": [
          {
            "name": "global",
            "hard_concurrency_limit": "20",
            "max_queued": "200",
            "scheduling_policy": "weighted_fair",
            "sub_groups": [
              {
                "name": "adm",
                "hard_concurrency_limit": "5",
                "max_queued": "50",
                "scheduling_weight": "5"
              },
              {
                "name": "bi",
                "hard_concurrency_limit": "10",
                "soft_memory_limit": "20%",
                "max_queued": "100",
                "scheduling_weight": "3",
                "sub_groups": [
                  {
                    "name": "${toolname}",
                    "hard_concurrency_limit": "5",
                    "max_queued": "50",
                    "sub_groups": [
                      {
                        "name": "${USER}",
                        "hard_concurrency_limit": "2",
                        "max_queued": "20"
                      }
                    ]
                  }
                ]
              },
              {
                "name": "etl",
                "hard_concurrency_limit": "5",
                "soft_cpu_limit": "20s",
                "hard_cpu_limit": "40s",
                "max_queued": "50",
                "scheduling_weight": "1"
              },
              {
                "name": "adhoc",
                "hard_concurrency_limit": "5",
                "max_queued": "100",
                "sub_groups": [
                  {
                    "name": "${USER}",
                    "hard_concurrency_limit": "2",
                    "max_queued": "20"
                  }
                ]
              }
            ]
          }
        ],
        "selectors": [
          {
            "user_group": "admins",
            "group": "global.adm"
          },
          {
            "client_tags": ["etl_job"],
            "group": "global.etl"
          },
          {
            "query_type": "DATA_DEFINITION",
            "group": "global.adm"
          },
          {
            "source": "jdbc#(?<toolname>[^#]+)",
            "query_type": "SELECT",
            "group": "global.bi.${toolname}.${USER}"
          },
          {
            "group": "global.adhoc.${USER}"
          }
        ],
        "cpu_quota_period": "1m"
      }
    }
  }
}