Updating the Metrics Server parameters in a Yandex Managed Service for Kubernetes cluster

Metrics Server is a Managed Service for Kubernetes cluster service installed by default. It collects metrics from each Managed Service for Kubernetes cluster node using kubelet and sends them through the Metrics API. Horizontal Pod Autoscaler and Vertical Pod Autoscaler run based on data from these metrics. You can get the metric data using the kubectl top node or kubectl top pod commands. For more information about Metrics Server, see this guide.

A Metrics Server pod has two containers: metrics-server and metrics-server-nanny, the latter acting as addon-resizer for metrics-server. The metrics-server-nanny container handles automatic allocation of resources to the metrics-server container based on the number of Managed Service for Kubernetes cluster nodes.

In some cases, metrics-server-nanny may run incorrectly. For example, if a Managed Service for Kubernetes cluster with few nodes runs a large number of pods. In this case, the Metrics Server pod will exceed its limits, which may degrade Metrics Server performance.

To avoid this, update the Metrics Server parameters manually:

1. View the resources allocated to the Metrics Server pod.
2. Update the Metrics Server parameters.
3. Check the result.

To restore the default values of the Metrics Server parameters, reset them.

View the resources allocated to the Metrics Server podView the resources allocated to the Metrics Server pod

1. Create security groups for the Managed Service for Kubernetes cluster and its node groups.

   Warning

   The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.

2. Install kubect and configure it to work with the new cluster.

3. Run this command:

   kubectl get pod <Metrics_Server_pod_name> \
     --namespace=kube-system \
     --output=json | \
     jq '.spec.containers[] | select(.name == "metrics-server") | .resources'
   

   The resources are calculated using the following formulas:

   cpu = baseCPU + cpuPerNode * nodesCount
   memory = baseMemory + memoryPerNode * nodesCount
   

   Where:

   • baseCPU: Base CPU count.
   • cpuPerNode: Number of CPUs per node.
   • nodesCount: Number of Managed Service for Kubernetes nodes.
   • baseMemory: Base RAM size.
   • memoryPerNode: RAM per node.

Update the Metrics Server parametersUpdate the Metrics Server parameters

1. Open the Metrics Server configuration file:

   kubectl edit configmap metrics-server-config \
     --namespace=kube-system \
     --output=yaml
   

2. Add or update the resource parameters under data.NannyConfiguration:

   apiVersion: v1
   data:
     NannyConfiguration: |-
       apiVersion: nannyconfig/v1alpha1
       kind: NannyConfiguration
       baseCPU: <base_CPU_count>m
       cpuPerNode: <number_of_CPUs_per_node>m
       baseMemory: <base_RAM_size>Mi
       memoryPerNode: <RAM_per_node>Mi
   ...
   

   Configuration file example

   apiVersion: v1
   data:
     NannyConfiguration: |-
       apiVersion: nannyconfig/v1alpha1
       kind: NannyConfiguration
       baseCPU: 48m
       cpuPerNode: 1m
       baseMemory: 104Mi
       memoryPerNode: 3Mi
   kind: ConfigMap
   metadata:
     creationTimestamp: "2022-12-15T06:28:22Z"
     labels:
       addonmanager.kubernetes.io/mode: EnsureExists
     name: metrics-server-config
     namespace: kube-system
     resourceVersion: "303569"
     uid: 931b88ca-21da-4d04-a3c1-da7e********
   

3. Restart Metrics Server. To do this, delete it and wait until the Kubernetes controller recreates it:

   kubectl delete deployment metrics-server \
     --namespace=kube-system
   

Check the resultCheck the result

View the resources allocated for the Metrics Server pod again and make sure the new pod's allocation is different.

Reset the Metrics Server parametersReset the Metrics Server parameters

To reset the parameters to their default values, delete the Metrics Server configuration file and its Deployment:

kubectl delete configmap metrics-server-config \
  --namespace=kube-system && \
kubectl delete deployment metrics-server \
  --namespace=kube-system

This will automatically create a new Metrics Server pod.