Contact UsGet started
© 2025 Direct Cursus Technology L.L.C.

FAQ about Managed Service for Kubernetes

Written by
Improved by
Updated at August 20, 2025

General questions about

Data storage

Automatic scaling

Setup and updates

Resources

Logs

Troubleshooting in

General questions about

What services are available in Managed Service for Kubernetes clusters by default?

The following services are available by default:

Which Kubernetes CLI (kubectl) version do I need to install for comprehensive cluster management?

We recommend using the latest official version of kubectl to avoid compatibility issues.

Can Yandex Cloud restore the health of the cluster if I configure it incorrectly?

The master is managed by Yandex Cloud, that's why you can't damage it. If you have issues with Kubernetes cluster components, contact support.

Who will be monitoring the cluster health?

Yandex Cloud. A cluster is monitored for corrupted file system, kernel deadlock, internet connection loss and Kubernetes component issues. We're also developing a self-healing mechanism for faulty components.

How quickly does Yandex Cloud address vulnerabilities discovered in the security system? What should I do if an attacker has taken advantage of a vulnerability and my data is compromised?

Yandex Cloud services, images and master configuration initially undergo various security tests and checks for standard compliance.

Users can choose frequency of updates depending on their tasks and cluster configuration. It is important to consider attack targets and vulnerabilities in applications deployed in a Kubernetes cluster. Application security can be affected by such factors as network security policies between applications, vulnerabilities inside Docker containers, and incorrect launch mode of containers in a cluster.

Can I connect to a cluster node via OS Login, similar to a Yandex Cloud VM?

Yes, you can. To do this, follow the guide.

Data storage

What are the features of disk storage when a database (for example, MySQL® or PostgreSQL) is located in a Kubernetes cluster?

For a database located in a Kubernetes cluster, use StatefulSet controllers. We do not recommend running stateful services with persistent volumes in Kubernetes. To work with databases of stateful applications, use Yandex Cloud managed databases, e.g., Managed Service for MySQL® or Managed Service for PostgreSQL.

How do I connect a pod to managed Yandex Cloud databases?

To connect to a Yandex Cloud managed database located in the same network, specify its host name and FQDN.

To connect a database certificate to a pod, use the secret or configmap objects.

What's the right way to add a persistent volume to a container?

You can select connection mode for Compute Cloud disks depending on your needs:

  • If you want Kubernetes to automatically provision a PersistentVolume object and configure a new disk, create a pod with a dynamically provisioned volume.
  • To use existing Compute Cloud volumes, create a pod with a statically provisioned pod.

For more information, see Working with persistent volumes.

What types of volumes does Managed Service for Kubernetes support?

Managed Service for Kubernetes supports temporary (Volume) and persistent (PersistentVolume) volumes. For more information, see Volume.

Automatic scaling

Why does my cluster have N nodes and is not scaling down?

Autoscaling does not stop nodes with pods that cannot be evicted. The scaling barriers include:

  • Pods whose eviction is limited with PodDisruptionBudget.
  • Pods in the kube-system namespace:
    • Those not created under the DaemonSet controller.
    • Those without PodDisruptionBudget installed or those whose eviction is limited with PodDisruptionBudget.
  • Pods that were not created under a replication controller (ReplicaSet, Deployment, or StatefulSet).
  • Pods with local-storage.
  • Pods that cannot be evicted anywhere due to limitations. For example, due to lack of resources or lack of nodes matching the affinity or anti-affinity selectors.
  • Pods with an annotation that prohibits eviction: "cluster-autoscaler.kubernetes.io/safe-to-evict": "false".

Note

You can evict kube-system pods, pods with local-storage, and pods without a replication controller. To do this, set the "safe-to-evict": "true" annotation:

kubectl annotate pod <pod_name> cluster-autoscaler.kubernetes.io/safe-to-evict=true

Other possible causes include:

  • The node group has already reached its minimum size.

  • The node is idle for less than 10 minutes.

  • During the last 10 minutes, the node group has been scaled up.

  • During the last 3 minutes, there was an unsuccessful attempt to scale down the node group.

  • There was an unsuccessful attempt to stop a certain node. In this case, the next attempt occurs in 5 minutes.

  • The node has an annotation that prohibits stopping it when scaling it down: "cluster-autoscaler.kubernetes.io/scale-down-disabled": "true". You can add or remove an annotation using kubectl.

    Check for annotation on the node:

    kubectl describe node <node_name> | grep scale-down-disabled

    Result:

    Annotations:        cluster-autoscaler.kubernetes.io/scale-down-disabled: true

    Set the annotation:

    kubectl annotate node <node_name> cluster-autoscaler.kubernetes.io/scale-down-disabled=true

    You can remove an annotation by running the kubectl command with -:

    kubectl annotate node <node_name> cluster-autoscaler.kubernetes.io/scale-down-disabled-

Why does the node group fail to scale down after a pod deletion?

If the node is underloaded, it is removed in 10 minutes.

Why does autoscaling fail to trigger even though the number of nodes is below the minimum or exeeds the maximum?

Autoscaling will not violate the preset limits, but Managed Service for Kubernetes does not explicitly control the limits. Upscaling will only happen if there are pods in an unschedulable status.

Why do Terminated pods remain in my cluster?

This happens because the Pod garbage collector (PodGC) fails to delete these pods during autoscaling. For more information, see Deleting Terminated pods.

To get answers to other questions about autoscaling, see the Kubernetes documentation.

Is Horizontal Pod Autoscaler supported?

Yes, Managed Service for Kubernetes supports horizontal pod autoscaling.

Setup and updates

What should I do if I lose some of my data during a Kubernetes version upgrade?

Your data will not get lost: Managed Service for Kubernetes creates a data backup prior to a Kubernetes version upgrade. You can manually configure cluster backup in Yandex Object Storage. We also recommend backing up your database using the application tools.

Can I configure a backup for a Kubernetes cluster?

The Yandex Cloud infrastructure provides secure storage and replication for data in Managed Service for Kubernetes clusters. However, you can back up data from Managed Service for Kubernetes cluster node groups at any time and store them in Object Storage or other types of storage.

For more information, see Managed Service for Kubernetes cluster backups in Object Storage.

Will the resources be idle while Kubernetes is going through a version upgrade?

When a master is going through an upgrade, Control Plane resources will be idle. For this reason, operations like creating or deleting a Managed Service for Kubernetes node group will be unavailable. The application's user load will continue to be processed.

If max_expansion is greater than zero, new nodes are created when upgrading Managed Service for Kubernetes node groups. All load is diverted to the new nodes, and the old node groups get deleted. The idle time will be equal to pod restart time when transferred to a new Managed Service for Kubernetes group.

Can I upgrade a Managed Service for Kubernetes cluster in one step?

It depends on the source and target version you want to migrate your Managed Service for Kubernetes cluster from/to. You can only upgrade your Managed Service for Kubernetes cluster in a single step to the next minor version from the current one. Upgrading to newer versions is done in steps, e.g.: 1.19 → 1.20 → 1.21. For more information, see Updating a cluster.

If you want to skip interim versions, create a Managed Service for Kubernetes cluster of the appropriate version and transfer the load from the old cluster to the new one.

Is the Container Network Interface plugin upgraded together with the Managed Service for Kubernetes cluster?

Yes. If you are using Calico and Cilium controllers, they are upgraded together with your Managed Service for Kubernetes cluster. To upgrade your Managed Service for Kubernetes cluster, do one of the following:

To get timely Managed Service for Kubernetes cluster version upgrades, set up auto upgrading.

Can I send you a YAML configuration file so that you apply it to my cluster?

No. You can use a kubeconfig file to apply a YAML cluster configuration file on your own.

Can you install Web UI Dashboard, Rook, and other tools?

No. You can install all the necessary tools on your own.

What should I do if volumes refuse to connect after I upgrade Kubernetes?

If you get the following error after you upgrade Kubernetes:

AttachVolume.Attach failed for volume "pvc":
Attach timeout for volume yadp-k8s-volumes/pvc

Upgrade the s3-CSI driver to the latest version.

Resources

What resources are needed to maintain a Kubernetes cluster with a group of, say, three nodes?

Each node needs resources to run the components in charge of running the node as part of the Kubernetes cluster. For more information, see Dynamic resource allocation.

Can I change resources for each node in a Kubernetes cluster?

You can change resources only for a node group. You can create groups with different configurations in a Kubernetes cluster and place them in different availability zones. For more information, see Updating a Managed Service for Kubernetes node group.

Who monitors the scaling of a Kubernetes cluster?

In Managed Service for Kubernetes, you can enable automatic cluster scaling.

Logs

How can I monitor the Managed Service for Kubernetes cluster state?

Get the cluster statistics. You can view the description of the available cluster metrics in the reference.

Can I get logs of my operations in Yandex Cloud?

Yes, you can request information about operations with your resources from Yandex Cloud logs. For more information, see Data requests.

Can I save logs myself?

For log collection and storage, use Fluent Bit.

Can I use Yandex Cloud Logging for viewing logs?

Yes, you can. To do this, set up sending logs to Cloud Logging when creating or updating a Managed Service for Kubernetes cluster. The setting is only available in the CLI, Terraform, and API.

Troubleshooting

This section describes typical problems you may encounter while using Managed Service for Kubernetes and gives troubleshooting recommendations.

Error creating a cluster in a different folder's cloud network

Error message:

Permission denied

The error occurs when the resource service account has no required roles in the folder whose cloud network is selected when creating a cluster.

To create a Managed Service for Kubernetes cluster in a cloud network of another folder, assign the resource service account the following roles in this folder:

To use a public IP address, also assign the vpc.publicAdmin role.

A namespace has been deleted but its status is still Terminating and its deletion cannot be completed

This happens when a namespace has stuck resources that cannot be deleted by the namespace controller.

To fix the issue, delete the stuck resources manually.

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

  1. Connect to the Managed Service for Kubernetes cluster.

  2. Get a list of resources that remain within the namespace:

    kubectl api-resources --verbs=list --namespaced --output=name \
  | xargs --max-args=1 kubectl get --show-kind \
  --ignore-not-found --namespace=<namespace>

  3. Delete the resources found:

    kubectl delete <resource_type> <resource_name> --namespace=<namespace>

That being done, if the namespace is still in the Terminating status and cannot be deleted, delete it forcibly using finalizer:

  1. Enable Kubernetes API proxy to your local computer:

    kubectl proxy

  2. Delete the namespace:

    kubectl get namespace <namespace> --output=json \
  | jq '.spec = {"finalizers":[]}' > temp.json && \
curl --insecure --header "Content-Type: application/json" \
  --request PUT --data-binary @temp.json \
  127.0.0.1:8001/api/v1/namespaces/<namespace>/finalize

We do not recommend deleting the namespace with the Terminating status using finalizer right away, as this may cause the stuck resources to remain in your Managed Service for Kubernetes cluster.

I am using Yandex Network Load Balancer together with an ingress controller. Why are some of my cluster's nodes UNHEALTHY?

This is normal behavior for a load balancer with External Traffic Policy: Local enabled. Only the Managed Service for Kubernetes nodes whose pods are ready to accept user traffic get the HEALTHY status. The rest of the nodes are labeled as UNHEALTHY.

To find out the policy type of a load balancer created using a LoadBalancer type service, run this command:

kubectl describe svc <LoadBalancer_type_service_name> \
| grep 'External Traffic Policy'

For more information, see Parameters of a LoadBalancer service.

Why does the newly created PersistentVolumeClaim remain in Pending status?

This is normal for a PersistentVolumeClaim (PVC). The newly created PVC remains Pending until you create a pod that must use it.

To change the PVC status to Running:

  1. View details of the PVC:

    kubectl describe pvc <PVC_name> \
  --namespace=<namespace>

    Where --namespace is the namespace in which the PVC is located.

    A message saying waiting for first consumer to be created before binding means that the PVC is waiting for a pod to be created.

  2. Create a pod for this PVC.

Why does my Managed Service for Kubernetes cluster fail to run after I change its node configuration?

Make sure the new configuration of Managed Service for Kubernetes nodes is within the quota:

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

To run diagnostics for your Managed Service for Kubernetes cluster nodes:

  1. Connect to the Managed Service for Kubernetes cluster.

  2. Check the health of Managed Service for Kubernetes nodes:

    yc managed-kubernetes cluster list-nodes <cluster_ID>

    A message saying that the allowed amount of Managed Service for Kubernetes cluster resources has been exceeded is displayed in the first column of the command output. Here is an example:

    +--------------------------------+-----------------+------------------+-------------+--------------+
|         CLOUD INSTANCE         | KUBERNETES NODE |     RESOURCES    |     DISK    |    STATUS    |
+--------------------------------+-----------------+------------------+-------------+--------------+
| fhmil14sdienhr5uh89no          |                 | 2 100% core(s),  | 64.0 GB hdd | PROVISIONING |
| CREATING_INSTANCE              |                 | 4.0 GB of memory |             |              |
| [RESOURCE_EXHAUSTED] The limit |                 |                  |             |              |
| on total size of network-hdd   |                 |                  |             |              |
| disks has exceeded.,           |                 |                  |             |              |
| [RESOURCE_EXHAUSTED] The limit |                 |                  |             |              |
| on total size of network-hdd   |                 |                  |             |              |
| disks has exceeded.            |                 |                  |             |              |
+--------------------------------+-----------------+------------------+-------------+--------------+

To run your Managed Service for Kubernetes cluster, increase the quotas.

After changing the node subnet mask in the cluster settings, the number of pods on the nodes does not match the estimated count

Solution: Create the node group one more time.

Error renewing ingress controller certificate

Error message:

ERROR controller-runtime.manager.controller.ingressgroup Reconciler error
{"name": "some-prod", "namespace": , "error": "rpc error: code = InvalidArgument
desc = Validation error:\nlistener_specs[1].tls.sni_handlers[2].handler.certificate_ids:
Number of elements must be less than or equal to 1"}

The error occurs if different certificates are specified for the same ingress controller listener.

Solution: Edit and apply the ingress controller specifications so that only one certificate is specified in each listener's description.

Why is DNS name resolution not working in my cluster?

There may be no name resolution for internal and external DNS queries in a Managed Service for Kubernetes cluster for several reasons. To fix the issue:

  1. Check the version of your Managed Service for Kubernetes cluster and node groups.
  2. Make sure that CoreDNS is up and running.
  3. Make sure the Managed Service for Kubernetes cluster has enough CPU resources available.
  4. Set up autoscaling.
  5. Set up local DNS caching.
Check the version of your cluster and node groups

  1. Get a list of current Kubernetes versions:

    yc managed-kubernetes list-versions

  2. Find out the Managed Service for Kubernetes cluster version:

    yc managed-kubernetes cluster get <cluster_name_or_ID> | grep version:

    You can get the Managed Service for Kubernetes cluster ID and name with a list of clusters in the folder.

  3. Find out the Managed Service for Kubernetes node group version:

    yc managed-kubernetes node-group get <node_group_name_or_ID> | grep version:

    You can get the ID and name of the Managed Service for Kubernetes node group with a list of node groups in your cluster.

  4. If the versions of your Managed Service for Kubernetes cluster and node groups are not on the list of current Kubernetes versions, upgrade them.

Make sure that CoreDNS is up and running

Get a list of CoreDNS pods and their statuses:

kubectl get pods -n kube-system -l k8s-app=kube-dns -o wide

Make sure all the pods have the Running status.

Make sure the cluster has enough CPU resources available
  1. Navigate to the folder dashboard and select Managed Service for Kubernetes.
  2. Click the name of the Managed Service for Kubernetes cluster you need and select the Node manager tab.
  3. Go to the Nodes tab and click the name of any Managed Service for Kubernetes node.
  4. Go to the Monitoring tab.
  5. Make sure that, in the CPU, [cores] chart, the used CPU values have not reached the total available CPU values. Check this for each Managed Service for Kubernetes cluster node.
Set up autoscaling

Set up automatic DNS scaling by Managed Service for Kubernetes cluster size.

Set up local DNS caching

Set up NodeLocal DNS Cache. To make sure that the settings are optimal, install NodeLocal DNS Cache from Yandex Cloud Marketplace.

There is a parameter conflict when creating a node group via the CLI. How do I fix it?

Check whether the --location, --network-interface, and --public-ip parameters are specified in the same command. If you provide these parameters together, the following errors occur:

  • For the --location and --public-ip or --location and --network-interface pairs:

    ERROR: rpc error: code = InvalidArgument desc = Validation error:
allocation_policy.locations[0].subnet_id: can't use "allocation_policy.locations[0].subnet_id" together with "node_template.network_interface_specs"

  • For the --network-interface and --public-ip pair:

    ERROR: flag --public-ip cannot be used together with --network-interface. Use '--network-interface' option 'nat' to get public address

Make sure you only provide one of the three parameters in a command. It is enough to specify the location of a Managed Service for Kubernetes node group either in --location or --network-interface.

To grant internet access to Managed Service for Kubernetes cluster nodes, do one of the following:

Error connecting to a cluster using kubectl

Error message:

ERROR: cluster has empty endpoint

The error occurs if you try to connect to a cluster with no public IP address and get kubectl credentials for a public IP address using this command:

yc managed-kubernetes cluster \
   get-credentials <cluster_name_or_ID> \
   --external

To connect to the cluster's private IP address from a VM located in the same network, get kubectl credentials using this command:

yc managed-kubernetes cluster \
   get-credentials <cluster_name_or_ID> \
   --internal

If you need to connect to a cluster from the internet, recreate the cluster and assign it a public IP address.

Errors connecting to a node over SSH

Error messages:

Permission denied (publickey,password)

Too many authentication failures

Errors occur when connecting to a Managed Service for Kubernetes node in the following cases:

How do I grant internet access to Managed Service for Kubernetes cluster nodes?

If Managed Service for Kubernetes cluster nodes have no access to the internet, the following error occurs when trying to connect to the internet:

Failed to pull image "cr.yandex/***": rpc error: code = Unknown desc = Error response from daemon: Gethttps://cr.yandex/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

There are several ways to grant internet access to Managed Service for Kubernetes cluster nodes:

Note

If you assigned public IP addresses to the cluster nodes and then configured the NAT gateway or NAT instance, internet access via the public IP addresses will be disabled. For more information, see the Yandex Virtual Private Cloud documentation.

Why cannot I choose Docker as the container runtime environment?

There is no support for Docker as a container runtime environment in clusters with Kubernetes version 1.24 or higher. Only containerd is available.

Error connecting a GitLab repository to Argo CD

Error message:

FATA[0000] rpc error: code = Unknown desc = error testing repository connectivity: authorization failed

This error occurs if access to GitLab over HTTP(S) is disabled.

Solution: Enable access. To do this:

  1. In GitLab, on the left-hand panel, select Admin → Settings → General.
  2. Under Visibility and access controls, find the Enabled Git access protocols setting.
  3. In the list, select the item which allows access over HTTP(S).

For more information, see the GitLab documentation.

Traffic loss when deploying app updates in a cluster with Yandex Application Load Balancer

When your app traffic is managed by an Application Load Balancer and the load balancer's ingress controller traffic policy is set to externalTrafficPolicy: Local, the app processes requests on the same node they were delivered to by the load balancer. There is no traffic flow between nodes.

The default health check monitors the status of the node, not application. Therefore, Application Load Balancer traffic may go to a node where there is no application running. When you deploy a new app version in a cluster, the Application Load Balancer ingress controller requests the load balancer to update the backend group configuration. It takes at least 30 seconds to process the request, during which time the app may not be getting any user traffic.

To prevent this, we recommend setting up backend health checks on your Application Load Balancer. Thanks to health checks, the load balancer spots unavailable backends in a timely manner and diverts traffic to other backends. Once the app update is over, traffic will once again be distributed between all backends.

For more information, see Tips for configuring Yandex Application Load Balancer health checks and Annotations (metadata.annotations).

System time displayed incorrectly on nodes, as well as in container and Managed Service for Kubernetes cluster pod logs

Managed Service for Kubernetes cluster time may deviate from the time of other resources, e.g., a VM, if they use different clock sources for synchronization. For example, a Managed Service for Kubernetes cluster synchronizes with a time server (by default), whereas a VM synchronizes with a private or public NTP server.

Solution: Set up Managed Service for Kubernetes cluster time synchronization with your private NTP server. To do this:

  1. Specify the NTP server's addresses in the DHCP settings of the master subnets.

    1. Navigate to the folder dashboard and select Managed Service for Kubernetes.
    2. Click the name of the Kubernetes cluster.
    3. Under Master configuration, click the subnet name.
    4. Click Edit in the top-right corner.
    5. In the window that opens, expand the DHCP settings section.
    6. Click Add and specify the IP address of your NTP server.
    7. Click Save changes.

    If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

    By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

    1. View the description of the CLI command for updating subnet parameters:

      yc vpc subnet update --help

    2. Run the subnet command with the NTP server's IP address in the --ntp-server parameter:

      yc vpc subnet update <subnet_ID> --ntp-server <server_address>

    Tip

    To find out the IDs of the subnets containing the cluster, get detailed information about the cluster.

    1. In the Terraform configuration file, change the cluster subnet description. Add the dhcp_options section (if there is none) with the ntp_servers parameter and specify the IP address of your NTP server:

      ...
resource "yandex_vpc_subnet" "lab-subnet-a" {
  ...
  v4_cidr_blocks = ["<IPv4_address>"]
  network_id     = "<network_ID>"
  ...
  dhcp_options {
    ntp_servers = ["<IPv4_address>"]
    ...
  }
}
...

      For more information about yandex_vpc_subnet properties in Terraform, see the relevant provider documentation.

    2. Apply the changes:

      1. In the terminal, go to the directory where you edited the configuration file.

      2. Make sure the configuration file is correct using this command:

        terraform validate

        If the configuration is correct, you will get this message:

        Success! The configuration is valid.

      3. Run this command:

        terraform plan

        You will see a detailed list of resources. No changes will be made at this step. If the configuration contains any errors, Terraform will show them.

      4. Apply the changes:

        terraform apply

      5. Type yes and press Enter to confirm the changes.

      Terraform will update all required resources. You can check the subnet update using the management console or this CLI command:

      yc vpc subnet get <subnet_name>

    Use the update method for the Subnet resource and include the following in the request:

    • NTP server's IP address in the dhcpOptions.ntpServers parameter.
    • The dhcpOptions.ntpServers parameter to update in the updateMask parameter.

    Tip

    To find out the IDs of the subnets containing the cluster, get detailed information about the cluster.

    Warning

    For a highly available master hosted across three availability zones, you need to update each of the three subnets.

  2. Enable connections from the cluster to NTP servers.

    Create a rule in the cluster and node groups security group:

    • Port range: 123. If using any port other than port 123 on the NTP server, specify it.
    • Protocol: UDP.
    • Destination name: CIDR.
    • CIDR blocks: <NTP_server_IP_address>/32. For a master hosted across three availability zones, specify three sections: <NTP_server_IP_address_in_subnet1>/32, <NTP_server_IP_address_in_subnet2>/32, <NTP_server_IP_address_in_subnet3>/32.

  3. Update the network settings in the cluster node group using one of the following methods:

    • Connect to each node in the group over SSH or via OS Login and run the sudo dhclient -v -r && sudo dhclient command.
    • Reboot the group nodes at a time convenient for you.

    Warning

    Updating network parameters may cause the services within the cluster to become unavailable for a few minutes.

What should I do if I have deleted my Yandex Network Load Balancer or target groups that were automatically created for a LoadBalancer service?

You cannot manually restore a Network Load Balancer or target groups. Recreate your LoadBalancer service. This will automatically create a load balancer and target groups.

Previous
Troubleshooting