Migrating from a public cloud to Managed Service for Kubernetes

This guide explains how to migrate Kubernetes workloads from a managed service of another cloud provider to Managed Service for Kubernetes.

The key strategy is to migrate step by step while keeping both the source and target clusters running in parallel. This minimizes the impact on production services and ensures a safe rollback path.

To migrate workloads to Managed Service for Kubernetes:

1. Develop a migration plan.
2. Set up your Yandex Cloud infrastructure.
3. Migrate container images.
4. Migrate stateful workloads.
5. Migrate stateless workloads.
6. Configure the load balancers and update your DNS records.
7. Complete your migration.

Required paid resourcesRequired paid resources

• Managed Service for Kubernetes master (see Managed Service for Kubernetes pricing).
• Managed Service for Kubernetes cluster nodes: use of computing resources and storage (see Yandex Compute Cloud pricing).
• Public IP addresses for the Managed Service for Kubernetes cluster's master and nodes with public access enabled (see Yandex Virtual Private Cloud pricing).
• Each active L7 load balancer, if using the Gwin controller: use of computing resources (see Application Load Balancer pricing).
• Each network load balancer, if using the LoadBalancer service type: incoming and outgoing traffic processed by the load balancer (see Yandex Network Load Balancer pricing).
• Container Registry: storage of migrated Docker images (see Container Registry pricing).
• Object Storage bucket: use of storage, data operations (see Yandex Object Storage pricing).
• Data Transfer transfers, if migrating managed databases (see Yandex Data Transfer pricing).

Getting startedGetting started

Sign up for Yandex Cloud and create a billing account:

1. Navigate to the management console and log in to Yandex Cloud or create a new account.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can create or select a folder for your infrastructure on the cloud page.

Learn more about clouds and folders here.

Develop a migration planDevelop a migration plan

Before the migration, make sure to inventory your workloads and assess the compatibility between the source and target clusters. This helps you estimate migration complexity, map out the order in which to migrate your services, and develop a rollback plan in case of critical issues.

Inventory your workloadsInventory your workloads

Get a list of source cluster resources for all namespaces: applications, versions, dependencies, and network rules. Review the full migration plan in advance to understand in what order you need to move your resources.

Make sure that all stateful workloads, including managed databases and StatefulSets with PersistentVolumes, have up-to-date backups.

Run this command to get a full list of your source cluster resources:

for ns in $(kubectl get ns --no-headers -o custom-columns=':metadata.name'); do
  echo "=== Namespace: $ns ==="
  kubectl get deploy,sts,ds,job,cronjob,svc,ingress,pvc,cm,secret,hpa,pdb -n $ns
done

Inventory checklist:

Assess compatibilityAssess compatibility

To make sure your cluster components work correctly after the migration, assess their compatibility with the target cluster in advance. Create a list of changes you need to make to the manifests and plan how to replace incompatible source cluster components with their Yandex Cloud alternatives.

The scope of changes depends on the version gap between the source and target Kubernetes clusters. The wider the gap, the more changes you need to make, as both vendor-specific CRDs and standard Kubernetes API groups may differ. For supported versions, see Support for Kubernetes versions.

Key considerations to keep in mind:

• Check for vendor-specific CRDs in the source cluster that may not be available in Yandex Cloud. Replace the components with Yandex Cloud services or solutions from Yandex Cloud Marketplace.
• Note that PodSecurityPolicy is removed in Kubernetes versions 1.25 and up. If using PodSecurityPolicy in the source cluster, migrate to PodSecurity Admission.
• Replace the storage classes (storageClassName) in your manifests with those supported in Managed Service for Kubernetes.
• We recommend using the Gwin controller for L7 load balancing.

Develop the service migration planDevelop the service migration plan

Decide on the order for migrating your services: start with the least critical ones and finish with production workloads. To redirect traffic, use a zero-downtime blue-green deployment strategy or a phased transition with a maintenance window.

Key considerations to keep in mind:

• Perform an end-to-end migration in a staging environment before migrating production workloads.
• Develop a rollback plan for critical issues and align it with your team in advance.
• Work closely with development and support teams at all stages.
• Choose off-peak hours to perform the migration.
• Monitor metrics before, during, and after the migration using Yandex Monitoring.

Set up your Yandex Cloud infrastructureSet up your Yandex Cloud infrastructure

Prepare the target environment in Yandex Cloud for your workloads.

Configure access and permissionsConfigure access and permissions

Create service accounts to manage cluster resources and worker nodes, and assign roles to them.

Set up the network infrastructureSet up the network infrastructure

Set up a network infrastructure for the cluster: an isolated network with security rules for all types of cluster traffic:

1. Create a VPC and subnets in three zones to ensure high availability.
2. Create security groups to isolate cluster traffic, node traffic, and inbound connections. You must create five groups in total.

Create a Managed Service for Kubernetes clusterCreate a Managed Service for Kubernetes cluster

1. Create a Managed Service for Kubernetes cluster. Use the same Kubernetes version as in your source cluster or higher.
2. Create a node group in each availability zone.

Migrate container imagesMigrate container images

Move container images to Yandex Container Registry one by one, keeping their tags and metadata.

You can also use Yandex Cloud Registry as an alternative storage for images. Unlike Yandex Container Registry, Cloud Registry supports not only Docker images but other artifact types as well.

Migrate stateful workloadsMigrate stateful workloads

Migrating stateful workloads, such as managed databases and StatefulSets with PersistentVolumes, is the most complex step of the migration. They store data and cannot be recreated with zero data loss.

Migrate managed databasesMigrate managed databases

Use Yandex Data Transfer to migrate managed databases. Yandex Data Transfer allows you to migrate data while keeping your source online and minimizing the downtime of applications using the source.

Use the transfer matrix to check that the Snapshot and increment transfer type is supported for your source and target, then refer to your database guide to complete the migration.

If Data Transfer does not support your source or you need more control, refer to your target database guides.

Migrate StatefulSets with PersistentVolumesMigrate StatefulSets with PersistentVolumes

To migrate StatefulSets with PersistentVolumes, use Velero with file system backups via node-agent (formerly kopia). This method works with any PVCs regardless of your CSI provider.

Proceed as follows:

1. Create a bucket in Yandex Object Storage to store backups.

2. Create a file named credentials-velero containing the static access key data for the service account that has access to the bucket:

   [default]
   aws_access_key_id = <key_ID>
   aws_secret_access_key = <secret_key>
   

3. Install the Velero CLI on the machine you will use to perform the migration. It must have kubectl access to the source and target clusters.

   Tip

   The velero-plugin-for-aws plugin version in the commands below may be outdated. For the latest version, refer to this Velero article.

4. Install Velero in the source cluster:

   1. Make sure the source cluster context is active:

      kubectl config current-context
      

   2. Create a namespace for Velero:

      kubectl create namespace velero
      

   3. Install Velero:

      velero install \
        --provider aws \
        --plugins velero/velero-plugin-for-aws:v1.14.0 \
        --bucket <bucket_name> \
        --secret-file ./credentials-velero \
        --backup-location-config \
          region=ru-central1,s3ForcePathStyle=true,s3Url=https://storage.yandexcloud.net \
        --use-volume-snapshots=false \
        --use-node-agent \
        --uploader-type=kopia
      

      Where:

      • --provider aws: Uses an AWS plugin compatible with Object Storage.
      • --use-node-agent: Enables DaemonSet to perform file system backups.
      • --uploader-type=kopia: Recommended uploader, works faster than restic.
      • --use-volume-snapshots=false: Disables CSI snapshots to transfer data via kopia.

   4. Check component status:

      kubectl get pods -n velero
      kubectl get backupstoragelocation -n velero
      velero backup-location get
      

      BackupStorageLocation must switch to Available.

5. Create a backup of the required namespaces:

   velero backup create migration-backup \
     --include-namespaces <namespace> \
     --default-volumes-to-fs-backup \
     --wait
   

   You can check the status and logs using the following commands:

   velero backup describe migration-backup --details
   velero backup logs migration-backup
   

6. Install Velero in the target Managed Service for Kubernetes cluster:

   1. Switch to the target cluster context:

      kubectl config use-context <target_cluster_context>
      

   2. Create a namespace for Velero:

      kubectl create namespace velero
      

   3. Install Velero with the same parameters as in the source cluster, specifying the same bucket:

      velero install \
        --provider aws \
        --plugins velero/velero-plugin-for-aws:v1.14.0 \
        --bucket <bucket_name> \
        --secret-file ./credentials-velero \
        --backup-location-config \
          region=ru-central1,s3ForcePathStyle=true,s3Url=https://storage.yandexcloud.net \
        --use-volume-snapshots=false \
        --use-node-agent \
        --uploader-type=kopia
      

   4. Confirm that the backup from the source cluster is visible in the target cluster:

      kubectl get pods -n velero
      velero backup-location get
      velero backup get
      

      BackupStorageLocation must switch to Available, and the migration-backup, to Completed.

7. Restore data from the backup:

   velero restore create \
     --from-backup migration-backup \
     --wait
   

   You can check the status and logs using the following commands:

   velero restore describe migration-backup --details
   velero restore logs migration-backup
   

   Tip

   If the StorageClass names in the source cluster and Managed Service for Kubernetes cluster do not match, create a ConfigMap with the StorageClass mapping before initiating recovery:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: change-storage-class-config
     namespace: velero
     labels:
       velero.io/plugin-config: ""
       velero.io/change-storage-class: RestoreItemAction
   data:
     <source_storage_class>: yc-network-ssd
   

Migrate stateless workloadsMigrate stateless workloads

To migrate stateless workloads, export your source cluster manifests, update them for Yandex Cloud, and apply to the target cluster.

Checklist for updating manifests:

• Replace vendor-specific CRDs with Yandex Cloud services or solutions from Yandex Cloud Marketplace.
• For any deprecated apiVersion, update the version to make it compatible with the target Kubernetes version.
• Use the Gwin controller (gwin-default) in ingressClassName and gatewayClassName.
• Replace resource annotations used for configuring the load balancer with Yandex Cloud annotations.
• Make sure image links lead to Yandex Container Registry (cr.yandex/<id>/...).

You can use kubectl-neat to clean up manifests.

Configure the load balancers and update your DNS recordsConfigure the load balancers and update your DNS records

Reroute traffic to the new cluster.

Managed Service for Kubernetes supports two load balancing methods:

• Gwin.
• Network Load Balancer: Created automatically for a service of the LoadBalancer type.

Steps to switch traffic:

1. Deploy Gateway or Ingress resources in the Managed Service for Kubernetes cluster.
2. Get the external IP addresses of the new load balancers.
3. Test your services using temporary DNS records.
4. Reduce the TTL of production DNS records in advance, then update them incrementally.
5. Monitor DNS propagation and TLS certificate updates.

Complete your migrationComplete your migration

Take final steps and optimize your configuration:

1. Make sure all services work correctly in Managed Service for Kubernetes.
2. Check metrics and performance in Yandex Monitoring.
3. Disable your source cloud resources after confirming stable operation.
4. Optimize the cluster configuration to match your actual workload.
5. Delete temporary resources: the Velero bucket, Data Transfer pipelines, VPN tunnels, and temporary login credentials.