Migrating services from a NLB with target resources from the Yandex Managed Service for Kubernetes cluster to an L7 ALB

You can use a Yandex Network Load Balancer load balancer as part of services within a Yandex Managed Service for Kubernetes cluster. The cluster itself creates network load balancer objects according to the manifests provided and monitors the load balancer's target group receiving VMs from that cluster's node groups.

One of the options for a network load balancer in a Managed Service for Kubernetes cluster is to use it as part of an nginx Ingress controller.

This tutorial covers the migration from a network load balancer to an Yandex Application Load Balancer L7 load balancer created by an Application Load Balancer Ingress controller, with a Yandex Smart Web Security security profile connected.

To migrate a service from a network load balancer to an L7 load balancer:

1. See recommendations for service migration.
2. Complete the prerequisite steps.
3. Create a Smart Web Security profile.
4. Install an Application Load Balancer Ingress controller and create resources in your Managed Service for Kubernetes cluster. At this step, you will connect your Smart Web Security profile to the L7 load balancer.
5. Migrate user load from the network load balancer to the L7 load balancer.

Service migration recommendationsService migration recommendations

1. In addition to DDoS protection at level L7 of the OSI model using Yandex Smart Web Security, we recommend enabling DDoS protection at L3-L4. To do this, reserve a public static IP address with DDoS protection in advance and use this address for the L7 load balancer's listener.

   If the network load balancer's listener already uses a public IP address with DDoS protection, you can save it and use it for the L7 load balancer.

   If the network load balancer's listener uses a public IP address without DDoS protection, the only way to enable L7 load balancer DDoS protection at level L3-L4 is to change the public IP address for your service.

   When using L3-L4 DDoS protection, configure a trigger threshold for the L3-L4 protection mechanisms aligned with the amount of legitimate traffic to the protected resource. To set up this threshold, contact support.

   Also, set the MTU value to 1450 for the target resources downstream of the load balancer. For more information, see Setting up MTU when enabling DDoS protection.

2. We recommend performing migration during the hours when user load is at its lowest. If you plan to keep your public IP address, bear in mind that migration involves moving this IP address from the load balancer to the L7 load balancer. Your service will be unavailable during this period. Under normal conditions, this may last for several minutes. After the user load is migrated from the network load balancer to the L7 load balancer, it may take time for the L7 balancer's resource unit group to autoscale depending on the external load on the load balancer nodes.

3. When using an L7 load balancer, requests to backends come with the source IP address from the range of internal IP addresses of the subnets specified when creating the L7 load balancer. The original IP address of the request source (user) is specified in the X-Forwarded-For header. If you want to log public IP addresses of users on the web server, reconfigure it.

4. For the L7 load balancer, two resource units will be created in each of the subnets specified when creating the Ingress resource. The Ingress resource annotations do not support specifying the minimum number of resource units for an L7 load balancer. A group of resource units is automatically scaled depending on the external load on load balancer nodes.

5. The features of the Application Load Balancer load balancer may differ from those of your load balancer deployed in the Managed Service for Kubernetes cluster. See the Application Load Balancer Ingress controller description and operating principles.

Getting startedGetting started

1. Create subnets in three availability zones. These will be used for the L7 load balancer.

2. Create security groups that allow the L7 load balancer to receive incoming traffic and send it to the target resources and allow the target resources to receive incoming traffic from the load balancer.

3. When using HTTPS, add your service's TLS certificate to Yandex Certificate Manager.

4. Reserve a static public IP address with DDoS protection at level L3-L4 for the L7 load balancer. See service migration recommendations.

5. The Managed Service for Kubernetes services used as backends must be of the NodePort type. If your services employ another type, change it to NodePort. For more details on this type, please see the Kubernetes documentation.

Create a Smart Web Security security profileCreate a Smart Web Security security profile

Create a Smart Web Security security profile by selecting From a preset template.

Use these settings when creating the profile:

• In the Action for the default base rule field, select Allow.
• For the Smart Protection rule, enable Only logging (dry run).

These settings are limited to logging the info about the traffic without applying any actions to it. This will reduce the risk of disconnecting users due to profile configuration issues. As you move along, you will be able to turn Only logging (dry run) off and configure some prohibiting rules for your use case in the security profile.

Install an Application Load Balancer Ingress controller and create resources in your Managed Service for Kubernetes clusterInstall an Application Load Balancer Ingress controller and create resources in your Managed Service for Kubernetes cluster

1. Install an Yandex Application Load Balancer Ingress controller.

2. Create an IngressClass resource for the L7 load balancer's Ingress controller:

   1. Create a YAML file and describe the IngressClass resource in it.

      Example of IngressClass resource

      apiVersion: networking.k8s.io/v1
      kind: IngressClass
      metadata:
        labels:
          app.kubernetes.io/component: controller
        name: ingress-alb
      spec:
        controller: ingress.alb.yc.io/yc-alb-ingress-controller
      

   2. Use the following command to create the IngressClass resource:

      kubectl apply -f <IngressClass_resource_file>
      

3. Create an Ingress resource:

   1. Read the descriptions of the Ingress resource fields and annotations and see the example.

   2. Create a YAML file and describe the Ingress resource in it:

      • Complete the annotations section for the L7 load balancer settings:

        • ingress.alb.yc.io/subnets: IDs of the subnets in the three availability zones for the L7 load balancer nodes. Specify the IDs separated by commas with no spaces.

        • ingress.alb.yc.io/security-groups: ID of one or more security groups for the L7 load balancer. For multiple groups, specify their IDs separated by commas with no spaces.

        • ingress.alb.yc.io/external-ipv4-address: Previously reserved static public IP address.

        • ingress.alb.yc.io/group-name: Name of the Ingress resource group. Ingress resources are grouped together, each group served by a separate Application Load Balancer instance with a dedicated public IP address.

        • ingress.alb.yc.io/security-profile-id: ID of the previously created Smart Web Security security profile.

          Warning

          The security profile will be linked to the virtual host of the L7 load balancer. Specifying your security profile is the key step to connecting Smart Web Security.

      • For the ingressClassName field, enter the name of the IngressClass resource you created earlier.

      • When using HTTPS, complete the tls section:

        • hosts: Your service domain name the TLS certificate corresponds to.
        • secretName: TLS certificate of your service in Yandex Certificate Manager, in yc-certmgr-cert-id-<certificate_ID> format.

      • Complete the rules section in line with your rules for distribution of incoming traffic among backends depending on the domain name (host field) and requested resource (http.paths field).

        • host: Your service domain name.

        • pathType: Type of reference to the requested resource:

          • Exact: Request URI path must match the path field value.
          • Prefix: Request URI path must start with the path field value.

        • path: Incoming request URI path (if Exact) or its prefix (if Prefix).

        • backend: Reference to a backend or group of backends to process the requests with the specified domain name and URI path. Specify either a service backend (service) or a backend group (resource) but not both.

          • service: Managed Service for Kubernetes service to process the requests as a backend:

            • name: Managed Service for Kubernetes service name. The Service resource this field refers to must be described in line with this configuration.
            • port: Service port Ingress is going to address. For the service port, specify either a number (number) or a name (name) but not both.

            Warning

            The Managed Service for Kubernetes services used as backends must be of the NodePort type.

          • resource: Reference to the HttpBackendGroup group of backends to process the requests. A group like this may have Managed Service for Kubernetes services or Yandex Object Storage buckets as backends. When using a backend group, advanced Application Load Balancer functionality is available. You can also specify relative backend weights to allocate traffic to them in proportion.

            • kind: HttpBackendGroup.
            • name: Backend group name. The name must match the value specified in the metadata.name field of the HttpBackendGroup resource. The HttpBackendGroup resource this field refers to must be described in line with this configuration.
            • apiGroup: alb.yc.io.
      Example of Ingress resource

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: <resource_name>
        annotations:
          ingress.alb.yc.io/subnets: <ru-central1-a_subnet_ID,ru-central1-b_subnet_ID,ru-central1-d_subnet_ID>
          ingress.alb.yc.io/security-groups: <L7_load_balancer_security_group_ID>
          ingress.alb.yc.io/external-ipv4-address: <static_public_IP_address>
          ingress.alb.yc.io/group-name: <resource_group_name>
          ingress.alb.yc.io/security-profile-id: <Smart_Web_Security_security_profile_ID>
      spec:
        ingressClassName: <IngressClass_resource_name>
        tls:
          - hosts:
              - <service_domain_name>
            secretName: yc-certmgr-cert-id-<certificate_ID>
        rules:
          - host: <service_domain_name>
            http:
              paths:
              - path: /
                pathType: Prefix
                backend:
                  service:
                    name: <Kubernetes_service_name>
                    port:
                      number: <port_number_e.g._443>
      

   3. Use the following command to create the Ingress resource:

      kubectl apply -f <Ingress_resource_file>
      

4. An L7 load balancer will be deployed based on the Ingress resource configuration. Wait until its creation is complete and Ingress has a public IP address linked. You will need this IP address to check requests. You can view resource info using this command:

   kubectl get ingress <Ingress_resource_name> -w
   

5. Run a test request to the service through the L7 load balancer, for example, using one of these methods:

   • Add this record to the hosts file on your workstation: <L7_load_balancer_public_IP_address> <service_domain_name>. Delete the record after the test.

   • Execute the request using cURL depending on the protocol type:

     curl http://<service_domain_name> \
         --resolve <service_domain_name>:80:<L7_load_balancer_public_IP_address>
     

     curl https://<service_domain_name> \
         --resolve <service_domain_name>:443:<L7_load_balancer_public_IP_address>
     

Migrate user load from the network load balancer to the L7 load balancerMigrate user load from the network load balancer to the L7 load balancer

Select one of the migration options:

• Keep the public IP address for your service.
• Do not keep public IP address for your service.

Keep the public IP address for your serviceKeep the public IP address for your service

1. If your external network load balancer uses a dynamic public IP address, convert it to static.

2. Delete all listeners in the network load balancer to release the static public IP address. This will make your service unavailable through the network load balancer.

3. In the L7 load balancer, assign to the listener the public IP address previously assigned to the network load balancer:

   1. Open the YAML file that describes the Ingress resource.

   2. Under annotations, for the ingress.alb.yc.io/external-ipv4-address field, specify the public IP address previously assigned to the network load balancer.

   3. Apply changes using this command:

      kubectl apply -f <Ingress_resource_file>
      

4. Wait for the Ingress resource to finish changing its public IP address. You can view resource info using this command:

   kubectl get ingress <Ingress_resource_name> -w
   

   After the IP addresses changes, your service will again be available through the L7 load balancer.

5. Go to the L7 load balancer:

   1. In the management console, go to the folder the Managed Service for Kubernetes cluster is in.
   2. Select Managed Service for Kubernetes.
   3. Select the cluster you need.
   4. Select Network on the left, and the Ingress tab on the right. For your Ingress resource, follow the L7 load balancer link in the Load balancer column.
   5. Monitor the L7 load balancer's user load from the load balancer statistics charts.

6. Delete the released static public IP address previously reserved for the L7 load balancer.

7. (Optional) Delete the network load balancer after migrating user load to the L7 load balancer.

Do not keep the public IP address for your serviceDo not keep the public IP address for your service

1. To migrate user load from a network load balancer to an L7 load balancer, in the DNS service of your domain's public zone, change the A record value for the service domain name to the public IP address of the L7 load balancer. If the public domain zone was created in Yandex Cloud DNS, change the record using this guide.

   Note

   The propagation of DNS record updates depends on the time-to-live (TTL) value and the number of links in the DNS request chain. This process can take a long time.

2. As the DNS record updates propagate, follow the increase of requests to the L7 load balancer:

   1. In the management console, go to the folder the Managed Service for Kubernetes cluster is in.
   2. Select Managed Service for Kubernetes.
   3. Select the cluster you need.
   4. Select Network on the left, and the Ingress tab on the right. For your Ingress resource, follow the L7 load balancer link in the Load balancer column.
   5. Monitor the L7 load balancer's user load from the load balancer statistics charts.

3. Follow the decrease of the network load balancer load using the processed_bytes and processed_packets load balancer metrics. You can create a dashboard to visualize these metrics. The absence of load on the network load balancer for a prolonged period of time indicates that the user load has been transfered to the L7 load balancer.

4. (Optional) Delete the network load balancer after migrating user load to the L7 load balancer.