Migrating services from an NLB load balancer with a Yandex Managed Service for Kubernetes cluster as a target to an L7 ALB load balancer using Terraform

To migrate a service from a network load balancer to an L7 load balancer:

1. See recommendations for service migration.
2. Create your infrastructure. At this step, you will connect your Smart Web Security profile to a virtual host of the L7 load balancer.
3. Install an Application Load Balancer Ingress controller and create resources in your Managed Service for Kubernetes cluster. At this step, you will connect your Smart Web Security profile to the L7 load balancer.
4. Test the L7 load balancer.
5. Migrate the user load from the network load balancer to the L7 load balancer.

Service migration recommendationsService migration recommendations

1. In addition to DDoS protection at OSI L7 using Yandex Smart Web Security, we recommend enabling DDoS protection at L3-L4. To do this, reserve a public static IP address with DDoS protection in advance and use this address for the L7 load balancer's listener.

   If the network load balancer's listener already uses a public IP address with DDoS protection, you can save it and use it for the L7 load balancer.

   If the network load balancer's listener uses a public IP address without DDoS protection, DDoS protection at L3-L4 when migrating to an L7 load balancer can only be achieved by changing the public IP for your service.

   When using L3-L4 DDoS protection, configure a trigger threshold for the L3-L4 protection mechanisms aligned with the amount of legitimate traffic to the protected resource. To set up this threshold, contact support.

   Also, set the MTU value to 1450 for the targets downstream of the load balancer. For more information, see Setting up MTU when enabling DDoS protection.

2. We recommend performing migration during the hours when the user load is at its lowest. If you plan to keep your public IP address, bear in mind that migration involves moving this IP address from the load balancer to the L7 load balancer. Your service will be unavailable during this period. Under normal conditions, this may last for several minutes.

3. When using an L7 load balancer, requests to backends come with the source IP address from the range of internal IP addresses of the subnets specified when creating the L7 load balancer. The original IP address of the request source (user) is specified in the X-Forwarded-For header. If you want to log public IP addresses of users on the web server, reconfigure it.

4. For the L7 load balancer, two resource units will be created in each of the subnets specified when creating the Ingress resource. The Ingress resource annotations do not support specifying the minimum number of resource units for an L7 load balancer. A group of resource units is automatically scaled depending on the external load on load balancer nodes.

5. The features of the Application Load Balancer load balancer may differ from those of your load balancer deployed in the Managed Service for Kubernetes cluster. See the Application Load Balancer Ingress controller description and operating principles.

Create an infrastructureCreate an infrastructure

1. If you do not have Terraform yet, install it.

2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

5. Download a configuration file to the same working directory based on the protocol you are using:

   • HTTP: alb-k8s-http.tf configuration file.
   • HTTPS: alb-k8s-https.tf configuration file.

   These files describe:

   • Subnets for the L7 load balancer.
   • Security group for the L7 load balancer.
   • Static address for the L7 load balancer.
   • Importing a TLS certificate to Certificate Manager (if HTTPS is used).
   • Smart Web Security security profile.

6. Specify the following variables in the configuration file:

   • domain_name: Your service domain name.
   • network_id: ID of the network containing the VMs from the network load balancer's target group.
   • certificate (for HTTPS): Path to the self-signed custom certificate.
   • private_key (for HTTPS): Path to the private key file.

7. Make sure the Terraform configuration files are correct using this command:

   terraform validate
   

   If there are any errors in the configuration files, Terraform will point them out.

8. Create the required infrastructure:

   1. Run the command to view planned changes:

      terraform plan
      

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

   2. If you are happy with the planned changes, apply them:

      1. Run the command:

         terraform apply
         

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

   All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

Install an Application Load Balancer Ingress controller and create resources in your Managed Service for Kubernetes clusterInstall an Application Load Balancer Ingress controller and create resources in your Managed Service for Kubernetes cluster

1. Install the Yandex Application Load Balancer Ingress controller.

2. Create an IngressClass resource for the L7 load balancer's Ingress controller:

   1. Create a YAML file and describe the IngressClass resource in it.

      IngressClass resource example:

      apiVersion: networking.k8s.io/v1
      kind: IngressClass
      metadata:
        labels:
          app.kubernetes.io/component: controller
        name: ingress-alb
      spec:
        controller: ingress.alb.yc.io/yc-alb-ingress-controller
      

   2. Use the following command to create the IngressClass resource:

      kubectl apply -f <IngressClass_resource_file>
      

3. Create an Ingress resource:

   1. Read the descriptions of the Ingress resource fields and annotations and see the example.

   2. Create a YAML file and describe the Ingress resource in it:

      • Complete the annotations section for the L7 load balancer settings:

        • ingress.alb.yc.io/subnets: IDs of the subnets in the three availability zones for the L7 load balancer nodes. Specify the IDs separated by commas with no spaces.

        • ingress.alb.yc.io/security-groups: ID of one or more security groups for the L7 load balancer. For multiple groups, specify their IDs separated by commas with no spaces.

        • ingress.alb.yc.io/external-ipv4-address: Previously reserved static public IP address.

        • ingress.alb.yc.io/group-name: Name of the Ingress resource group. Ingress resources are grouped together, each group served by a separate Application Load Balancer instance with a dedicated public IP address.

        • ingress.alb.yc.io/security-profile-id: ID of the previously created Smart Web Security security profile.

          Warning

          The security profile will be linked to the virtual host of the L7 load balancer. Specifying your security profile is the key step to connecting Smart Web Security.

        • ingress.alb.yc.io/autoscale-min-zone-size: Minimum number of resource units per availability zone, based on expected load.

          We recommend selecting the number of resource units based on the load expressed in:

          • Number of requests per second (RPS)
          • Number of concurrent active connections
          • Number of new connections per second
          • Traffic processed per second

      • For the ingressClassName field, enter the name of the IngressClass resource you created earlier.

      • When using HTTPS, complete the tls section:

        • hosts: Your service domain name the TLS certificate corresponds to.
        • secretName: TLS certificate of your service in Yandex Certificate Manager, in yc-certmgr-cert-id-<certificate_ID> format.

      • Complete the rules section in line with your rules for distribution of incoming traffic among backends depending on the domain name (host field) and requested resource (http.paths field).

        • host: Your service domain name.

        • pathType: Type of reference to the requested resource:

          • Exact: Request URI path must match the path field value.
          • Prefix: Request URI path must start with the path field value.

        • path: Incoming request URI path (if Exact) or its prefix (if Prefix).

        • backend: Reference to a backend or group of backends to process the requests with the specified domain name and URI path. Specify either a service backend (service) or a backend group (resource) but not both.

          • service: Managed Service for Kubernetes service to process the requests as a backend:

            • name: Managed Service for Kubernetes service name. The Service resource this field refers to must be described in line with this configuration.
            • port: Service port Ingress is going to address. For the service port, specify either a number (number) or a name (name) but not both.

            Warning

            The Managed Service for Kubernetes services used as backends must be of the NodePort type.

          • resource: Reference to the HttpBackendGroup group of backends to process the requests. A group like this may have Managed Service for Kubernetes services or Yandex Object Storage buckets as backends. When using a backend group, advanced Application Load Balancer functionality is available. You can also specify relative backend weights to allocate traffic to them in proportion.

            • kind: HttpBackendGroup
            • name: Backend group name. The name must match the value specified in the metadata.name field of the HttpBackendGroup resource. The HttpBackendGroup resource this field refers to must be described in line with this configuration.
            • apiGroup: alb.yc.io

      Ingress resource example:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: <resource_name>
        annotations:
          ingress.alb.yc.io/subnets: <ru-central1-a_subnet_ID,ru-central1-b_subnet_ID,ru-central1-d_subnet_ID>
          ingress.alb.yc.io/security-groups: <L7_load_balancer_security_group_ID>
          ingress.alb.yc.io/external-ipv4-address: <static_public_IP_address>
          ingress.alb.yc.io/group-name: <resource_group_name>
          ingress.alb.yc.io/security-profile-id: <Smart_Web_Security_security_profile_ID>
          ingress.alb.yc.io/autoscale-min-zone-size: <minimum_number_of_L7_load_balancer_resource_units_per_zone>
      spec:
        ingressClassName: <IngressClass_resource_name>
        tls:
          - hosts:
              - <service_domain_name>
            secretName: yc-certmgr-cert-id-<certificate_ID>
        rules:
          - host: <service_domain_name>
            http:
              paths:
              - path: /
                pathType: Prefix
                backend:
                  service:
                    name: <Kubernetes_service_name>
                    port:
                      number: <port_number_e.g._443>
      

   3. Use the following command to create the Ingress resource:

      kubectl apply -f <Ingress_resource_file>
      

4. An L7 load balancer will be deployed based on the Ingress resource configuration. Wait until its creation is complete and Ingress has a public IP address linked. You will need this IP address to check requests. You can view resource info using this command:

   kubectl get ingress <Ingress_resource_name> -w
   

Test the L7 load balancerTest the L7 load balancer

Run a test request to the service through the L7 load balancer, for example, using one of these methods:

• Add this record to the hosts file on your workstation: <L7_load_balancer_public_IP_address> <service_domain_name>. Delete the record after the test.

• Execute the request using cURL depending on the protocol type:

  curl http://<service_domain_name> \
      --resolve <service_domain_name>:<service_port>:<public_IP_address_of_L7_load_balancer>
  

  curl https://<service_domain_name> \
      --resolve <service_domain_name>:<service_port>:<public_IP_address_of_L7_load_balancer>
  

Migrate the user load from the network load balancer to the L7 load balancerMigrate the user load from the network load balancer to the L7 load balancer

Select one of the migration options:

• Keep the public IP address for your service.
• Do not keep public IP address for your service.

Keep the public IP address for your serviceKeep the public IP address for your service

1. If your external network load balancer uses a dynamic public IP address, convert it to static.

2. Delete all listeners in the network load balancer to release the static public IP address. This will make your service unavailable through the network load balancer.

3. In the L7 load balancer, assign to the listener the public IP address previously used by the network load balancer:

   1. Open the YAML file that describes the Ingress resource.

   2. Under annotations, for the ingress.alb.yc.io/external-ipv4-address field, specify the public IP address previously assigned to the network load balancer.

   3. Apply changes using this command:

      kubectl apply -f <Ingress_resource_file>
      

4. Wait for the Ingress resource to finish changing its public IP address. You can view resource information using this command:

   kubectl get ingress <Ingress_resource_name> -w
   

   After the IP address changes, your service will again be available through the L7 load balancer.

5. Go to the L7 load balancer:

   1. In the management console, go to the folder the Managed Service for Kubernetes cluster is in.
   2. Select Managed Service for Kubernetes.
   3. Select the cluster.
   4. Select Network on the left, and the Ingress tab on the right. For your Ingress resource, follow the L7 load balancer link in the Load balancer column.
   5. Monitor the L7 load balancer's user load from the load balancer statistics charts.

6. Delete the released static public IP address previously reserved for the L7 load balancer.

   1. Open the configuration file you used to create the L7 load balancer (alb-k8s-http.tf or alb-k8s-https.tf).

   2. Delete the yandex_vpc_address resource description from the file:

      resource "yandex_vpc_address" "static-address" {
        description = "Static public IP address for the Application Load Balancer"
        name        = "alb-static-address"
        external_ipv4_address {
          zone_id                  = "ru-central1-a"
          ddos_protection_provider = "qrator"
        }
      }
      

   3. Apply the changes:

      1. In the terminal, change to the folder where you edited the configuration file.

      2. Make sure the configuration file is correct using the command:

         terraform validate
         

         If the configuration is correct, the following message is returned:

         Success! The configuration is valid.
         

      3. Run the command:

         terraform plan
         

         The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.

      4. Apply the configuration changes:

         terraform apply
         

      5. Confirm the changes: type yes in the terminal and press Enter.

7. (Optional) Delete the network load balancer after migrating the user load to the L7 load balancer.

Do not keep the public IP address for your serviceDo not keep the public IP address for your service

1. To migrate user load from a network load balancer to an L7 load balancer, in the DNS service of your domain's public zone, change the A record value for the service domain name to the public IP address of the L7 load balancer. If the public domain zone was created in Yandex Cloud DNS, change the record using this guide.

   Note

   The propagation of DNS record updates depends on the time-to-live (TTL) value and the number of links in the DNS request chain. This process can take a long time.

2. As the DNS record updates propagate, monitor the increase of requests coming to the L7 load balancer:

   1. In the management console, go to the folder the Managed Service for Kubernetes cluster is in.
   2. Select Managed Service for Kubernetes.
   3. Select the cluster.
   4. Select Network on the left, and the Ingress tab on the right. For your Ingress resource, follow the L7 load balancer link in the Load balancer column.
   5. Monitor the L7 load balancer's user load from the load balancer statistics charts.

3. You can monitor the decrease of the network load balancer load using the processed_bytes and processed_packets load balancer metrics. You can create a dashboard to visualize these metrics. The absence of load on the network load balancer for a prolonged period of time indicates that the user load has been transferred to the L7 load balancer.

4. (Optional) Delete the network load balancer after migrating the user load to the L7 load balancer.