Migrating services from an NLB with a Yandex Managed Service for Kubernetes cluster as a target to an L7 ALB using Terraform

To migrate a service from a network load balancer to an L7 load balancer:

1. See the service migration recommendations.
2. Create your infrastructure. At this step, you will associate the Smart Web Security profile with a virtual host of the L7 load balancer.
3. Install an Application Load Balancer ingress controller and create resources in your Managed Service for Kubernetes cluster. At this step, you will associate your Smart Web Security profile with the L7 load balancer.
4. Test the L7 load balancer.
5. Migrate user traffic from the network load balancer to the L7 load balancer.

Service migration recommendationsService migration recommendations

1. In addition to DDoS protection at OSI L7 using Yandex Smart Web Security, we recommend enabling DDoS protection at L3-L4. To do this, reserve a public static IP address with DDoS protection in advance and use this address for the L7 load balancer's listener.

   If the network load balancer's listener already uses a public IP address with DDoS protection, you can save it and use it for the L7 load balancer.

   If the network load balancer's listener uses a public IP address without DDoS protection, DDoS protection at L3-L4 when migrating to an L7 load balancer can only be achieved by changing the public IP for your service.

   When using L3-L4 DDoS protection, configure a trigger threshold for the L3-L4 protection mechanisms aligned with the amount of legitimate traffic to the protected resource. To set up this threshold, contact support.

   Also, set the MTU value to 1450 for the targets downstream of the load balancer. For more information, see Setting up MTU when enabling DDoS protection.

2. We recommend performing migration during the hours when the user load is at its lowest. If you plan to keep your public IP address, bear in mind that migration involves moving this IP address from the load balancer to the L7 load balancer. Your service will be unavailable during this period. Under normal conditions, this may last for several minutes.

3. When using an L7 load balancer, requests to backends come with the source IP address from the range of internal IP addresses of the subnets specified when creating the L7 load balancer. The original IP address of the request source (user) is specified in the X-Forwarded-For header. If you want to log public IP addresses of users on the web server, reconfigure it.

4. For the L7 load balancer, two resource units will be created in each of the subnets specified when creating the Ingress resource. The Ingress resource annotations do not support specifying the minimum number of resource units for an L7 load balancer. The system automatically scales the resource unit group based on the load balancer node’s external workload.

5. The features of the Application Load Balancer load balancer may differ from those of your load balancer deployed in the Managed Service for Kubernetes cluster. See Application Load Balancer ingress controller description and operating principles.

6. We recommend setting up backend health checks on your Application Load Balancer. Health checks enable the load balancer to timely spot unavailable backends and reroute traffic to alternative backends. Once the application is updated, traffic will again be distributed across all backends.

   For more information, see Tips for configuring Yandex Application Load Balancer health checks and Annotations (metadata.annotations).

Create your infrastructureCreate your infrastructure

1. If you do not have Terraform yet, install it.

2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

5. Download the configuration file to the same working directory based on the protocol you are using:

   • HTTP: alb-k8s-http.tf configuration file
   • HTTPS: alb-k8s-https.tf configuration file

   These files describe:

   • Subnets for the L7 load balancer.
   • Security group for the L7 load balancer.
   • Static address for the L7 load balancer.
   • Importing a TLS certificate to Certificate Manager (if using HTTPS).
   • Smart Web Security profile.

6. Specify the following variables in the configuration file:

   • domain_name: Your service domain name.
   • network_id: ID of the network containing the VMs from the network load balancer's target group.
   • certificate (for HTTPS): Path to the self-signed custom certificate.
   • private_key (for HTTPS): Path to the private key file.

7. Make sure the Terraform configuration files are correct using this command:

   terraform validate
   

   Terraform will show any errors found in your configuration files.

8. Create the required infrastructure:

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

   All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

Install an Application Load Balancer ingress controller and create resources in your Managed Service for Kubernetes clusterInstall an Application Load Balancer ingress controller and create resources in your Managed Service for Kubernetes cluster

1. Install the Yandex Application Load Balancer ingress controller.

2. Create an IngressClass resource for the L7 load balancer's Ingress controller:

   1. Create a YAML file and describe the IngressClass resource in it.

      IngressClass resource example:

      apiVersion: networking.k8s.io/v1
      kind: IngressClass
      metadata:
        labels:
          app.kubernetes.io/component: controller
        name: ingress-alb
      spec:
        controller: ingress.alb.yc.io/yc-alb-ingress-controller
      

   2. Use the following command to create the IngressClass resource:

      kubectl apply -f <IngressClass_resource_file>
      

3. Create an Ingress resource:

   1. Read the descriptions of the Ingress resource fields and annotations and see the example.

   2. Create a YAML file and describe the Ingress resource in it:

      • Complete the annotations section for the L7 load balancer settings:

        • ingress.alb.yc.io/subnets: IDs of the subnets in the three availability zones for the L7 load balancer nodes. Specify the IDs separated by commas with no spaces.

        • ingress.alb.yc.io/security-groups: ID of one or more security groups for the L7 load balancer. For multiple groups, specify their IDs separated by commas with no spaces.

        • ingress.alb.yc.io/external-ipv4-address: Previously reserved static public IP address.

        • ingress.alb.yc.io/group-name: Name of the Ingress resource group. Ingress resources are grouped together, each group served by a separate Application Load Balancer instance with a dedicated public IP address.

        • ingress.alb.yc.io/security-profile-id: ID of the previously created Smart Web Security security profile.

          Warning

          The security profile will be linked to the virtual host of the L7 load balancer. Specifying your security profile is the key step to connecting Smart Web Security.

        • ingress.alb.yc.io/autoscale-min-zone-size: Minimum number of resource units per availability zone, based on expected load.

          We recommend selecting the number of resource units based on the load expressed in:

          • Number of requests per second (RPS)
          • Number of concurrent active connections
          • Number of new connections per second
          • Traffic processed per second

      • For the ingressClassName field, enter the name of the IngressClass resource you created earlier.

      • When using HTTPS, complete the tls section:

        • hosts: Your service domain name the TLS certificate corresponds to.
        • secretName: TLS certificate of your service in Yandex Certificate Manager, in yc-certmgr-cert-id-<certificate_ID> format.

      • Complete the rules section in line with your rules for distribution of incoming traffic among backends depending on the domain name (host field) and requested resource (http.paths field).

        • host: Your service domain name.

        • pathType: Type of reference to the requested resource:

          • Exact: Request URI path must match the path field value.
          • Prefix: Request URI path must start with the path field value.

        • path: Incoming request URI path (if Exact) or its prefix (if Prefix).

        • backend: Reference to a backend or group of backends to process the requests with the specified domain name and URI path. Specify either a service backend (service) or a backend group (resource) but not both.

          • service: Managed Service for Kubernetes backend service for processing requests:

            • name: Managed Service for Kubernetes service name. The Service resource this field refers to must be described in line with this configuration.
            • port: Service port Ingress is going to address. For the service port, specify either a number (number) or a name (name) but not both.

            Warning

            The Managed Service for Kubernetes services used as backends must be of the NodePort type.

          • resource: Reference to the HttpBackendGroup group of backends to process the requests. A backend group can route traffic to either Managed Service for Kubernetes services or Yandex Object Storage buckets. When using a backend group, advanced Application Load Balancer functionality is available. You can also specify relative backend weights to allocate traffic to them in proportion.

            • kind: HttpBackendGroup
            • name: Backend group name. The name must match the value specified in the metadata.name field of the HttpBackendGroup resource. The HttpBackendGroup resource this field refers to must be described in line with this configuration.
            • apiGroup: alb.yc.io

      Ingress resource example:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: <resource_name>
        annotations:
          ingress.alb.yc.io/subnets: <IDs_of_subnets_in_three_availability_zones>
          ingress.alb.yc.io/security-groups: <L7_load_balancer_security_group_ID>
          ingress.alb.yc.io/external-ipv4-address: <static_public_IP_address>
          ingress.alb.yc.io/group-name: <resource_group_name>
          ingress.alb.yc.io/security-profile-id: <Smart_Web_Security_security_profile_ID>
          ingress.alb.yc.io/autoscale-min-zone-size: <minimum_number_of_L7_load_balancer_resource_units_per_zone>
      spec:
        ingressClassName: <IngressClass_resource_name>
        tls:
          - hosts:
              - <service_domain_name>
            secretName: yc-certmgr-cert-id-<certificate_ID>
        rules:
          - host: <service_domain_name>
            http:
              paths:
              - path: /
                pathType: Prefix
                backend:
                  service:
                    name: <Kubernetes_service_name>
                    port:
                      number: <443_or_another_port_number>
      

   3. Use the following command to create the Ingress resource:

      kubectl apply -f <Ingress_resource_file>
      

4. An L7 load balancer will be deployed based on the Ingress resource configuration. Wait until its creation is complete and Ingress has a public IP address linked. You will need this IP address to check requests. You can view resource info using this command:

   kubectl get ingress <Ingress_resource_name> -w
   

Test the L7 load balancerTest the L7 load balancer

Run a test request to the service through the L7 load balancer, for example, using one of these methods:

• Add this record to the hosts file on your workstation: <L7_load_balancer_public_IP_address> <service_domain_name>. Delete the record after the test.

• Execute the request using cURL depending on the protocol type:

  curl http://<service_domain_name> \
      --resolve <service_domain_name>:<service_port>:<public_IP_address_of_L7_load_balancer>
  

  curl https://<service_domain_name> \
      --resolve <service_domain_name>:<service_port>:<public_IP_address_of_L7_load_balancer>
  

Migrate user traffic from the network load balancer to the L7 load balancerMigrate user traffic from the network load balancer to the L7 load balancer

Select one of these migration options:

• Keep the public IP address for your service.
• Do not keep the public IP address for your service.

Keep the public IP address for your serviceKeep the public IP address for your service

1. If your external network load balancer uses a dynamic public IP address, convert it to a static one.

2. Delete all listeners in the network load balancer to release the static public IP address. This will make your service unavailable through the network load balancer.

3. In the L7 load balancer, assign to the listener the public IP address previously used by the network load balancer:

   1. Open the YAML file that describes the Ingress resource.

   2. Under annotations, in the ingress.alb.yc.io/external-ipv4-address field, specify the public IP address previously assigned to the network load balancer.

   3. Apply the changes using this command:

      kubectl apply -f <Ingress_resource_file>
      

4. Wait for the Ingress resource to finish switching its public IP address. You can view resource information using this command:

   kubectl get ingress <Ingress_resource_name> -w
   

   After the IP address changes, your service will again be available through the L7 load balancer.

5. Navigate to the L7 load balancer:

   1. In the management console, navigate to the folder with the Managed Service for Kubernetes cluster.
   2. Select Managed Service for Kubernetes.
   3. Select the cluster.
   4. Select Network on the left and then the Ingress tab on the right. For your Ingress resource, follow the L7 load balancer link in the Load balancer column.
   5. Monitor the L7 load balancer's user traffic on the load balancer statistics charts.

6. Delete the released static public IP address previously reserved for the L7 load balancer.

   1. Open the configuration file you used to create the L7 load balancer (alb-k8s-http.tf or alb-k8s-https.tf).

   2. Delete the yandex_vpc_address resource description from the file:

      resource "yandex_vpc_address" "static-address" {
        description = "Static public IP address for the Application Load Balancer"
        name        = "alb-static-address"
        external_ipv4_address {
          zone_id                  = "ru-central1-a"
          ddos_protection_provider = "qrator"
        }
      }
      

   3. Apply the changes:

      1. In the terminal, go to the directory where you edited the configuration file.

      2. Make sure the configuration file is correct using this command:

         terraform validate
         

         If the configuration is correct, you will get this message:

         Success! The configuration is valid.
         

      3. Run this command:

         terraform plan
         

         You will see a detailed list of resources. No changes will be made at this step. If the configuration contains any errors, Terraform will show them.

      4. Apply the changes:

         terraform apply
         

      5. Type yes and press Enter to confirm the changes.

7. Optionally, delete the network load balancer after migrating user traffic to the L7 load balancer.

Do not keep the public IP address for your serviceDo not keep the public IP address for your service

1. To migrate user traffic from a network load balancer to an L7 load balancer, in the DNS service of your domain's public zone, update the A record value for the service domain name to point to the public IP address of the L7 load balancer. If the public domain zone was created in Yandex Cloud DNS, update the record using this guide.

   Note

   The propagation of DNS record updates depends on the time-to-live (TTL) value and the number of links in the DNS request chain. This process can take a while.

2. As the DNS record updates propagate, monitor the increase in requests to the L7 load balancer:

   1. In the management console, navigate to the folder with the Managed Service for Kubernetes cluster.
   2. Select Managed Service for Kubernetes.
   3. Select the cluster.
   4. Select Network on the left and then the Ingress tab on the right. For your Ingress resource, follow the L7 load balancer link in the Load balancer column.
   5. Monitor the L7 load balancer's user traffic on the load balancer statistics charts.

3. Monitor the decrease in traffic on the network load balancer using the processed_bytes and processed_packets load balancer metrics. You can create a dashboard to visualize these metrics. No traffic on the external network load balancer over time indicates the L7 load balancer is now handling all user traffic.

4. Optionally, delete the network load balancer after migrating user traffic to the L7 load balancer.