Migrating services from a NLB with target resources from an instance group to an L7 ALB

In this tutorial, you will learn how to integrate Yandex Network Load Balancer with a group of VMs. This VM group tracks the number of VMs in the group and changes the composition of the load balancer's target group as needed. A VM group can only integrate with one of the load balancers: either the network load balancer or the Yandex Application Load Balancer L7 load balancer. Therefore, when migrating, you need to change the integration with the target group for the VM group: replace the network load balancer target group with the L7 load balancer target group.

To migrate a service from a network load balancer to an L7 load balancer:

1. See recommendations for service migration.
2. Complete the prerequisite steps.
3. Create a Yandex Smart Web Security profile.
4. Create an L7 load balancer. At this step, you will connect your Smart Web Security profile to a virtual host of the L7 load balancer.
5. Migrate user load from the network load balancer to the L7 load balancer.

Service migration recommendationsService migration recommendations

1. In addition to DDoS protection at level L7 of the OSI model using Yandex Smart Web Security, we recommend enabling DDoS protection at L3-L4. To do this, reserve a public static IP address with DDoS protection in advance and use this address for the L7 load balancer's listener.

   If the network load balancer's listener already uses a public IP address with DDoS protection, you can save it and use it for the L7 load balancer.

   If the network load balancer's listener uses a public IP address without DDoS protection, the only way to enable L7 load balancer DDoS protection at level L3-L4 is to change the public IP address for your service.

   When using L3-L4 DDoS protection, configure a trigger threshold for the L3-L4 protection mechanisms aligned with the amount of legitimate traffic to the protected resource. To set up this threshold, contact support.

   Also, set the MTU value to 1450 for the target resources downstream of the load balancer. For more information, see Setting up MTU when enabling DDoS protection.

2. We recommend performing migration during the hours when user load is at its lowest. The migration process for a VM group changes the integration with the target group and migrates the public IP address from the network load balancer to the L7 load balancer. Your service will be unavailable during this period. The downtime depends on the number of VMs in the group, deployment policy settings and may take from several minutes to tens of minutes under normal conditions.

3. When using an L7 load balancer, requests to backends come with the source IP address from the range of internal IP addresses of the subnets specified when creating the L7 load balancer. The original IP address of the request source (user) is specified in the X-Forwarded-For header. If you want to log public IP addresses of users on the web server, reconfigure it.

4. See the autoscaling and resource units in the L7 load balancer.

Getting startedGetting started

1. Create subnets in three availability zones. These will be used for the L7 load balancer.

2. Create security groups that allow the L7 load balancer to receive incoming traffic and send it to the target resources, and allow the target resources to receive incoming traffic from the load balancer.

3. When using HTTPS, add your service's TLS certificate to Yandex Certificate Manager.

4. Reserve a static public IP address with DDoS protection at level L3-L4 for the L7 load balancer. See service migration recommendations.

Create a Smart Web Security security profileCreate a Smart Web Security security profile

Create a Smart Web Security security profile by selecting From a preset template.

Use these settings when creating the profile:

• In the Action for the default base rule field, select Allow.
• For the Smart Protection rule, enable Only logging (dry run).

These settings are limited to logging the info about the traffic without applying any actions to it. This will reduce the risk of disconnecting users due to profile configuration issues. As you move along, you will be able to turn Only logging (dry run) off and configure some prohibiting rules for your use case in the security profile.

Create an L7 load balancerCreate an L7 load balancer

1. Create a target group for the L7 load balancer. Under Targets, select the VMs in your network load balancer's target group.

2. Create a group of backends with the following parameters:

   1. Select HTTP as the backend group type.

   2. If your service requires requests to be processed within a single user session by the same backend resource, enable session affinity for the backend group.

   3. Under Backends, click Add and set up the backend:

      • Type: Target group.
      • Target groups: Target group you created earlier.
      • Port: Your service's TCP port the VMs are accepting incoming traffic on.
      • Under Protocol settings, specify the settings for connecting the L7 load balancer to the backend. Depending on the protocol type on your backend, select HTTP or HTTPS.
      • Under HTTP health check, configure the health check using these recommendations.
      • (Optional) Set other settings as per this guide.

3. Create an HTTP router. Under Virtual hosts, click Add virtual host and specify the virtual host settings:

   • Authority: Your service domain name.

   • Security profile: Smart Web Security profile you created earlier.

     Warning

     Linking your security profile to a virtual host of the L7 load balancer is the key step to connecting Smart Web Security.

   • Click Add route and specify the route settings:

     • Path: Starts with /.
     • Action: Routing.
     • Backend group: Backend group you created earlier.

4. Create an L7 load balancer by selecting Manual:

   • Specify the previously created security group.

   • Under Allocation, select the subnets in three availability zones for the load balancer nodes. Enable traffic in these subnets.

   • Under Autoscaling settings, specify the minimum number of resource units per availability zone based on expected load.

     We recommend selecting the number of resource units based on load expressed in:

     • Number of requests per second (RPS)
     • Number of concurrent active connections
     • Number of new connections per second
     • Traffic processed per second

   • Under Listeners, click Add listener and set up the listener:

     • Under Public IP address, specify:

       • Port: Your service's TCP port the VMs are accepting incoming traffic on.
       • Type: List. Select from the list a public IP address with DDoS protection at L3-L4. For more information, see service migration recommendations.

     • Under Receiving and processing traffic, specify:

       • Listener type: HTTP.
       • Protocol: Depending on your service, select HTTP or HTTPS.
       • If you select HTTPS, specify the TLS certificate you added to Certificate Manager earlier in the Certificates field.
       • HTTP router: HTTP router you created earlier.

5. Wait until the L7 load balancer goes Active.

6. Go to the new L7 load balancer and select Health checks on the left. Make sure you get HEALTHY for all the L7 load balancer's health checks.

7. Run a test request to the service through the L7 load balancer, for example, using one of these methods:

   • Add this record to the hosts file on your workstation: <L7_load_balancer_public_IP_address> <service_domain_name>. Delete the record after the test.

   • Execute the request using cURL depending on the protocol type:

     curl http://<service_domain_name> \
         --resolve <service_domain_name>:<service_port>:<public_IP_address_of_L7_load_balancer>
     

     curl https://<service_domain_name> \
         --resolve <service_domain_name>:<service_port>:<public_IP_address_of_L7_load_balancer>
     

Migrate user load from the network load balancer to the L7 load balancerMigrate user load from the network load balancer to the L7 load balancer

1. Change the integration with the target group for the VM group:

   1. In the management console, select the folder containing your VM group.
   2. Select Compute Cloud.
   3. In the left-hand panel, select Instance groups.
   4. Select the group to update.
   5. In the top-right corner of the page, click Edit.
   6. Under Integration with Application Load Balancer, enable Create target group.
   7. Specify the name of the L7 load balancer target group and, optionally, other target group settings.
   8. Click Save.

   When changing the VM group:

   • VMs in the group are automatically recreated.
   • Target resources are removed from the network balancer target group, and the user load is distributed among the remaining target resources. The service becomes partially unavailable to users through the network load balancer during this period.
   • After all target resources are deleted, the target group is deleted. The service becomes unavailable through the network load balancer.

   Proceed to the next step without waiting for the VM group change to end.

2. In the L7 load balancer backend group, change the backend target group. Specify only the target group created in the previous step.

   As you perform the operation specified in the previous step, VMs from the VM group will be automatically added to the L7 load balancer target group.

3. Select one of the options to further migrate the user load from the network load balancer to the L7 load balancer depending on whether the network load balancer listener has a public IP address with or without DDoS protection:

   • The network load balancer listener uses a public IP address with DDoS protection. During migration, the public IP address for your service will remain the same.
   • The network load balancer listener uses a public IP address without DDoS protection. During migration, the public IP address for your service will change.

The network load balancer listener uses a public IP address with DDoS protectionThe network load balancer listener uses a public IP address with DDoS protection

1. Monitor the status of the target resources of the network load balancer. Wait until the target resources are automatically deleted from the target group.

2. Delete the listener in the network load balancer to release the static public IP address.

3. In the L7 load balancer, assign to the listener the public IP address previously assigned to the network load balancer:

   CLI

   Terraform

   If you do not have the Yandex Cloud command line interface yet, install and initialize it.

   The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

   To change a public IP address, run this command:

   yc application-load-balancer load-balancer update-listener <load_balancer_name> \
      --listener-name <listener_name> \
      --external-ipv4-endpoint address=<service_public_IP_address>,port=<service_port>
   

   Where address is the public IP address previously assigned to the network load balancer.

   1. Open the current Terraform configuration file with an infrastructure plan.

      For how to create this file, see Creating an L7 load balancer.

      For more information about the yandex_alb_load_balancer resource parameters in Terraform, see the provider documentation.

   2. In the load balancer description, change the address parameter value under listener.endpoint.address.external_ipv4_address:

      resource "yandex_alb_load_balancer" "<load_balancer_name>" {
        ...
        listener {
          ...
          endpoint {
            address {
              external_ipv4_address {
                address = <service_public_IP_address>
              }
            }
            ports = [ <service_port> ]
          }
        }
      }
      

      Where address is the public IP address previously assigned to the network load balancer.

   3. Apply the changes:

      1. In the terminal, change to the folder where you edited the configuration file.

      2. Make sure the configuration file is correct using the command:

         terraform validate
         

         If the configuration is correct, the following message is returned:

         Success! The configuration is valid.
         

      3. Run the command:

         terraform plan
         

         The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.

      4. Apply the configuration changes:

         terraform apply
         

      5. Confirm the changes: type yes in the terminal and press Enter.

4. After the IP addresses changes, your service will again be available through the L7 load balancer. Monitor the L7 load balancer's user load from the load balancer statistics charts.

5. Delete the now free static public IP address you selected when creating the L7 load balancer.

6. (Optional) Delete the network load balancer after migrating user load to the L7 load balancer.

The network load balancer listener uses a public IP address without DDoS protectionThe network load balancer listener uses a public IP address without DDoS protection

1. Memorize the current settings of health checks for the target group in the network load balancer.

2. Monitor the status of the target resources of the network load balancer. Wait until the target resources are automatically deleted from the target group.

3. Create a target group for the network load balancer. Add the VMs created when changing the VM group.

4. In network load balancer, connect the target group created in the previous step. When connecting the target group, configure the health checks the original target group had.

5. Wait until the VM health checks in the network load balancer target group get the Healthy status. This will make your service once again available through the network load balancer.

6. To migrate user load from a network load balancer to an L7 load balancer, in the DNS service of your domain's public zone, change the A record value for the service domain name to the public IP address of the L7 load balancer. If the public domain zone was created in Yandex Cloud DNS, change the record using this guide.

   Note

   The propagation of DNS record updates depends on the time-to-live (TTL) value and the number of links in the DNS request chain. This process can take a long time.

7. As the DNS record updates propagate, follow the increase of requests to the L7 load balancer from the load balancer statistics charts.

8. Follow the decrease of the network load balancer load using the processed_bytes and processed_packets load balancer metrics. You can create a dashboard to visualize these metrics. The absence of load on the network load balancer for a prolonged period of time indicates that the user load has been transfered to the L7 load balancer.

9. (Optional) Delete the network load balancer after migrating user load to the L7 load balancer.