Contact UsGet started
© 2024 Iron Hive LLC Belgrade

Using Pandora to run a step-load HTTPS test

Written by
Updated at June 6, 2024

You can use Yandex Load Testing to run incremental HTTPS load tests of the service with the Pandora load generator.

To perform load testing:

  1. Prepare your cloud.
  2. Prepare a test target.
  3. Prepare the infrastructure.
  4. Create an agent.
  5. Prepare a file with test data.
  6. Run a test.

If you no longer need the resources you created, delete them.

Prepare your cloud

Sign up for Yandex Cloud and create a billing account:

  1. Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
  2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

If the agent is hosted on Yandex Cloud, a fee is charged for computing resources (see Yandex Compute Cloud pricing).

At the Preview stage, Load Testing can be used free of charge.

Prepare a test target

In this example, we'll test a service with the 172.17.0.10 internal IP address in the same subnet as the agent.

Make sure the service is accessed over HTTPS using the default port: 443.

You can also use Load Testing for loading testing of a service that is public or located in a subnet and security group other than those of the agent.

For a public service, allow incoming HTTPS traffic to port 443.

For a service whose subnet and security group differ from the agent's ones, create a rule for incoming HTTPS traffic to port 443 in the security group where the test target is located.

Prepare the infrastructure

Create a service account

  1. Create a service account named sa-loadtest in the folder to host the agent that will generate the load.
  2. Assign the following roles to the service account:
    • loadtesting.generatorClient: Allows running the agent, performing tests on the agent, and uploading the results to storage.
    • compute.admin: Allows managing VMs in Compute Cloud.
    • vpc.user: Allows connecting to Virtual Private Cloud network resources and using them.

Configure a network

Create and configure a NAT gateway in the subnet where your test target is and the agent will be hosted. This ensures the agent has access to Load Testing.

Configure security groups

  1. Set up the test agent's security group:

    1. Create an agent security group named agent-sg.
    2. Add rules:

      1. Rule for outgoing HTTPS traffic to the Load Testing public API:

        • Port range: 443
        • Protocol: TCP
        • Destination name: CIDR
        • CIDR blocks: 0.0.0.0/0

        This will allow connecting the agent to Load Testing to manage tests from the interface and get test results.

      2. Rule for incoming SSH traffic:

        • Port range: 22
        • Protocol: TCP
        • Destination name: CIDR
        • CIDR blocks: 0.0.0.0/0

        This will allow you to connect to the agent over SSH and manage tests from the console or collect debugging information.

      3. Rule for outgoing traffic when generating load to the test target:

        • Port range: 0-65535
        • Protocol: Any
        • Destination name: Security group
          Select From list. Specify the security group where the test target is located.

        Create this rule for each test target with a unique security group.

  2. Set up the test target's security group:

    1. Create the test target's security group named load-target-sg.

    2. Add a rule for incoming traffic when generating load to the test target:

      • Port range: 0-65535.
      • Protocol: Any.
      • Select From list. Specify the security group where the test target is located.

      This rule allows agents to generate load to this target or enable additional monitoring tools.

Create a test agent

  1. If you do not have an SSH key pair yet, create one.

  2. Create an agent.

    1. In the management console, select the folder where you want to create your agent.

    2. In the list of services, select Load Testing.

    3. In the Agents tab, click Create agent.

    4. Give your agent a name, e.g., agent-008.

    5. Specify the same availability zone where the test target is located.

    6. Under Agent:

    7. Under Access, specify the information required to access the agent:

      • Select the sa-loadtest service account.

      • Enter the username in the Login field.

        Alert

        Do not use the root username or other names reserved by the operating system. To perform operations that require superuser permissions, use the sudo command.

      • In the SSH key field, paste the contents of the public key file.

    8. Click Create.

    9. Wait for the VM instance to create. Make sure the agent status changes to Ready for test.

      Note

      The agent creation process may stop at the Initializing connection status unless the following conditions are met:

      • The agent has access to loadtesting.api.cloud.yandex.net:443 and is assigned a public IP address.
      • A NAT gateway is set up in the target subnet.
      • The service account assigned to the agent has the required roles.

  3. Assign a public IP to the agent to enable access over SSH:

    1. In the management console, select the folder where the agent is located.
    2. Select Compute Cloud.
    3. Select the VM named agent-008.
    4. Under Network interface, in the top-right corner, click and select Add public IP address.
    5. In the window that opens:
      • In the Public address field, select obtaining a Auto address.
      • Click Add.

Prepare a file with test data

  1. Generate payloads in URI format:

    [Host: 172.17.0.10]
[Connection: Close]
/ index
/test?param1=1&param2=2 get_test

    Please note that the Connection: Close header means each connection is terminated after making a request. This mode is heavier on the application and load generator. If you do not need to close connections, set Keep-Alive.

    There are also two requests tagged index and get_test. The load generator will repeat them within a given load profile.

  2. Save the payloads to a file named data.uri.

Run a test

  1. In the management console, select Load Testing.

  2. In the left-hand panel, select Tests. Click Create test.

  3. In the Agents parameter, select agent-008.

  4. Under Attached files, select From computer, click Attach file, and select the data.uri file.

  5. Under Test settings, select a configuration method: Form or Config.

  6. Depending on the selected method, specify the test parameters:

    1. In the Load generator field, select PANDORA.

    2. In the Target address field, specify the address of the service to test: 172.17.0.10.

    3. In the Target port field, set 443 (default HTTPS port).
      Allow using a secure connection.

    4. In the Testing threads field, specify 5000.

      This means that the load generator can simultaneously process 5000 operations: either create 5000 connections or wait for 5000 responses from the service at the same time.

      Tip

      For most tests, 1000–10000 threads are enough.

      Using a larger number of threads requires more resources of the VM the agent is running on. Compute Cloud also has a limit of 50000 of concurrent connections to a VM.

      Learn more about working with threads.

    5. In the Load type menu, select RPS.

    6. Click Load profile and enter the following description:

      • Profile 1: Select step.

      • From: Set 1000.

      • To: Set 5000.

      • Step: Set 1000.

      • Duration: Specify 120s.

        This is an instruction for the generator to increase the load from 1000 to 5000 requests per second for 120 seconds in increments of 1000 requests.

        Learn more about load profiles.

    7. In the Request type field, select URI.

    8. In the Autostop menu, click Autostop and enter the following description:

      • Autostop type 1: Select QUANTILE.

      • Quantile: Set 75.

      • Response time limit: Specify 100ms.

      • Window duration: Specify 10s.

        This criterion stops the test if the 75th percentile exceeds 100 milliseconds for 10 seconds (for 10 seconds, the processing time of 25% of queries exceeds 100 milliseconds).

    9. Specify one more autostop:

      • Autostop type 2: Select INSTANCES.

      • Limit: 90%.

      • Window duration: 60s.

        This criterion will stop the test if over 90% of the testing threads are busy for 60 seconds.

        As load increases, the system being tested will start to degrade at some point. Subsequent load increases will result in either an increased response time or an increased error rate.

        To avoid significantly increasing the test time, make sure to set Autostop as a termination criterion for these tests.

        Learn more about autostop.

    10. Under Test information, specify the name, description, and number of the test version. This will make the report easier to read.

    1. In the configuration input field, specify the testing thread settings in yaml format:

      pandora:
  enabled: true
  package: yandextank.plugins.Pandora
  config_content:
    pools:
      - id: HTTP
        gun:
          type: http # protocol
          target: 172.17.0.10:443 # address of testing objective
          ssl: true
        ammo:
          type: uri
          file: ammo_131b131b-131b-131b-131b-131b13
        result:
          type: phout
          destination: ./phout.log
        rps:
          - duration: 120s # test time
            type: step # load type
            from: 1000
            to: 5000
            step: 1000
        startup:
          type: once
          times: 5000 # number of threads
    log:
      level: error
    monitoring:
      expvar:
        enabled: true
        port: 1234
autostop: # autostop
  enabled: true
  package: yandextank.plugins.Autostop
  autostop:
    - quantile(75,100ms,10s) # stop test if 75th percentile exceeds 100 milliseconds for 10 seconds (for 10 seconds, processing time of 25% of queries exceeds 100 milliseconds).
    - instances(90%,60s) # stop test if 90% of testing threads are being used for 60 seconds.
core: {}
uploader:
  enabled: true
  package: yandextank.plugins.DataUploader
  job_name: '[example][pandora][step]'
  job_dsc: 'example'
  ver: '0.5.5'
  api_address: loadtesting.api.cloud.yandex.net:443

      As load increases, the system being tested will start to degrade at some point. Subsequent load increases will result in either an increased response time or an increased error rate.

      To avoid significantly increasing the test time, make sure to set Autostop as a termination criterion for these tests.

      Learn more about autostop.

      Tip

      View a sample configuration file. You can also find sample configuration files in existing tests.

  7. Click Create.

Afterwards, the configuration will be verified, and the agent will start loading the service being tested.

To see the testing progress, select the created test and go to the Test results tab.

How to delete the resources you created

To stop paying for the resources created, just delete the agent.

Previous
Fixed-load HTTPS testing with Phantom
Next
Load testing using multiple agents