Creating test VMs via GitLab CI

Written by

Updated at February 27, 2025

Get your cloud ready
- Required paid resources
Create the test application VM
Configure the test application VM
Check whether the test application is working
Create a VM disk snapshot
Create a GitLab VM
Configure GitLab
Configure your runner
Configure CI
Test the application on the CI VM
- Delete the resources you created

You can use Yandex Cloud to automate routine tasks, such as running a specific script after each Git master branch commit. In the following example, we create and test a VM after each commit.

To configure continuous integration (CI) for VM disk snapshots:

Create the test application VM: Create a virtual machine allowing you to make new CI VMs using its disk snapshot.
Configure the test application VM: Install a web server and test application components on the VM. Write a test application reversing words in a text sent to the server.
Check how the application works: Send a test request to check whether the server settings are correct.
Create a VM disk snapshot: Create a disk snapshot that CI will use to create new VMs.
Create a GitLab VM: Create a GitLab VM with a repository to store CI settings and a test script.
Configure GitLab: Create a file repository and get configuration parameters.
Configure Runner: Tool for performing tasks.
Configure CI: Specify the required command and testing parameters.
Make sure the application runs on the CI VM: Check whether the system created CI snapshot VMs and they run your test application.

If you no longer need the VMs you created, delete them.

Get your cloud ready

Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Before creating a VM:

Navigate to the Yandex Cloud management console and select the folder to work with.
Make sure the selected folder has a network with a subnet where you can connect your VM. To do this, select Virtual Private Cloud on the folder page. If the list contains a network, click its name to see the list of subnets. If there are no networks in the list or the selected network has no subnets, create them as required.

Required paid resources

The infrastructure support costs include:

Fee for continuously running VMs (see Yandex Compute Cloud pricing).
Fee for storing created images (see Compute Cloud pricing).
Fee for a dynamic public IP address (see Yandex Virtual Private Cloud pricing).

Create the test application VM

Create a VM where you will install a web server, your test application, and its components:

In the management console, select the folder where you want to create your VM.
In the list of services, select Compute Cloud.
In the left-hand panel, select Virtual machines.
Click Create virtual machine.
Under Boot disk image, select the Ubuntu 18.04 public image.
Under Location, select an availability zone where your VM will reside.
Under Computing resources, navigate to the Custom tab and specify these parameters:
- Platform: Intel Ice Lake
- vCPU: 2
- Guaranteed vCPU performance: 20%
- RAM: 1 GB
In Network settings, select the subnet where you will connect your VM.
Under Access, select SSH key and specify the VM access credentials:
- In the Login field, specify the VM user name, e.g., yc-user.
  
  Alert
  
  Do not use root or other reserved usernames. To perform operations requiring root privileges, use the sudo command.
- In the SSH key field, select the SSH key saved in your organization user profile.
  
  If there are no saved SSH keys in your profile, or you want to add a new key:
  - Click Add key.
  - Enter a name for the SSH key.
  - Upload or paste the contents of the public key file. You need to create a key pair for the SSH connection to a VM yourself.
  - Click Add.
  The SSH key will be added to your organization user profile.
  
  If users cannot add SSH keys to their profiles in the organization, the added public SSH key will only be saved to the user profile of the VM being created.
Under General information, specify the VM name: ci-tutorial-test-app.
Click Create VM.

It may take a few minutes to create your VM. When its status changes to RUNNING, you can start configuring it.

Once your VM is created, the system will assign it an IP address and a host name (FQDN). You can use them for SSH access.

Configure the test application VM

On the new VM, install a web server and test application components. You will use Python 2 to create your test application.

In the management console, under Network on the VM page, find the VM public IP address.
Connect to the VM over SSH. You can do it using ssh in Linux or macOS or PuTTY in Windows.
```
ssh <login>@<VM_public_IP_address>
```
Run the apt update command to update the lists of packages available for installation.
Install the required packages: jq JSON processor, Git client, PIP package manager, virtualenv virtual environment management system, Python C API header files, and Nginx web server:
```
sudo apt-get --yes install jq git python-pip virtualenv python-dev nginx-full
```

Create your application directory and make yourself its owner:

sudo mkdir /srv/test-app
sudo chown -R $USER /srv/test-app

Navigate to this directory and create the virtualenv virtual environment in it:
```
cd /srv/test-app
virtualenv test-venv
```
Activate the virtual environment:
```
. test-venv/bin/activate
```
Install the Flask framework and the uWSGI web server in the virtual environment:
```
pip install flask uwsgi
```
Deactivate the virtual environment:
```
deactivate
```
Create the api.py file in the /srv/test-app directory:
```
touch api.py
```

Open the api.py file in any text editor and paste the Python code below into it; the code does the following:

Accepts a text string from the text input argument.
Reverses each word in the string while retaining spaces.
Returns a response:
- In JSON format if the client accepts it.
- In plain text if the client does not accept JSON.

# api.py
import json
from flask import Flask, request, make_response as response

app = Flask(__name__)

@app.route("/")
def index():
    text = request.args.get('text')
    json_type = 'application/json'
    json_accepted = json_type in request.headers.get('Accept', '')
    if text:
        words = text.split()
        reversed_words = [word[::-1] for word in words]
        if json_accepted:
            res = response(json.dumps({'text': reversed_words}), 200)
        else:
            res = response(' '.join(reversed_words), 200)
    else:
        res = response('text not found', 501)
    res.headers['Content-Type'] = json_type if json_accepted else 'text/plain'
    return res

Create the wsgi.py file in the /srv/test-app directory:
```
touch wsgi.py
```
Open the wsgi.py file in any text editor and paste the application starting code in it:
```
# wsgi.py
from api import app

if __name__ == "__main__":
    app.run()
```
Create the test-app.ini file in the /srv/test-app directory:
```
touch test-app.ini
```

Open the test-app.ini file in any text editor and paste the uWSGI server configuration in it:

#test-app.ini
[uwsgi]
module = wsgi:app

master = true
processes = 1

socket = test-app.sock
chmod-socket = 660
vacuum = true

die-on-term = true

Make the www-data user owner of the /srv/test-app directory and its contents:
```
sudo chown -R www-data:www-data /srv/test-app
```

Prepare the service that will start your uWSGI server. To do this, edit the /etc/systemd/system/test-app.service file as follows:

#/etc/systemd/system/test-app.service
[Unit]
Description=uWSGI instance to serve test API
After=network.target

[Service]
User=www-data
Group=www-data
WorkingDirectory=/srv/test-app
Environment="PATH=/srv/test-app/test-venv/bin"
ExecStart=/srv/test-app/test-venv/bin/uwsgi --ini test-app.ini

[Install]
WantedBy=multi-user.target

Specify the new virtual server settings in the Nginx configuration by changing the /etc/nginx/sites-available/test-app.conf file as follows:

#/etc/nginx/sites-available/test-app.conf
server {
    #server_name test-app.yandex www.test-app.yandex;

    listen 80;

    location /test/ {
        include uwsgi_params;
        uwsgi_param SCRIPT_NAME /test;
        uwsgi_modifier1 30;
        uwsgi_pass unix:/srv/test-app/test-app.sock;
    }
}

Create a symbolic link pointing to the test-app.conf Nginx configuration file:

sudo ln -s /etc/nginx/sites-available/test-app.conf /etc/nginx/sites-enabled/

Delete the symbolic link pointing to the default Nginx configuration:
```
sudo unlink /etc/nginx/sites-enabled/default
```
Configure the service to run at boot:
```
sudo systemctl enable test-app.service
```

Check whether the test application is working

To make sure the web server is properly configured and the test application is working as expected, make a simple request.

In the browser address bar, enter the following URL:

http://<VM_public_IP_address>/test/?text=hello_world

If everything works correctly, you will see reversed words from the text argument string.

Create a VM disk snapshot

Create a VM disk snapshot to transfer your web server configuration and test application to CI VMs.

In the Yandex Cloud management console, select the folder where you created your VM.
Select Compute Cloud.
Find the ci-tutorial-test-app VM and select it.
Click Stop.
In the window that opens, click Stop.
Once the VM gets stopped, select the Disks tab.
Select the disk you need, click , and select Create snapshot.
In the window that opens, specify the snapshot name: test-app-snap.
Click Create.

Create a GitLab VM

You can set up CI in Yandex Cloud by using a public image with GitLab pre-installed. GitLab includes tools for managing Git repositories and configuring CI.

In the management console, select the folder where you want to create your VM.
In the list of services, select Compute Cloud.
In the left-hand panel, select Virtual machines.
Click Create virtual machine.
Under Boot disk image, navigate to the Marketplace tab, click Show all Marketplace products, and select the GitLab image.
Under Location, select an availability zone where your VM will reside.
Under Computing resources, navigate to the Custom tab and specify these parameters:
- Platform: Intel Ice Lake
- vCPU: 2
- Guaranteed vCPU performance: 100%
- RAM: 2 GB
In Network settings, select the subnet where you will connect your VM.
Under Access, select SSH key and specify the VM access credentials:
- In the Login field, specify the VM user name, e.g., yc-user.
  
  Alert
  
  Do not use root or other reserved usernames. To perform operations requiring root privileges, use the sudo command.
- In the SSH key field, select the SSH key saved in your organization user profile.
  
  If there are no saved SSH keys in your profile, or you want to add a new key:
  - Click Add key.
  - Enter a name for the SSH key.
  - Upload or paste the contents of the public key file. You need to create a key pair for the SSH connection to a VM yourself.
  - Click Add.
  The SSH key will be added to your organization user profile.
  
  If users cannot add SSH keys to their profiles in the organization, the added public SSH key will only be saved to the user profile of the VM being created.
Under General information, specify the VM name: ci-tutorial-gitlab.
Click Create VM.

It may take a few minutes to create your VM. When its status changes to RUNNING, you can start configuring it.

Once your VM is created, the system will assign it an IP address and a host name (FQDN). You can use them for SSH access.

Configure GitLab

To set GitLab up and configure the CI process, create a new project and enter your CI login credentials:

On the Compute Cloud page, select the created VM and copy its public IP address.
Connect to the VM over SSH.
Get the GitLab administrator password with the following VM command:
```
sudo cat /etc/gitlab/initial_root_password
```
Copy the password without spaces from the Password line to the clipboard or a separate file.
In your browser, open http://<VM_public_IP_address>. This will take you to the GitLab web interface.
Log in as the administrator:
- Username or email: root.
- Password: Password you copied in the previous step.
If you are unable to log in, reset the administrator password.
Change the administrator password.
Log in as the administrator with the new password.
Select Create a project.
Specify the project name: gitlab-test.
Click Create project.
Get a Yandex OAuth token. To do this, follow this link and click Allow.
In your browser, open this link: http://<VM_public_IP_address>/root.
Select the gitlab-test project.
On the screen that opens, click Settings on the left and, in the popup menu, select CI/CD.
Under Variables, click Expand.
Create a new variable:
- Specify the variable name: YC_OAUTH.
- As the value of the variable, specify the OAuth token you received previously.
- Click Save variables.
Under Runners, click Expand.
Under Set up a specific Runner automatically, you will see the GitLab server address and the Runner token. You will need them when registering your runner.

Runner is a tool for running user tasks. You need to create a runner on your VM and register it in GitLab. To make your runner work, install the Yandex Cloud CLI and create a test to check the new VM.

Connect to the GitLab VM over SSH:

ssh <login>@<public_IP_address_of_VM_with_GitLab>

Add a new repository to the package manager:

curl --location https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash

Install GitLab Runner to run your CI scripts:
```
sudo apt-get -y install gitlab-runner
```

In response to Please enter the gitlab-ci coordinator URL, specify your GitLab IP address.
In response to Please enter the gitlab-ci token for this runner, speciffy your GitLab Runner token.
In response to Please enter the gitlab-ci description for this runner, specify the gitlab test runner description.
In response to Please enter the gitlab-ci tags for this runner, press Enter.
In response to Please enter the executor, specify shell.

sudo gitlab-runner register
Runtime platform                                    arch=amd64 os=linux pid=8197 revision=3afdaba6 version=11.5.0
Running in system-mode.

Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
http://<GitLab>_CI_IP_address/
Please enter the gitlab-ci token for this runner:
<Runner_token>
Please enter the gitlab-ci description for this runner:
[ci-tutorial-gitlab]: gitlab test runner
Please enter the gitlab-ci tags for this runner (comma separated):

Registering runner... succeeded                     runner=wZqzyy9s
Please enter the executor: virtualbox, docker+machine, docker-ssh+machine, kubernetes, docker, docker-ssh, shell, ssh, parallels:
shell
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

Install the Yandex Cloud CLI so you will be able to create VMs with the CI script:

curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh --output install.sh
sudo bash install.sh -n -i /opt/yc

Install the pytest package for functional testing:
```
sudo apt-get install python-pytest
```

Create the test.py file with the functional testing script:

Open the gitlab-test repository home page.
Click + and select New file.
In the window that opens, specify the file name: test.py.

Add the following code to the file:

# test.py
import pytest
import json
import socket as s

@pytest.fixture
def hostname(request):
    with open("instance-creation.out", "r") as fd:
        fqdn = json.loads(fd.read()).get("fqdn")

    return fqdn

@pytest.fixture
def socket(request):
    _socket = s.socket(s.AF_INET, s.SOCK_STREAM)
    def socket_teardown():
        _socket.close()
    request.addfinalizer(socket_teardown)
    return _socket

def test_server_connect(socket, hostname):
    socket.connect((hostname, 80))
    assert socket

Specify your commit message and click Commit changes.

Configure CI

Now you need to configure CI.

Open the gitlab-test repository home page:

http://<public_IP_address_of_GitLab_VM>/root/gitlab-test

Click Set up CI/CD. This will open a window for adding a new file.

GitLab will automatically name the file as .gitlab-ci.yml. Do not rename it. Add the following configuration to the file:

#.gitlab-ci.yml
stages:
  - build
  - test

build:
  stage: build
  variables:
    snapshot_name: test-app-snap
    folder_id: <folder_ID>
    subnet_name: <subnet_name>
  script:
    - export instance_name="ci-tutorial-test-app-$(date +%s)"
    - export PATH="/opt/yc/bin:${PATH}"
    - yc config set token $YC_OAUTH
    - yc compute instance create
      --format json
      --name $instance_name
      --folder-id $folder_id
      --zone ru-central1-d
      --network-interface subnet-name=$subnet_name,nat-ip-version=ipv4
      --create-boot-disk name=$instance_name-boot,type=network-ssd,size=15,snapshot-name=$snapshot_name,auto-delete=true
      --memory 1
      --cores 1
      --hostname $instance_name > instance-creation.out
    - sleep 30
  artifacts:
    when: on_success
    paths:
    - instance-creation.out
    expire_in: 10 mins

test_external:
  stage: test
  script:
    - py.test test.py > pytest-external.out
  artifacts:
    paths:
    - pytest-external.out
    expire_in: 1 hour

In the snapshot_name field, specify the first VM snapshot name.
In the folder_id field, specify the VM folder ID.
In the subnet_name field, specify your VM subnet name; you can find this name on the Virtual Private Cloud page in the relevant folder of the management console.
Click Commit changes.

Test the application on the CI VM

After making a commit, make sure that CI performed its function correctly. You should see a new VM with a web server and your test application in the relevant folder..

To test the new VM:

Open the Yandex Cloud management console.
In the folder with the VMs, select Compute Cloud.
If you did everything correctly, you will see a new VM in the list, e.g., ci-tutorial-test-app-1543910277.
Select the new VM and copy its public IP address.

In your browser, open this link:

http://<public_IP_address_of_created_VM>/test/?text=hello_world

Your test application should also work on the new VM, returning reversed words from the text argument.

Delete the resources you created

If you no longer need the VMs and images you created:

Creating test VMs via GitLab CI

Get your cloud readyGet your cloud ready

Required paid resourcesRequired paid resources

Create the test application VMCreate the test application VM

Configure the test application VMConfigure the test application VM

Check whether the test application is workingCheck whether the test application is working

Create a VM disk snapshotCreate a VM disk snapshot

Create a GitLab VMCreate a GitLab VM

Configure GitLabConfigure GitLab

Configure your runnerConfigure your runner

Configure CIConfigure CI

Test the application on the CI VMTest the application on the CI VM

Delete the resources you createdDelete the resources you created

Was the article helpful?