Managing the MLOps lifecycle with ML Registry in Yandex Managed Service for GitLab

The machine learning model lifecycle comprises development, model training, quality evaluation, and model deployment. This guide explores GitLab's capabilities related to storing and versioning of the ML artifacts you get during experiments with the ML model within the Yandex Cloud infrastructure. The experiments change the model's hyperparameters, i.e., settings configured prior to the start of model training, which determine its architecture, training strategy, and overall behavior. Unlike the model's parameters (weights, coefficients), which are picked out in the course of data-based training, hyperparameters are not changed automatically: they are selected by the researcher or ML engineer.

The diagram depicts the model development and testing lifecycle in the GitLab environment:

The development and testing environment is represented by a DSVM. Your GitLab instance with the ML Registry package preinstalled is enabled by the scalable Yandex Managed Service for GitLab. This makes the following MLOps tools available to you:

• Model Registry, which you can use to log metrics, analyze model application experiments, and perform quality assessment.
• The Model Experiments catalog used for model storage and versioning management.

There is the MLFlow platform's Python API library for integration of your model with ML Registry.

In addition to Managed Service for GitLab, you use Yandex Compute Cloud to work with the model's source code, and Yandex Virtual Private Cloud for the network infrastructure.

System architectureSystem architecture

NetworkNetwork

As part of the solution, you create a Virtual Private Cloud cloud network named net-gitlab.

SubnetsSubnets

Within net-gitlab, you create a subnet in an availability zone of your choice to host the Managed Service for GitLab instance and the VM.

Security groupsSecurity groups

Network access to resources within the infrastructure is controlled with the help of security groups. More on configuring security group rules for Managed Service for GitLab.

Resource addressesResource addresses

The new infrastructure uses a public IP address for the new VM and the URL of the GitLab instance in the gitlab.yandexcloud.net domain.

Managed Service for GitLabManaged Service for GitLab

The GitLab instance is deployed on a VM managed by Managed Service for GitLab. The instance can be accessed by its address via the standard GitLab web interface.

To execute GitLab CI/CD jobs, create and configure GitLab Runner for your Managed Service for GitLab instance.

Compute CloudCompute Cloud

Local testing and uploads of changes to the model's source code repository are done with the help of a Data Science Virtual Machine virtual machine. A separate VM can be used to deploy GitLab Runner.

Test modelTest model

The test ML model deployed in this guide simulates the development and versioning cycle of a credit pipeline model. The model is adapted for use in a cloud infrastructure.

To deploy the model's development environment in the Yandex Cloud environment:

1. Get your cloud ready.
2. Create your infrastructure.
3. Create a project and configure the environment.
4. Create an experiment and a model version.

If you no longer need the resources you created, delete them.

Get your cloud readyGet your cloud ready

Sign up for Yandex Cloud and create a billing account:

1. Navigate to the management console and log in to Yandex Cloud or create a new account.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can create or select a folder for your infrastructure on the cloud page.

Learn more about clouds and folders here.

Required paid resourcesRequired paid resources

• Managed Service for GitLab: use of computing resources of the instance (VM) and its data storage capacity (see Managed Service for GitLab pricing). Depending on where GitLab Runner is deployed, you may have to pay for the Compute Cloud VM used to install GitLab Runner.
• VMs: use of computing resources, storage, public IP address, and OS (see Compute Cloud pricing).
• Object Storage: storage of Managed Service for GitLab backups (see Object Storage pricing).

Create your infrastructureCreate your infrastructure

Create a project and configure the environmentCreate a project and configure the environment

1. Create a GitLab project, select Import project on the home page, and specify the import settings:

   • Import project from: Repository by URL.
   • Git Repository URL: https://github.com/yandex-cloud-examples/yc-ml-ops-managed-gitlab.git.
   • Project name: gitlab-mlflow.

2. Deploy a runner for the new GitLab project according to the instruction. During deployment, specify the previously created infrastructure components:

   • If installing the runner on your VM manually, select the subnet-gitlab-a subnet and the gitlab-sg security group when creating the VM.
   • If creating the runner using the management console, specify the gitlab-sa service account and the gitlab-sg security group.

Configure the environment variablesConfigure the environment variables

1. Open the GitLab project named gitlab-mlflow.

2. Navigate to Settings in the left-hand panel and select Access Tokens from the drop-down list.

3. Create a new token with the following settings:

   • Token name: mlflow.
   • Select a role: Maintainer.
   • Select scopes: api, manage_runner, read_repository, write_repository.

4. Click Create project access token.

5. Copy the token value.

6. Select the Settings tab on the left and CI/CD in the popup list.

7. Under Variables, click Expand.

8. Add these environment variables:

   • MLFLOW_TRACKING_TOKEN: New token.
   • MLFLOW_TRACKING_URI: https://<GitLab_instance_address>.gitlab.yandexcloud.net/api/v4/projects/4/ml/mlflow.
   • REPO_TOKEN: New token.

   To add a variable:

   • Click Add variable.
   • In the window that opens, specify a variable name in the Key field and its value in the Value field.
   • Click Add variable.

Create an experiment and a model versionCreate an experiment and a model version

1. Connect to the vm-mlops VM over SSH.

2. Add an SSH key for secure access to GitLab.

3. Clone the gitlab-mlflow project repository using SSH.

4. Navigate to the repository directory and create a branch named mlops-experiment-1:

   git checkout -b mlops-experiment-1
   

5. Edit the model's parameters in the loan_prediction.py file, for example, edit RANDOM_SEED.

6. Upload the changes to GitLab:

   git add -A
   git commit -m "Change model parameter"
   git push
   

7. Open the GitLab project named gitlab-mlflow and create a merge request from the new branch. A training and testing scenario for the modified model will be created automatically.

8. Run the scenario:

   1. In the left-hand panel, select Build.
   2. Select Pipelines from the drop-down list.
   3. Click in the Actions column and select trigger_train.

   Wait for the scenario to end.

9. Create a registry of model versions:

   1. In the left-hand panel, select Deploy.
   2. Select Model Registry from the drop-down list.
   3. Click Create/import model and select Create new model.
   4. Enter loan-prediction-demo as the model's name and click Create.

10. Check the results of the experiment:

    1. In the left-hand panel, select Analyze.
    2. Select Model Experiments from the drop-down list.
    3. Select the Loan_prediction experiment from the list in the right and navigate to the Runs tab. The list displays all the model training methods, their parameters, and training results.
    4. Click an item in the Name column. The Artifacts tab displays the training artifacts of the method you select.
    5. Click Promote run to register the model version. Select the loan-prediction-demo model from the drop-down list in the Model field, enter 0.0.1 in the Version field, and click Promote. A new version and all its training artifacts will be added to the version registry.

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:

See alsoSee also

• Machine Learning Model Experiments
• Model Registry