Storing Docker images created in Yandex Managed Service for GitLab projects

Container Registry is integrated in GitLab. It enables you to store Docker images for each of your projects in GitLab.

You can use Yandex Container Registry instead of GitLab Container Registry. This service enables you to store Docker images in the cloud or distribute them across Yandex Cloud managed services, for example, Yandex Managed Service for Kubernetes or Yandex Managed Service for GitLab.

Storing images from GitLab projects in Yandex Container Registry has several benefits:

• GitLab Container Registry stores images and tags on the GitLab instance disk. When you run out of disk space, you get the HTTP 500 error, and the instance becomes unavailable. You can recover the instance only by contacting tech support.

  Yandex Container Registry stores images and tags in registries for which individual quotas are allocated. Because of this, accumulating Docker images and tags does not affect the space available on the instance disk.

• The images are still available in Yandex Container Registry, even if Managed Service for GitLab is not.

• Yandex Container Registry supports the Docker image vulnerability scanner. Use the scanner to detect vulnerabilities and fix them before deploying your application. You can learn more about security scans in the Yandex Cloud blog.

To set up storage of Managed Service for GitLab Docker images in Yandex Container Registry:

1. Create a GitLab instance.
2. Configure GitLab.
3. Create a test application.
4. Create a GitLab Runner.
5. Create GitLab environment variables.
6. Create the CI script configuration file.
7. Check the result.
8. Enable a Docker image lifecycle policy.
9. (Optional) Scan your Docker images for vulnerabilities.

If you have already set up your Yandex Managed Service for GitLab instance for Continuous Integration (CI), check that you have prepared the infrastructure for Docker images. That done, begin the setup by creating environment variables.

If you no longer need the resources you created, delete them.

Getting startedGetting started

Sign up for Yandex Cloud and create a billing account:

1. Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Required paid resourcesRequired paid resources

Infrastructure support costs include fees for the following resources:

• Disks and continuously running VMs (see Yandex Compute Cloud pricing).
• Usage of a dynamic public IP (see Yandex Virtual Private Cloud pricing).
• Storing the Docker images you created and a vulnerability scanner, if activated (see Container Registry pricing).
• Usage of a Managed Service for Kubernetes master (see Managed Service for Kubernetes pricing).

Prepare the infrastructurePrepare the infrastructure

Create a GitLab instanceCreate a GitLab instance

Create a Managed Service for GitLab instance or a VM with a GitLab image on the same cloud network as the Managed Service for Kubernetes cluster.

Configure GitLabConfigure GitLab

To configure GitLab and enable Continuous Integration (CI), create a new project and enter the CI authorization parameters:

Create a test applicationCreate a test application

Create a test application that can be deployed in a Managed Service for Kubernetes cluster. To do this, add the following to the Dockerfile project:

1. Log in to GitLab.

2. Open the GitLab project.

3. Click in the repository navigation bar and select New file from the drop-down menu.

4. Name the file as Dockerfile and add the following code to it:

   FROM alpine:3.10
   CMD echo "Hello"
   

5. Add a comment to the commit in the Commit message field: Dockerfile for a test application.

6. Click Commit changes.

Create a GitLab RunnerCreate a GitLab Runner

To run build tasks in the Yandex Managed Service for Kubernetes cluster, create a GitLab Runner. To do this, install GitLab Runner by following this guide.

Once it is installed, you can run automated builds inside your Managed Service for Kubernetes cluster.

For more information about installing and running GitLab Runner, see the GitLab documentation.

Create GitLab environment variablesCreate GitLab environment variables

To allow Managed Service for GitLab to save Docker images and their tags in Yandex Container Registry, create the GitLab environment variables:

1. If you do not have the Yandex Cloud command line interface yet, install and initialize it.

   The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

2. Create a static key for the account-for-container-registry service account you created earlier:

   yc iam key create --service-account-name account-for-container-registry -o key.json
   

   The key is saved in the key.json file in the directory you ran the command in.

3. Open your project in GitLab.

4. Go to Settings in the left-hand panel, then select CI/CD from the drop-down list.

5. Click Expand next to Variables.

6. Add environment variables with the protection option disabled:

   Variable	Its value
   CI_REGISTRY	cr.yandex/<registry_ID>. Specify the ID of the Yandex Container Registry registry that you created previously.
   CI_REGISTRY_KEY	key.json file contents.

   To add a variable:

   1. Click Add variable.
   2. In the window that opens, enter the variable name in the Key field and the value in the Value field.
   3. Disable the Protect variable option.
   4. Click Add variable.

Create the CI script configuration fileCreate the CI script configuration file

To build images from a Dockerfile without Docker, use kaniko.

To publish Docker images from your GitLab project in Yandex Container Registry, create a CI script:

1. Open the gitlab-test project.

2. Click in the repository navigation bar and select New file from the drop-down menu.

3. Name your file .gitlab-ci.yml. Add to it the steps to build a Docker image and push it to Yandex Container Registry:

   .gitlab-ci.yml

   build:
      stage: build
      # Using kaniko to create a container inside the container for enhanced security.
      image:
         name: gcr.io/kaniko-project/executor:debug
         entrypoint: [""]
      script:
         - mkdir -p /kaniko/.docker
         # Upload the container image to the registry. The image is marked with the commit hash.
         - echo "{\"auths\":{\"$CI_REGISTRY\":{\"auth\":\"$(echo -n "json_key:${CI_REGISTRY_KEY}" | base64 | tr -d '\n' )\"}}}" > /kaniko/.docker/config.json
         - >-
            /kaniko/executor
            --context "${CI_PROJECT_DIR}"
            --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
            --destination "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}"
   

   This file includes the variables:

   • CI_REGISTRY and CI_REGISTRY_KEY: Added to GitLab at the previous step.
   • CI_PROJECT_DIR, CI_PROJECT_PATH, and CI_COMMIT_SHORT_SHA: Preset in GitLab.

4. Add a comment to the commit in the Commit message field: Create a CI pipeline.

5. Click Commit changes.

Check the resultCheck the result

Each commit is followed by a build script. To check the script execution results:

1. Select Build on the left-hand panel in the gitlab-test project and select Pipelines from the drop-down menu.

2. Make sure the build step gets the passed status. This means that the CI script has been executed successfully.

3. Go to the management console, then open the Yandex Container Registry registry.

   If the script completes successfully, a new repository is added to the registry. New Docker images from the GitLab project are added to this repository at each commit.

Enable a Docker image lifecycle policyEnable a Docker image lifecycle policy

To avoid storing outdated Docker images and their tags, configure a Docker image lifecycle policy. The policy manages the images stored in your Container Registry repository to free up its space in a timely manner. This way, you no longer pay for storing outdated images.

To create a policy, follow the guide.

Specifics of working with the policy:

• External Container Registry and Docker image lifecycle policies affect CI script performance.

• The policy imposes a limit on the maximum number of images that you can check per policy run. If the number of images in your Container Registry repository exceeds this limit, run the policy several times. You can check all your images in this manner.

Scan your Docker images for vulnerabilitiesScan your Docker images for vulnerabilities

To detect vulnerabilities in your Docker images, you can additionally activate a vulnerability scanner in Yandex Container Registry. The scanner checks the versions of packages installed in your images against CVE vulnerability databases.

To enable scanning, expand your GitLab project's CI script:

1. Open the gitlab-test project.

2. Open the .gitlab-ci.yml file.

3. Add to it the steps for vulnerability scanning of your Docker image:

   .gitlab-ci.yml

   stages:
      - build
      - test
   
   <build_block_previously_added_to_file>
   
   container_scanning_free_yc:
      stage: test
      # Using the jq utility to search for ID and write logs.
      image: 
         name: pindar/jq
         entrypoint: [""]
      artifacts:
         when: always
         paths:
            - gl-container-scanning-report-yc.json
      variables:
         # Specify the ID of the registry you previously created.
         CI_REGISTRY_ID: "<registry_ID>"
      script:
         - export CI_COMMIT_SHORT_SHA=${CI_COMMIT_SHORT_SHA}
         # Installing Yandex Cloud CLI.
         - curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bash -s -- -a && cp /root/yandex-cloud/bin/yc /usr/bin/
         # Start of scanning.
         - echo "Scanning image ${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}..."
         - export IMAGE_ID=$(yc container image list --registry-id $CI_REGISTRY_ID --format=json | jq -r --arg CI_COMMIT_SHORT_SHA $CI_COMMIT_SHORT_SHA '.[] | select(.tags[0]==$CI_COMMIT_SHORT_SHA) | .id ')
         # Logging.
         - export SCAN_RESULT=$(yc container image scan $IMAGE_ID --format=json)
         - export CRIT_VULN=$(echo $SCAN_RESULT | jq -r '.vulnerabilities.critical // 0')
         - export HIGH_VULN=$(echo $SCAN_RESULT | jq -r '.vulnerabilities.high // 0')
         - export SCAN_ID=$(echo $SCAN_RESULT | jq -r '.id')
         - echo "Scan results:"
         - yc container image list-vulnerabilities --scan-result-id="${SCAN_ID}" --format json | jq -r '.[] | select(.severity=="CRITICAL", .severity=="HIGH")'
         - yc container image list-vulnerabilities --scan-result-id="${SCAN_ID}" --format json | jq -r '.[] | select(.severity=="CRITICAL", .severity=="HIGH")' > gl-container-scanning-report-yc.json
         # Checking the result.
         - (( SUM = $CRIT_VULN + $HIGH_VULN )) && (( RES = (SUM >= 1) )) && echo $RES && echo "image has $CRIT_VULN critical vulnerabilities and $HIGH_VULN high vulnerabilities" && exit 1 || echo "image has no high or critical vulnerabilities" exit 0
   

4. Add a comment to the commit in the Commit message field: Turn on a vulnerability scanner.

5. Click Commit changes. After that, the updated script will run.

To make sure that the image scan was successful:

1. Select Build on the left-hand panel in the gitlab-test project and select Pipelines from the drop-down menu.

2. Make sure that the build and test steps got the passed status. This means that the CI script has been executed successfully.

3. Go to the management console, then open the Yandex Container Registry registry.

4. Open your repository with Docker images from the GitLab project.

5. Go to the directory with the GitLab project name.

6. Make sure that the Scan status column includes Ready.

7. In the Date of last scan column, click the link with the scan time.

   You will see the scan results. If vulnerabilities were found in the images, you will see them in the results.

Delete the resources you createdDelete the resources you created

If you no longer need the resources you created, delete them:

1. Delete the Managed Service for GitLab instance or the created VM with the GitLab image.
2. Delete all Docker images from the Container Registry registry.

Delete the other resources depending on how they were created: