Continuous deployment of containerized Managed Service for Kubernetes applications

GitLab is a tool for Continuous integration (CI).

This tutorial describes:

• Building an application into a Docker container.
• Deploying an application from a container in a Yandex Managed Service for Kubernetes cluster via GitLab using the Yandex Cloud tools.

Each commit to GitLab is followed by:

• Running a script that includes steps to build the Docker image.
• Applying a new Managed Service for Kubernetes cluster configuration specifying the application to deploy.

To set up the infrastructure required to store the source code, build the Docker image, and deploy your applications, follow these steps:

1. Prepare your cloud.

   1. Review the list of paid resources available.

2. Prepare your infrastructure.

3. Create a GitLab instance.

4. Configure GitLab.

5. Create a test application.

6. Create a GitLab Runner.

7. Set up Kubernetes authentication in GitLab.

8. Configure the CI script.

9. Check the result.

If you no longer need the resources you created, delete them.

Prepare your cloudPrepare your cloud

Sign up for Yandex Cloud and create a billing account:

1. Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Required paid resourcesRequired paid resources

Infrastructure support costs include fees for the following resources:

• Disks and continuously running VMs (see the Yandex Compute Cloud pricing).
• Usage of a dynamic public IP (see Yandex Virtual Private Cloud pricing).
• Storage of created Docker images (see Container Registry pricing).
• Usage of a Managed Service for Kubernetes master (see Managed Service for Kubernetes pricing).

Prepare the infrastructurePrepare the infrastructure

Install additional dependenciesInstall additional dependencies

Install the following items in the local environment:

• jq JSON stream processor
• kubectl command-line tool. Configure it to work with the created Managed Service for Kubernetes cluster.

Create a GitLab instanceCreate a GitLab instance

Create a Managed Service for GitLab instance or a VM with a GitLab image on the same cloud network as the Managed Service for Kubernetes cluster.

Configure GitLabConfigure GitLab

To configure GitLab and enable Continuous Integration (CI), create a new project and enter the CI authorization parameters:

Create a test applicationCreate a test application

Create a test application that can be deployed in a Yandex Managed Service for Kubernetes cluster:

1. Add a Dockerfile to the project:

   1. Log in to GitLab.

   2. Open the GitLab project.

   3. Click in the repository navigation bar and select New file from the drop-down menu.

   4. Name the file as Dockerfile and add the following code to it:

      FROM alpine:3.10
      CMD echo "Hello"
      

   5. Add a comment to the commit in the Commit message field: Dockerfile for test application.

   6. Click Commit changes.

2. Add the manifest for creating Managed Service for Kubernetes cluster resources to the project:

   1. Open the GitLab project.

   2. Click in the repository navigation bar and select New file from the drop-down menu.

   3. Name the file as k8s.yaml:

      k8s.yaml

      apiVersion: v1
      kind: Namespace
      metadata:
        name: hello-world
      ---
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: hello-world-deployment
        namespace: hello-world
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: hello
        template:
          metadata:
            namespace: hello-world
            labels:
              app: hello
          spec:
            containers:
              - name: hello-world
                image: __VERSION__
                imagePullPolicy: Always
      

   4. Add a comment to the commit in the Commit message field: Docker image deployment config.

   5. Click Commit changes.

Create a GitLab RunnerCreate a GitLab Runner

To run build tasks in the Yandex Managed Service for Kubernetes cluster, create a GitLab Runner. To do this, install the GitLab Runner application by following this guide.

Once it is installed, you can run automated builds inside your Managed Service for Kubernetes cluster.

For more information about installing and running GitLab Runner, see the GitLab documentation.

Set up Kubernetes authentication in GitLabSet up Kubernetes authentication in GitLab

You can set up authentication in GitLab using a Kubernetes service account token or the GitLab Agent application:

Configure the CI scriptConfigure the CI script

1. Create the GitLab environment variables:

   1. In GitLab, go to Settings in the left-hand panel and select CI/CD from the drop-down list.

   2. Click Expand next to Variables.

   3. Add the following environment variables depending on the Kubernetes authentication method in GitLab:

      Service account token

      GitLab Agent

      • KUBE_URL: Managed Service for Kubernetes master address You can retrieve it using the following command:

        yc managed-kubernetes cluster get <cluster_ID_or_name> --format=json \
           | jq -r .master.endpoints.external_v4_endpoint
        

      • KUBE_TOKEN: Token that will use GitLab to apply the configuration. Use the token obtained earlier.
      • CI_REGISTRY: Address of the previously created registry in cr.yandex/<registry_ID> format.
      • CI_REGISTRY_KEY: Key that GitLab will use to access the registry. Copy the contents of the previously obtained key.json static key file to access the registry.

      To add a variable:

      • Click Add variable.
      • In the window that opens, enter the variable name in the Key field and the value in the Value field.
      • Click Add variable.

2. Create the CI script configuration file:

   1. Open the gitlab-test project.

   2. Click in the repository navigation bar and select New file from the drop-down menu.

   3. Name the file as .gitlab-ci.yml. Add the steps to build and push a Docker image and update the application configuration in the Managed Service for Kubernetes cluster. The file structure depends on the Kubernetes authentication method in GitLab:

      Service account token

      GitLab Agent

      • To build a container through kaniko without using the GitLab Runner privileged mode:

        .gitlab-ci.yml

        stages:
          - build
          - deploy
        
        build:
          stage: build
          image:
            name: gcr.io/kaniko-project/executor:debug
            entrypoint: [""]
          script:
            - mkdir -p /kaniko/.docker
            - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(echo -n "json_key:${CI_REGISTRY_KEY}" | base64 | tr -d '\n' )\"}}}" > /kaniko/.docker/config.json
            - >-
              /kaniko/executor
              --context "${CI_PROJECT_DIR}"
              --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
              --destination "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}"
        
        deploy:
          image: gcr.io/cloud-builders/kubectl:latest
          stage: deploy
          script:
            - kubectl config set-cluster k8s --server="$KUBE_URL" --insecure-skip-tls-verify=true
            - kubectl config set-credentials admin --token="$KUBE_TOKEN"
            - kubectl config set-context default --cluster=k8s --user=admin
            - kubectl config use-context default
            - sed -i "s,__VERSION__,${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}," k8s.yaml
            - kubectl apply -f k8s.yaml
        

      • To build a container through docker:dind using the GitLab Runner privileged mode:

        .gitlab-ci.yml

        stages:
          - build
          - deploy
        
        image: docker:20.10.16
        
        variables:
          DOCKER_HOST: tcp://docker:2376
          DOCKER_TLS_CERTDIR: "/certs"
          DOCKER_TLS_VERIFY: 1
          DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
          DOCKER_DRIVER: overlay2
        
        services:
          - docker:20.10.16-dind
        
        before_script:
          - for try in {1..10}; do sleep 0.5; docker info && break ; done
        
        build:
          stage: build
          script:
            - echo "${CI_REGISTRY_KEY}" | docker login ${CI_REGISTRY} -u json_key --password-stdin
            - >-
              docker build
              "${CI_PROJECT_DIR}"
              --file "${CI_PROJECT_DIR}/Dockerfile"
              --tag "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}"
            - docker push "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}"
        
        deploy:
          image: gcr.io/cloud-builders/kubectl:latest
          stage: deploy
          script:
            - kubectl config set-cluster k8s --server="$KUBE_URL" --insecure-skip-tls-verify=true
            - kubectl config set-credentials admin --token="$KUBE_TOKEN"
            - kubectl config set-context default --cluster=k8s --user=admin
            - kubectl config use-context default
            - sed -i "s,__VERSION__,${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}," k8s.yaml
            - kubectl apply -f k8s.yaml
        

      • To build a container through kaniko without using the GitLab Runner privileged mode:

        .gitlab-ci.yml

        stages:
          - build
          - deploy
        
        build:
          stage: build
          image:
            name: gcr.io/kaniko-project/executor:debug
            entrypoint: [""]
        script:
            - mkdir -p /kaniko/.docker
            - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(echo -n "json_key:${CI_REGISTRY_KEY}" | base64 | tr -d '\n' )\"}}}" > /kaniko/.docker/config.json
            - >-
              /kaniko/executor
              --context "${CI_PROJECT_DIR}"
              --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
              --destination "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}"
        
        deploy:
          image: bitnami/kubectl:latest
          stage: deploy
          script:
            - kubectl config use-context ${CI_PROJECT_PATH}:<GitLab_Agent_name>
            - cat k8s.yaml | sed -e "s,__VERSION__,${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}," | kubectl apply -f -
        

      • To build a container through docker:dind using the GitLab Runner privileged mode:

        .gitlab-ci.yml

        stages:
          - build
          - deploy
        
        image: docker:20.10.16
        
        variables:
          DOCKER_HOST: tcp://docker:2376
          DOCKER_TLS_CERTDIR: "/certs"
          DOCKER_TLS_VERIFY: 1
          DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
          DOCKER_DRIVER: overlay2
        
        services:
          - docker:20.10.16-dind
        
        before_script:
          - for try in {1..10}; do sleep 0.5; docker info && break ; done
        
        build:
          stage: build
          script:
            - echo "${CI_REGISTRY_KEY}" | docker login ${CI_REGISTRY} -u json_key --password-stdin
            - >-
              docker build
              "${CI_PROJECT_DIR}"
              --file "${CI_PROJECT_DIR}/Dockerfile"
              --tag "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}"
            - docker push "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}"
        
        deploy:
          image: bitnami/kubectl:latest
          stage: deploy
          script:
            - kubectl config use-context ${CI_PROJECT_PATH}:<GitLab_Agent_name>
            - cat k8s.yaml | sed -e "s,__VERSION__,${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}," | kubectl apply -f -
        

      Replace <GitLab_Agent_name> with the agent name in Managed Service for GitLab.

   4. Add a comment to the commit in the Commit message field: CI scripts.

   5. Click Commit changes.

   In the .gitlab-ci.yml file, the following two steps of project build are described:

   • Building a Docker image using the Dockerfile and pushing the image to Container Registry.
   • Setting up an environment to work with Kubernetes and applying k8s.yaml configurations to Managed Service for Kubernetes clusters. This way, the application is deployed on the previously created Managed Service for Kubernetes cluster.

Check the resultCheck the result

1. After you save the .gitlab-ci.yml configuration file, the build script will start. To check its results, select Build on the left-hand panel in the gitlab-test project, then select Pipelines from the drop-down menu, and wait for both build stages to complete successfully.

2. To check how the created application is running in your Managed Service for Kubernetes cluster, view its container logs:

   kubectl logs deployment/hello-world-deployment -n hello-world
   

   Result:

   Hello
   

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:

1. Delete the created Docker images.

2. Delete the Managed Service for Kubernetes cluster and Container Registry registry:

   Manually

   Terraform

   1. Delete the Managed Service for Kubernetes cluster.
   2. Delete the Container Registry registry.
   3. Delete the created subnets and networks.
   4. Delete the created service accounts.

   1. In the command line, go to the directory with the current Terraform configuration file with an infrastructure plan.

   2. Delete the k8s-and-registry-for-gitlab.tf configuration file.

   3. Make sure the Terraform configuration files are correct using this command:

      terraform validate
      

      If there are any errors in the configuration files, Terraform will point them out.

   4. Confirm updating the resources.

      1. Run the command to view planned changes:

         terraform plan
         

         If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

      2. If you are happy with the planned changes, apply them:

         1. Run the command:

            terraform apply
            

         2. Confirm the update of resources.

         3. Wait for the operation to complete.

      All resources described in the k8s-and-registry-for-gitlab.tf configuration file will be deleted.

3. Delete the GitLab VM or Managed Service for GitLab instance that you created.

See alsoSee also

• Creating test VMs using GitLab CI.