Continuous deployment of containerized applications using GitLab
GitLab
This tutorial describes:
- Building an application into a Docker container.
- Deploying an application from a container in a Yandex Managed Service for Kubernetes cluster via GitLab using the Yandex Cloud tools.
Each commit to GitLab is followed by:
- Running a script that includes steps to build the Docker image.
- Applying a new Managed Service for Kubernetes cluster configuration specifying the application to deploy.
To set up the infrastructure required to store the source code, build the Docker image, and deploy your applications, follow these steps:
If you no longer need the resources you created, delete them.
Prepare your cloud
Sign up for Yandex Cloud and create a billing account:
- Go to the management console
and log in to Yandex Cloud or create an account if you do not have one yet. - On the Yandex Cloud Billing
page, make sure you have a billing account linked and it has theACTIVE
orTRIAL_ACTIVE
status. If you do not have a billing account, create one.
If you have an active billing account, you can go to the cloud page
Learn more about clouds and folders.
Required paid resources
Infrastructure support costs include fees for the following resources:
- Disks and continuously running VMs (see the Yandex Compute Cloud pricing).
- Usage of a dynamic public IP (see Yandex Virtual Private Cloud pricing).
- Storage of created Docker images (see Container Registry pricing).
- Usage of a Managed Service for Kubernetes master (see Managed Service for Kubernetes pricing).
Prepare the infrastructure
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
-
If you do not have a network yet, create one.
-
If you do not have any subnets, create them in the availability zones where your Yandex Managed Service for Kubernetes cluster and node group will be created.
-
- For resources with the editor role for the folder where your Managed Service for Kubernetes cluster will be created. This service account will be used to create the resources required for the Managed Service for Kubernetes cluster.
- For nodes with the container-registry.images.puller and container-registry.images.pusher roles for the folder with the Docker image registry. This service account will be used by the Managed Service for Kubernetes nodes to push the Docker images built in GitLab to the registry and pull them to run pods.
Tip
You can use the same service account for both operations.
-
Create security groups for the Managed Service for Kubernetes cluster and its node groups.
Warning
The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.
-
Create a security group for the Managed Service for GitLab instance.
-
Create a Managed Service for Kubernetes cluster and a node group. When creating a Managed Service for Kubernetes cluster, specify the previously created service accounts for the resources and nodes and the security groups.
-
Create an authorized key for the service account with the
container-registry.images.pusher
role and save it to thekey.json
file:yc iam key create \ --service-account-name <service_account_name> \ --output key.json
This key is required to access the registry from GitLab.
-
If you do not have Terraform yet, install it.
-
Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.
-
Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it
. -
Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.
-
Download the k8s-and-registry-for-gitlab.tf
configuration file to the same working directory.This file describes:
-
Service account required for the Managed Service for Kubernetes cluster and node group.
-
Security groups which contain rules required for the Managed Service for Kubernetes cluster and its node groups.
Warning
The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.
-
Default security group and rules needed to run the Managed Service for GitLab instance.
-
Authorized key for the service account. This key is required to access the registry from GitLab.
-
Local
key.json
file with authorized key data.
-
In the
k8s-and-registry-for-gitlab.tf
file, specify:- Folder ID.
- Kubernetes version for the Managed Service for Kubernetes cluster and node groups.
- Name of the Managed Service for Kubernetes cluster service account.
- Name of the Container Registry registry.
-
Make sure the Terraform configuration files are correct using this command:
terraform validate
If there are any errors in the configuration files, Terraform will point them out.
-
Create the required infrastructure:
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console
. -
Warning
For applications running in production environments, make sure to restrict access of Managed Service for Kubernetes cluster service accounts to pushing Docker images to a registry. This is required for security reasons. In that case, create a separate service account with the container-registry.images.pusher
role and specify it for deploying applications.
Install additional dependencies
Install the following items in the local environment:
jq
JSON stream processor- kubectl command-line tool
. Configure it to work with the created Managed Service for Kubernetes cluster.
Create a GitLab instance
Create a Managed Service for GitLab instance or a VM with a GitLab image on the same cloud network as the Managed Service for Kubernetes cluster.
Create a Managed Service for GitLab instance by following this guide.
Launch GitLab on a VM with a public IP.
- On the folder page in the management console
, click Create resource and select Virtual machine instance. - In the Name field, enter the VM name as follows:
ci-tutorial-gitlab
. - Select an availability zone to place your VM in.
- Under Boot disk image, go to the Marketplace tab and click Show all Marketplace products. In the window that opens, select GitLab as your image and click Use.
- Under Computing resources, specify the following configuration:
- vCPU:
4
- Guaranteed vCPU performance:
100%
- RAM:
8 GB
- vCPU:
- Under Network settings:
-
Select a subnet to connect your VM to. If the required network or subnet is not listed, create it using the Create network and Create subnet buttons.
Warning
For the time being, Yandex Cloud technical restrictions do not allow selecting a subnet with an address range of
192.168.0.0/24
. -
In the Public IP field, select
Auto
.
-
- Under Access, specify the information required to access the instance:
-
Enter the username in the Login field.
Alert
Do not use the
root
username or other names reserved by the operating system. To perform operations that require superuser permissions, use thesudo
command. -
In the SSH key field, paste the contents of the public key file. You need to create a key pair for the SSH connection yourself. To learn how, see Connecting to a VM via SSH.
-
- Click Create VM.
It may take a few minutes to create the VM. When the VM changes its status to RUNNING
and GitLab starts, you can proceed to setup.
Configure GitLab
To configure GitLab and enable Continuous Integration (CI), create a new project and enter the CI authorization parameters:
-
Log in to the Managed Service for GitLab instance web interface.
-
Click Create a project.
-
Click Create blank project.
-
Fill out the fields below:
- Project name:
gitlab-test
. - Project URL: Select the administrator user in the field next to the Managed Service for GitLab instance FQDN.
Leave the other fields unchanged.
- Project name:
-
Click Create project.
-
On the Yandex Compute Cloud page, select the created VM and copy its public IP.
-
Connect to the VM via SSH.
-
Get the GitLab administrator password using the following VM command:
sudo cat /etc/gitlab/initial_root_password
-
Copy the password (without spaces) from the
Password
row to the clipboard or a separate file. -
Open
http://<VM_public_IP_address>
in your browser. This will take you to the GitLab web interface. -
Log in using the administrator account:
- Username or email:
root
- Password: Password you copied earlier
If you are unable to log in, reset the administrator account password
. - Username or email:
-
Log in to the system again using the administrator account and the new password.
-
Select Create a project.
-
Set the project name:
gitlab-test
. -
Click Create project.
Create a test application
Create a test application that can be deployed in a Yandex Managed Service for Kubernetes cluster:
- Add a
Dockerfile
to the project:-
Log in to GitLab.
-
Open the GitLab project.
-
Click
in the repository navigation bar and select New file from the drop-down menu. -
Name the file as
Dockerfile
and add the following code to it:FROM alpine:3.10 CMD echo "Hello"
-
Add a comment to the commit in the Commit message field:
Dockerfile for test application
. -
Click Commit changes.
-
- Add the manifest for creating Managed Service for Kubernetes cluster resources to the project:
-
Open the GitLab project.
-
Click
in the repository navigation bar and select New file from the drop-down menu. -
Name the file as
k8s.yaml
:k8s.yaml
apiVersion: v1 kind: Namespace metadata: name: hello-world --- apiVersion: apps/v1 kind: Deployment metadata: name: hello-world-deployment namespace: hello-world spec: replicas: 1 selector: matchLabels: app: hello template: metadata: namespace: hello-world labels: app: hello spec: containers: - name: hello-world image: __VERSION__ imagePullPolicy: Always
-
Add a comment to the commit in the Commit message field:
Docker image deployment config
. -
Click Commit changes.
-
Create a GitLab Runner
To run build tasks in the Yandex Managed Service for Kubernetes cluster, create a GitLab Runner
Once it is installed, you can run automated builds inside your Managed Service for Kubernetes cluster.
For more information about installing and running GitLab Runner, see the GitLab documentation
Set up Kubernetes authentication in GitLab
You can set up authentication in GitLab using a Kubernetes service account token or the GitLab Agent application:
Note
The Kubernetes service account is different from the Yandex Identity and Access Management service account.
To get the Kubernetes service account token:
- Create a service account.
- Get a service account token.
- Save the token: you need it for the next steps.
To connect your Yandex Managed Service for Kubernetes cluster to GitLab, create a GitLab Agent
Once it is installed, you can connect your Managed Service for Kubernetes cluster to a GitLab instance.
For more information about installing and running GitLab Agent, see the GitLab documentation
Configure the CI script
-
Create the GitLab environment variables
:-
In GitLab, go to Settings in the left-hand panel and select CI/CD from the drop-down list.
-
Click Expand next to Variables.
-
Add the following environment variables depending on the Kubernetes authentication method in GitLab:
Service account tokenGitLab Agent-
KUBE_URL
: Managed Service for Kubernetes master address You can retrieve it using the following command:yc managed-kubernetes cluster get <cluster_ID_or_name> --format=json \ | jq -r .master.endpoints.external_v4_endpoint
-
KUBE_TOKEN
: Token that will use GitLab to apply the configuration. Use the token obtained earlier.
CI_REGISTRY
: Address of the previously created registry incr.yandex/<registry_ID>
format.CI_REGISTRY_KEY
: Key that GitLab will use to access the registry. Copy the contents of the previously obtainedkey.json
static key file to access the registry.
To add a variable:
- Click Add variable.
- In the window that opens, enter the variable name in the Key field and the value in the Value field.
- Click Add variable.
-
-
-
Create the CI script configuration file:
-
Open the
gitlab-test
project. -
Click
in the repository navigation bar and select New file from the drop-down menu. -
Name the file as
.gitlab-ci.yml
. Add the steps to build and push a Docker image and update the application configuration in the Managed Service for Kubernetes cluster. The file structure depends on the Kubernetes authentication method in GitLab:Service account tokenGitLab Agent-
To build a container through
kaniko
without using the GitLab Runner privileged mode:.gitlab-ci.yml
stages: - build - deploy build: stage: build image: name: gcr.io/kaniko-project/executor:debug entrypoint: [""] script: - mkdir -p /kaniko/.docker - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(echo -n "json_key:${CI_REGISTRY_KEY}" | base64 | tr -d '\n' )\"}}}" > /kaniko/.docker/config.json - >- /kaniko/executor --context "${CI_PROJECT_DIR}" --dockerfile "${CI_PROJECT_DIR}/Dockerfile" --destination "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}" deploy: image: gcr.io/cloud-builders/kubectl:latest stage: deploy script: - kubectl config set-cluster k8s --server="$KUBE_URL" --insecure-skip-tls-verify=true - kubectl config set-credentials admin --token="$KUBE_TOKEN" - kubectl config set-context default --cluster=k8s --user=admin - kubectl config use-context default - sed -i "s,__VERSION__,${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}," k8s.yaml - kubectl apply -f k8s.yaml
-
To build a container through
docker:dind
using the GitLab Runner privileged mode:.gitlab-ci.yml
stages: - build - deploy image: docker:20.10.16 variables: DOCKER_HOST: tcp://docker:2376 DOCKER_TLS_CERTDIR: "/certs" DOCKER_TLS_VERIFY: 1 DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client" DOCKER_DRIVER: overlay2 services: - docker:20.10.16-dind before_script: - for try in {1..10}; do sleep 0.5; docker info && break ; done build: stage: build script: - echo "${CI_REGISTRY_KEY}" | docker login ${CI_REGISTRY} -u json_key --password-stdin - >- docker build "${CI_PROJECT_DIR}" --file "${CI_PROJECT_DIR}/Dockerfile" --tag "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}" - docker push "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}" deploy: image: gcr.io/cloud-builders/kubectl:latest stage: deploy script: - kubectl config set-cluster k8s --server="$KUBE_URL" --insecure-skip-tls-verify=true - kubectl config set-credentials admin --token="$KUBE_TOKEN" - kubectl config set-context default --cluster=k8s --user=admin - kubectl config use-context default - sed -i "s,__VERSION__,${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}," k8s.yaml - kubectl apply -f k8s.yaml
-
To build a container through
kaniko
without using the GitLab Runner privileged mode:.gitlab-ci.yml
stages: - build - deploy build: stage: build image: name: gcr.io/kaniko-project/executor:debug entrypoint: [""] script: - mkdir -p /kaniko/.docker - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(echo -n "json_key:${CI_REGISTRY_KEY}" | base64 | tr -d '\n' )\"}}}" > /kaniko/.docker/config.json - >- /kaniko/executor --context "${CI_PROJECT_DIR}" --dockerfile "${CI_PROJECT_DIR}/Dockerfile" --destination "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}" deploy: image: bitnami/kubectl:latest stage: deploy script: - kubectl config use-context ${CI_PROJECT_PATH}:<GitLab_Agent_name> - cat k8s.yaml | sed -e "s,__VERSION__,${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}," | kubectl apply -f -
-
To build a container through
docker:dind
using the GitLab Runner privileged mode:.gitlab-ci.yml
stages: - build - deploy image: docker:20.10.16 variables: DOCKER_HOST: tcp://docker:2376 DOCKER_TLS_CERTDIR: "/certs" DOCKER_TLS_VERIFY: 1 DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client" DOCKER_DRIVER: overlay2 services: - docker:20.10.16-dind before_script: - for try in {1..10}; do sleep 0.5; docker info && break ; done build: stage: build script: - echo "${CI_REGISTRY_KEY}" | docker login ${CI_REGISTRY} -u json_key --password-stdin - >- docker build "${CI_PROJECT_DIR}" --file "${CI_PROJECT_DIR}/Dockerfile" --tag "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}" - docker push "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}" deploy: image: bitnami/kubectl:latest stage: deploy script: - kubectl config use-context ${CI_PROJECT_PATH}:<GitLab_Agent_name> - cat k8s.yaml | sed -e "s,__VERSION__,${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHORT_SHA}," | kubectl apply -f -
Replace
<GitLab_Agent_name>
with the agent name in Managed Service for GitLab. -
-
Add a comment to the commit in the Commit message field:
CI scripts
. -
Click Commit changes.
In the
.gitlab-ci.yml
file, the following two steps of project build are described:- Building a Docker image using the
Dockerfile
and pushing the image to Container Registry. - Setting up an environment to work with Kubernetes and applying
k8s.yaml
configurations to Managed Service for Kubernetes clusters. This way, the application is deployed on the previously created Managed Service for Kubernetes cluster.
-
Check the result
-
After you save the
.gitlab-ci.yml
configuration file, the build script will start. To check its results, select Build on the left-hand panel in thegitlab-test
project, then select Pipelines from the drop-down menu, and wait for both build stages to complete successfully. -
To check how the created application is running in your Managed Service for Kubernetes cluster, view its container logs:
kubectl logs deployment/hello-world-deployment -n hello-world
Result:
Hello
Delete the resources you created
Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:
-
Delete the Managed Service for Kubernetes cluster and Container Registry registry:
ManuallyTerraform-
In the command line, go to the directory with the current Terraform configuration file with an infrastructure plan.
-
Delete the
k8s-and-registry-for-gitlab.tf
configuration file. -
Make sure the Terraform configuration files are correct using this command:
terraform validate
If there are any errors in the configuration files, Terraform will point them out.
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
All resources described in the
k8s-and-registry-for-gitlab.tf
configuration file will be deleted. -
-
-
Delete the GitLab VM or Managed Service for GitLab instance that you created.