Creating a new Kubernetes project in Yandex Cloud

This tutorial describes how to run a new Kubernetes project in Yandex Cloud. You will deploy an application from Yandex Container Registry in a Managed Service for Kubernetes cluster and publish it on the internet via the Yandex Application Load Balancer ingress controller.

If you need to create a Managed Service for Kubernetes cluster with no internet access, see Creating and configuring a Kubernetes cluster with no internet access.

To launch an app:

1. Create service accounts.
2. Create security groups.
3. Set up Kubernetes resources.
4. Connect to the Managed Service for Kubernetes cluster.
5. Set up Container Registry resources.
6. Set up an Application Load Balancer.
7. Create a load balancer.

If you no longer need the resources you created, delete them.

Required paid resourcesRequired paid resources

The support cost for this solution includes:

• Fee for a DNS zone and DNS requests (see Cloud DNS pricing).
• Fee for using the master and outgoing traffic in a Managed Service for Kubernetes cluster (see Managed Service for Kubernetes pricing).
• Fee for using computing resources, OS, and storage in cluster nodes (VMs) (see Compute Cloud pricing).
• Container Registry fee for using the storage and outgoing traffic (see Container Registry pricing).
• Fee for using L7 load balancer's computing resources (see Application Load Balancer pricing).
• Fee for a public IP address for an L7 load balancer (see Virtual Private Cloud pricing).

Getting startedGetting started

1. If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

   By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

2. Install jq:

   sudo apt update && sudo apt install jq
   

Create a network and a subnetCreate a network and a subnet

1. Create a network named yc-auto-network:

   yc vpc network create --name yc-auto-network
   

2. Create a subnet in the ru-central1-a availability zone:

   yc vpc subnet create \
     --name yc-auto-subnet-0 \
     --network-name yc-auto-network \
     --range 192.168.1.0/24 \
     --zone ru-central1-a
   

Register a domain zone and add a certificateRegister a domain zone and add a certificate

1. Register a public domain zone and delegate your domain.

2. If you already have a certificate for the domain zone, add its details to Yandex Certificate Manager.

   If you have no certificate, issue a new Let's Encrypt® certificate and add it to Certificate Manager.

3. Get the certificate ID:

   yc certificate-manager certificate list
   

   Result:

   +-----------------+-------+----------------+---------------------+----------+--------+
   |       ID        | NAME  |    DOMAINS     |      NOT AFTER      |   TYPE   | STATUS |
   +-----------------+-------+----------------+---------------------+----------+--------+
   | <ID> | <name> | <domain_name> | 2022-04-06 17:19:37 | IMPORTED | ISSUED |
   +-----------------+-------+----------------+---------------------+----------+--------+
   

Create service accountsCreate service accounts

For a Managed Service for Kubernetes cluster and load balancer, you need the following service accounts:

• Service account with the k8s.clusters.agent and vpc.publicAdmin roles for the folder where you want to create a Managed Service for Kubernetes cluster. This service account will be used to create the resources for the Managed Service for Kubernetes cluster.
• Service account with the container-registry.images.puller role for the folder containing the Docker image registry. The nodes will use this service account to pull the required Docker images from the registry.
• For the Application Load Balancer ingress controller, you need service accounts with the following roles:
  • alb.editor: To create the required resources.
  • vpc.publicAdmin: To manage external connectivity.
  • certificate-manager.certificates.downloader: To use certificates registered in Certificate Manager.
  • compute.viewer: To use Managed Service for Kubernetes cluster nodes in the load balancer target groups.

Service account for resourcesService account for resources

To create a service account that will be used to create the resources for theManaged Service for Kubernetes cluster:

1. Save the folder ID from your Yandex Cloud CLI profile configuration to a variable:

   Bash

   PowerShell

   FOLDER_ID=$(yc config get folder-id)
   

   $FOLDER_ID = yc config get folder-id
   

2. Create a service account:

   Bash

   PowerShell

   yc iam service-account create --name k8s-res-sa-$FOLDER_ID
   

   yc iam service-account create --name k8s-res-sa-$FOLDER_ID
   

3. Save the service account ID to a variable:

   Bash

   PowerShell

   RES_SA_ID=$(yc iam service-account get --name k8s-res-sa-$FOLDER_ID --format json | jq .id -r)
   

   $RES_SA_ID = (yc iam service-account get --name k8s-res-sa-$FOLDER_ID --format json | ConvertFrom-Json).id
   

4. Assign the editor role for the folder to the service account:

   yc resource-manager folder add-access-binding \
     --id $FOLDER_ID \
     --role editor \
     --subject serviceAccount:$RES_SA_ID
   

Service account for nodesService account for nodes

To create a service account the nodes will use to pull the required Docker images from the registry:

1. Save the folder ID from your Yandex Cloud CLI profile configuration to a variable:

   Bash

   PowerShell

   FOLDER_ID=$(yc config get folder-id)
   

   $FOLDER_ID = yc config get folder-id
   

2. Create a service account:

   Bash

   PowerShell

   yc iam service-account create --name k8s-node-sa-$FOLDER_ID
   

   yc iam service-account create --name k8s-node-sa-$FOLDER_ID
   

3. Save the service account ID to a variable:

   Bash

   PowerShell

   NODE_SA_ID=$(yc iam service-account get --name k8s-node-sa-$FOLDER_ID --format json | jq .id -r)
   

   $NODE_SA_ID = (yc iam service-account get --name k8s-node-sa-$FOLDER_ID --format json | ConvertFrom-Json).id
   

4. Assign the container-registry.images.puller role for the folder to the service account:

   yc resource-manager folder add-access-binding \
     --id $FOLDER_ID \
     --role container-registry.images.puller \
     --subject serviceAccount:$NODE_SA_ID
   

Service account for the Application Load Balancer ingress controllerService account for the Application Load Balancer ingress controller

1. Save the folder ID from your Yandex Cloud CLI profile configuration to a variable:

   Bash

   PowerShell

   FOLDER_ID=$(yc config get folder-id)
   

   $FOLDER_ID = yc config get folder-id
   

2. Create a service account:

   Bash

   PowerShell

   yc iam service-account create --name k8s-ic-sa-$FOLDER_ID
   

   yc iam service-account create --name k8s-ic-sa-$FOLDER_ID
   

3. Save the service account ID to a variable:

   Bash

   PowerShell

   IC_SA_ID=$(yc iam service-account get --name k8s-ic-sa-$FOLDER_ID --format json | jq .id -r)
   

   $RES_SA_ID = (yc iam service-account get --name k8s-ic-sa-$FOLDER_ID --format json | ConvertFrom-Json).id
   

4. Assign the following roles for the folder to the service account:

   • alb.editor.
   • vpc.publicAdmin.
   • certificate-manager.certificates.downloader.
   • compute.viewer.

   yc resource-manager folder add-access-binding \
     --id $FOLDER_ID \
     --role alb.editor \
     --role vpc.publicAdmin \
     --role certificate-manager.certificates.downloader \
     --role compute.viewer \
     --subject serviceAccount:$IC_SA_ID
   

5. Create an authorized key for the service account and save it to a file named sa-key.json:

   yc iam key create \
     --service-account-id $IC_SA_ID \
     --output sa-key.json
   

Create security groupsCreate security groups

Create security groups for the Managed Service for Kubernetes cluster and its node groups.

Also configure the security groups required for Application Load Balancer.

Set up Kubernetes resourcesSet up Kubernetes resources

Create a Managed Service for Kubernetes clusterCreate a Managed Service for Kubernetes cluster

Create a Managed Service for Kubernetes cluster and specify the previously created service accounts in the --service-account-id and --node-service-account-id parameters and security groups in the --security-group-ids parameter.

yc managed-kubernetes cluster create \
  --name k8s-demo \
  --network-name yc-auto-network \
  --zone ru-central1-a \
  --subnet-name yc-auto-subnet-0 \
  --public-ip \
  --service-account-id $RES_SA_ID \
  --node-service-account-id $NODE_SA_ID \
  --security-group-ids <security_group_IDs>

yc managed-kubernetes cluster create `
  --name k8s-demo `
  --network-name yc-auto-network `
  --zone ru-central1-a `
  --subnet-name yc-auto-subnet-0 `
  --public-ip `
  --service-account-id $RES_SA_ID `
  --node-service-account-id $NODE_SA_ID `
  --security-group-ids <security_group_IDs>

Create a Managed Service for Kubernetes node groupCreate a Managed Service for Kubernetes node group

1. Make sure the Managed Service for Kubernetes cluster was created successfully.

   1. In the management console, select the folder the Managed Service for Kubernetes cluster was created in.
   2. In the list of services, select Managed Service for Kubernetes.
   3. Check that your Managed Service for Kubernetes cluster was created successfully:
      • The Status column should state Running.
      • The State column should state Healthy.

2. Create a Managed Service for Kubernetes node group and specify the previously created security groups in the --network-interface security-group-ids parameter:

   Bash

   PowerShell

   yc managed-kubernetes node-group create \
     --name k8s-demo-ng \
     --cluster-name k8s-demo \
     --platform standard-v3 \
     --cores 2 \
     --memory 4 \
     --core-fraction 50 \
     --disk-type network-ssd \
     --fixed-size 2 \
     --network-interface subnets=yc-auto-subnet-0,ipv4-address=nat,security-group-ids=[<security_group_IDs>] \
     --async
   

   yc managed-kubernetes node-group create `
     --name k8s-demo-ng `
     --cluster-name k8s-demo `
     --platform standard-v3 `
     --cores 2 `
     --memory 4 `
     --core-fraction 50 `
     --disk-type network-ssd `
     --fixed-size 2 `
     --network-interface subnets=yc-auto-subnet-0,ipv4-address=nat,security-group-ids=[<security_group_IDs>] `
     --async
   

Connect to the Managed Service for Kubernetes clusterConnect to the Managed Service for Kubernetes cluster

Install kubect and configure it to work with the new cluster.

Set up Container Registry resourcesSet up Container Registry resources

Create a registryCreate a registry

Create a container registry:

yc container registry create --name yc-auto-cr

Configure a Docker credential helperConfigure a Docker credential helper

To simplify authentication in Container Registry, configure a Docker credential helper. It enables you to use private Yandex Cloud registries without running the docker login command.

To configure a credential helper, run this command:

yc container registry configure-docker

Set up a Docker imageSet up a Docker image

Build a Docker image and push it to the registry.

1. Create a Dockerfile named hello.dockerfile and paste the following lines into it:

   FROM nginx
   CMD echo "Hi, I'm inside"
   

2. Build the Docker image.

   1. Get the ID of the previously created registry and save it to a variable:

      Bash

      PowerShell

      REGISTRY_ID=$(yc container registry get --name yc-auto-cr --format json | jq .id -r)
      

      $REGISTRY_ID = (yc container registry get --name yc-auto-cr --format json | ConvertFrom-Json).id
      

   2. Build the Docker image:

      docker build . -f hello.dockerfile -t cr.yandex/$REGISTRY_ID/nginx:hello
      

   3. Push the Docker image to the registry:

      docker push cr.yandex/$REGISTRY_ID/nginx:hello
      

3. Make sure the image is now in the registry:

   yc container image list
   

   Result:

   +----------------------+---------------------+----------------------------+-------+-----------------+
   |          ID          |       CREATED       |            NAME            | TAGS  | COMPRESSED SIZE |
   +----------------------+---------------------+----------------------------+-------+-----------------+
   | crpa2mf008mp******** | 2019-11-20 11:52:17 | crp71hkgiolp********/nginx | hello | 27.5 MB         |
   +----------------------+---------------------+----------------------------+-------+-----------------+
   

Run the test appRun the test app

Create a pod with the app from the Docker image and make sure no additional authentication in Container Registry was required to push the Docker image.

1. Run the pod with the app from the Docker image:

   kubectl run --attach hello-nginx --image cr.yandex/$REGISTRY_ID/nginx:hello
   

2. Make sure the pod status changed to Running state and get the full pod name.

   kubectl get pods
   

   Result:

   NAME                          READY  STATUS   RESTARTS  AGE
   hello-nginx-5847fb96**-*****  1/1    Running  0         1h
   

3. Check the logs of the container running on that pod:

   kubectl logs hello-nginx-5847fb96**-*****
   

   Result:

   Hi, I'm inside
   

   The pod pulled the Docker image with no additional authentication required on the Container Registry side.

Set up an Application Load BalancerSet up an Application Load Balancer

To set up an Application Load Balancer, follow this guide.

Create a load balancerCreate a load balancer

1. Create a load balancer for Kubernetes services.

   1. Create the ingress.yaml file with the ingress controller manifest:

      ---
      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: alb-demo-tls
        annotations:
          ingress.alb.yc.io/subnets: <list_of_subnet_IDs>
          ingress.alb.yc.io/security-groups: <list_of_security_group_IDs>
          ingress.alb.yc.io/external-ipv4-address: <IP_address_assignment_method>
          ingress.alb.yc.io/group-name: <Ingress_resource_group_name>
      spec:
        tls:
          - hosts:
              - <domain_name>
            secretName: yc-certmgr-cert-id-<TLS_certificate_ID>
        rules:
          - host: <domain_name>
            http:
              paths:
                - pathType: Prefix
                  path: "/"
                  backend:
                    service:
                      name: service-hello
                      port:
                        name: http
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: service-hello
      spec:
        selector:
          run: hello-nginx
        type: NodePort
        ports:
          - name: http
            port: 80
            targetPort: 80
            protocol: TCP
            nodePort: 30080
      

      Where:

      • ingress.alb.yc.io/subnets: Specify one or more subnets for the Application Load Balancer.

      • ingress.alb.yc.io/security-groups: Specify one or more security groups for the Application Load Balancer. If you skip this parameter, the default security group will be used. At least one of the security groups must allow an outgoing TCP connection to port 10501 in the node group subnet or to its security group.

      • ingress.alb.yc.io/external-ipv4-address: To get a new IP address and enable public access to an Application Load Balancer from the internet, specify the previously obtained IP address or set the value to auto.

        If you set auto, deleting the ingress controller will also remove its associated IP address from the cloud. To avoid this, use a reserved IP address.

      • ingress.alb.yc.io/group-name: Specify the group name. The group combines Kubernetes Ingress resources served by a single Application Load Balancer instance.

   2. Create a load balancer:

      kubectl apply -f ingress.yaml
      

   3. Wait until the load balancer is created and assigned a public IP address. This may take several minutes:

      kubectl get ingress alb-demo-tls
      

      The expected result is a non-empty value in the ADDRESS field for the new load balancer:

      NAME          CLASS   HOSTS           ADDRESS     PORTS    AGE
      alb-demo-tls  <none>  <domain_name>  <IP_address>  80, 443  15h
      

      The system will automatically deploy an L7 load balancer based on the load balancer configuration.

2. Follow the https://<domain_name> link to make sure your application is successfully published.

   Note

   If the resource is unavailable at the specified URL, make sure that the security groups for the Managed Service for Kubernetes cluster and its node groups are configured correctly. If any rule is missing, add it.

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:

1. Delete the Managed Service for Kubernetes cluster:

   yc managed-kubernetes cluster delete --name k8s-demo
   

2. Delete the service accounts.

   • Delete the service account for resources:

     yc iam service-account delete --id $RES_SA_ID
     

   • Delete the service account for nodes:

     yc iam service-account delete --id $NODE_SA_ID
     

   • Delete the service account for the load balancer:

     yc iam service-account delete --id $IC_SA_ID
     

3. Delete the Container Registry resources.

   1. Get the ID of the Docker image pushed to the registry:

      Bash

      PowerShell

      IMAGE_ID=$(yc container image list --format json | jq .[0].id -r)
      

      $IMAGE_ID = (yc container image list --format json | ConvertFrom-Json).id
      

   2. Delete the Docker image:

      yc container image delete --id $IMAGE_ID
      

   3. Delete the registry:

      yc container registry delete --name yc-auto-cr
      

4. Delete the L7 load balancer.