Creating a new Kubernetes project in Yandex Cloud

This article describes how to run a new Kubernetes project in Yandex Cloud. An application from Yandex Container Registry is deployed in a Managed Service for Kubernetes cluster and published on the internet via the Yandex Application Load Balancer Ingress controller.

If you need to create a Managed Service for Kubernetes cluster with no internet access, see Creating and configuring a Kubernetes cluster with no internet access.

To launch an app:

1. Create service accounts.
2. Create security groups.
3. Prepare Kubernetes resources.
4. Connect to the Managed Service for Kubernetes cluster.
5. Prepare Container Registry resources.
6. Install Application Load Balancer.
7. Create a load balancer.

If you no longer need the resources you created, delete them.

Required paid resourcesRequired paid resources

The support cost includes:

• Fee for a DNS zone and DNS requests (see Cloud DNS pricing).
• Managed Service for Kubernetes cluster fee: using the master and outgoing traffic (see Managed Service for Kubernetes pricing).
• Cluster nodes (VM) fee: using computing resources, operating system, and storage (see Compute Cloud pricing).
• Fee for Container Registry: using the storage and outgoing traffic (see Container Registry pricing).
• Fee for using the computing resources of the L7 load balancer (see Application Load Balancer pricing).
• Fee for a public IP address for an L7 load balancer (see Virtual Private Cloud pricing).

Getting startedGetting started

1. If you do not have the Yandex Cloud CLI yet, install and initialize it.

   The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

2. Install the jq JSON stream processor.

   sudo apt update && sudo apt install jq
   

Create a network with a subnetCreate a network with a subnet

1. Create a network named yc-auto-network:

   yc vpc network create --name yc-auto-network
   

2. Create a subnet in the ru-central1-a availability zone:

   yc vpc subnet create \
     --name yc-auto-subnet-0 \
     --network-name yc-auto-network \
     --range 192.168.1.0/24 \
     --zone ru-central1-a
   

Register a domain zone and add a certificateRegister a domain zone and add a certificate

1. Register a public domain zone and delegate your domain.

2. If you already have a certificate for the domain zone, add its details to Yandex Certificate Manager.

   If you have no certificate, issue a new Let's Encrypt® certificate and add it to Certificate Manager.

3. Get the certificate ID:

   yc certificate-manager certificate list
   

   Result:

   +-----------------+-------+----------------+---------------------+----------+--------+
   |       ID        | NAME  |    DOMAINS     |      NOT AFTER      |   TYPE   | STATUS |
   +-----------------+-------+----------------+---------------------+----------+--------+
   | <ID> | <name> | <domain_name> | 2022-04-06 17:19:37 | IMPORTED | ISSUED |
   +-----------------+-------+----------------+---------------------+----------+--------+
   

Create service accountsCreate service accounts

For a Managed Service for Kubernetes cluster and load balancer to run, the following service accounts are required:

• Service account with the k8s.clusters.agent and vpc.publicAdmin roles for the folder where the Managed Service for Kubernetes cluster is created. This service account will be used to create the resources required for the Managed Service for Kubernetes cluster.
• Service account with the container-registry.images.puller role for the folder containing the Docker image registry. Nodes will pull the required Docker images from the registry on behalf of this service account.
• For the Application Load Balancer Ingress controller to run, you need service accounts with the following roles:
  • alb.editor: To create the required resources.
  • vpc.publicAdmin: To manage external connectivity.
  • certificate-manager.certificates.downloader: To use certificates registered in Certificate Manager.
  • compute.viewer: To use Managed Service for Kubernetes cluster nodes in the load balancer target groups.

Service account for resourcesService account for resources

To create a service account which will create the resources for the Managed Service for Kubernetes cluster:

1. Write the folder ID from your Yandex Cloud CLI profile configuration to the variable:

   Bash

   PowerShell

   FOLDER_ID=$(yc config get folder-id)
   

   $FOLDER_ID = yc config get folder-id
   

2. Create a service account:

   Bash

   PowerShell

   yc iam service-account create --name k8s-res-sa-$FOLDER_ID
   

   yc iam service-account create --name k8s-res-sa-$FOLDER_ID
   

3. Write the service account ID to the variable:

   Bash

   PowerShell

   RES_SA_ID=$(yc iam service-account get --name k8s-res-sa-$FOLDER_ID --format json | jq .id -r)
   

   $RES_SA_ID = (yc iam service-account get --name k8s-res-sa-$FOLDER_ID --format json | ConvertFrom-Json).id
   

4. Assign the service account the editor role for the folder:

   yc resource-manager folder add-access-binding \
     --id $FOLDER_ID \
     --role editor \
     --subject serviceAccount:$RES_SA_ID
   

Service account for nodesService account for nodes

To create a service account that lets nodes download the necessary Docker images from the registry:

1. Write the folder ID from your Yandex Cloud CLI profile configuration to the variable:

   Bash

   PowerShell

   FOLDER_ID=$(yc config get folder-id)
   

   $FOLDER_ID = yc config get folder-id
   

2. Create a service account:

   Bash

   PowerShell

   yc iam service-account create --name k8s-node-sa-$FOLDER_ID
   

   yc iam service-account create --name k8s-node-sa-$FOLDER_ID
   

3. Write the service account ID to the variable:

   Bash

   PowerShell

   NODE_SA_ID=$(yc iam service-account get --name k8s-node-sa-$FOLDER_ID --format json | jq .id -r)
   

   $NODE_SA_ID = (yc iam service-account get --name k8s-node-sa-$FOLDER_ID --format json | ConvertFrom-Json).id
   

4. Assign the service account the container-registry.images.puller role for the folder:

   yc resource-manager folder add-access-binding \
     --id $FOLDER_ID \
     --role container-registry.images.puller \
     --subject serviceAccount:$NODE_SA_ID
   

Service account required for the Application Load Balancer Ingress controllerService account required for the Application Load Balancer Ingress controller

1. Write the folder ID from your Yandex Cloud CLI profile configuration to the variable:

   Bash

   PowerShell

   FOLDER_ID=$(yc config get folder-id)
   

   $FOLDER_ID = yc config get folder-id
   

2. Create a service account:

   Bash

   PowerShell

   yc iam service-account create --name k8s-ic-sa-$FOLDER_ID
   

   yc iam service-account create --name k8s-ic-sa-$FOLDER_ID
   

3. Write the service account ID to the variable:

   Bash

   PowerShell

   IC_SA_ID=$(yc iam service-account get --name k8s-ic-sa-$FOLDER_ID --format json | jq .id -r)
   

   $RES_SA_ID = (yc iam service-account get --name k8s-ic-sa-$FOLDER_ID --format json | ConvertFrom-Json).id
   

4. Assign the service account the following roles for the folder:

   • alb.editor.
   • vpc.publicAdmin.
   • certificate-manager.certificates.downloader.
   • compute.viewer.

   yc resource-manager folder add-access-binding \
     --id $FOLDER_ID \
     --role alb.editor \
     --role vpc.publicAdmin \
     --role certificate-manager.certificates.downloader \
     --role compute.viewer \
     --subject serviceAccount:$IC_SA_ID
   

5. Create an authorized key for the service account and save it to a file named sa-key.json:

   yc iam key create \
     --service-account-id $IC_SA_ID \
     --output sa-key.json
   

Create security groupsCreate security groups

Create security groups for the Managed Service for Kubernetes cluster and its node groups.

Also configure the security groups required for Application Load Balancer.

Create Kubernetes resourcesCreate Kubernetes resources

Create a Managed Service for Kubernetes clusterCreate a Managed Service for Kubernetes cluster

Create a Managed Service for Kubernetes cluster and specify the previously created service accounts in the --service-account-id and --node-service-account-id flags and security groups in the --security-group-ids flag.

yc managed-kubernetes cluster create \
  --name k8s-demo \
  --network-name yc-auto-network \
  --zone ru-central1-a \
  --subnet-name yc-auto-subnet-0 \
  --public-ip \
  --service-account-id $RES_SA_ID \
  --node-service-account-id $NODE_SA_ID \
  --security-group-ids <security_group_IDs>

yc managed-kubernetes cluster create `
  --name k8s-demo `
  --network-name yc-auto-network `
  --zone ru-central1-a `
  --subnet-name yc-auto-subnet-0 `
  --public-ip `
  --service-account-id $RES_SA_ID `
  --node-service-account-id $NODE_SA_ID `
  --security-group-ids <security_group_IDs>

Create a Managed Service for Kubernetes node groupCreate a Managed Service for Kubernetes node group

1. Make sure the Managed Service for Kubernetes cluster was created successfully.

   1. In the management console, select the folder the Managed Service for Kubernetes cluster was created in.
   2. In the list of services, select Managed Service for Kubernetes.
   3. Check that your Managed Service for Kubernetes cluster was created successfully:
      • The Status column should state Running.
      • The State column should state Healthy.

2. Create a Managed Service for Kubernetes node group and specify the previously created security groups in the --network-interface security-group-ids flag:

   Bash

   PowerShell

   yc managed-kubernetes node-group create \
     --name k8s-demo-ng \
     --cluster-name k8s-demo \
     --platform standard-v3 \
     --cores 2 \
     --memory 4 \
     --core-fraction 50 \
     --disk-type network-ssd \
     --fixed-size 2 \
     --network-interface subnets=yc-auto-subnet-0,ipv4-address=nat,security-group-ids=[<security_group_IDs>] \
     --async
   

   yc managed-kubernetes node-group create `
     --name k8s-demo-ng `
     --cluster-name k8s-demo `
     --platform standard-v3 `
     --cores 2 `
     --memory 4 `
     --core-fraction 50 `
     --disk-type network-ssd `
     --fixed-size 2 `
     --network-interface subnets=yc-auto-subnet-0,ipv4-address=nat,security-group-ids=[<security_group_IDs>] `
     --async
   

Connect to the Managed Service for Kubernetes clusterConnect to the Managed Service for Kubernetes cluster

Install kubect and configure it to work with the new cluster.

Create Container Registry resourcesCreate Container Registry resources

Create a registryCreate a registry

Create a container registry:

yc container registry create --name yc-auto-cr

Configure Docker credential helperConfigure Docker credential helper

To facilitate authentication in Container Registry, configure a Docker credential helper. It enables you to use private Yandex Cloud registries without running the docker login command.

To configure a credential helper, run the following command:

yc container registry configure-docker

Prepare a Docker imagePrepare a Docker image

Build a Docker image and push it to the registry.

1. Create a Dockerfile named hello.dockerfile and add the following lines to it:

   FROM nginx
   CMD echo "Hi, I'm inside"
   

2. Assemble the Docker image.

   1. Get the ID of the previously created registry and write it to the variable:

      Bash

      PowerShell

      REGISTRY_ID=$(yc container registry get --name yc-auto-cr --format json | jq .id -r)
      

      $REGISTRY_ID = (yc container registry get --name yc-auto-cr --format json | ConvertFrom-Json).id
      

   2. Build the Docker image:

      docker build . -f hello.dockerfile -t cr.yandex/$REGISTRY_ID/nginx:hello
      

   3. Push the Docker image to the registry:

      docker push cr.yandex/$REGISTRY_ID/nginx:hello
      

3. Make sure the Docker image was pushed to the registry:

   yc container image list
   

   Result:

   +----------------------+---------------------+----------------------------+-------+-----------------+
   |          ID          |       CREATED       |            NAME            | TAGS  | COMPRESSED SIZE |
   +----------------------+---------------------+----------------------------+-------+-----------------+
   | crpa2mf008mp******** | 2019-11-20 11:52:17 | crp71hkgiolp********/nginx | hello | 27.5 MB         |
   +----------------------+---------------------+----------------------------+-------+-----------------+
   

Run the test appRun the test app

Create a pod with the app from the Docker image and make sure that no additional authentication in Container Registry was required to push the Docker image.

1. Run the pod with the app from the Docker image:

   kubectl run --attach hello-nginx --image cr.yandex/$REGISTRY_ID/nginx:hello
   

2. Make sure the pod has entered the Running state and learn its full name.

   kubectl get pods
   

   Result:

   NAME                          READY  STATUS   RESTARTS  AGE
   hello-nginx-5847fb96**-*****  1/1    Running  0         1h
   

3. Check the logs of the container running on this pod:

   kubectl logs hello-nginx-5847fb96**-*****
   

   Result:

   Hi, I'm inside
   

   The pod pushed the Docker image with no additional authentication on the Container Registry side.

Install Application Load BalancerInstall Application Load Balancer

To install Application Load Balancer, follow this guide.

Create a load balancerCreate a load balancer

1. Create a load balancer for Kubernetes services.

   1. Create the ingress.yaml file with the Ingress controller manifest:

      ---
      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: alb-demo-tls
        annotations:
          ingress.alb.yc.io/subnets: <list_of_subnet_IDs>
          ingress.alb.yc.io/security-groups: <list_of_security_group_IDs>
          ingress.alb.yc.io/external-ipv4-address: <IP_address_assignment_method>
          ingress.alb.yc.io/group-name: <Ingress_resource_group_name>
      spec:
        tls:
          - hosts:
              - <domain_name>
            secretName: yc-certmgr-cert-id-<TLS_certificate_ID>
        rules:
          - host: <domain_name>
            http:
              paths:
                - pathType: Prefix
                  path: "/"
                  backend:
                    service:
                      name: service-hello
                      port:
                        name: http
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: service-hello
      spec:
        selector:
          run: hello-nginx
        type: NodePort
        ports:
          - name: http
            port: 80
            targetPort: 80
            protocol: TCP
            nodePort: 30080
      

      Where:

      • ingress.alb.yc.io/subnets: Specify one or more subnets that Application Load Balancer will work with.

      • ingress.alb.yc.io/security-groups: Specify one or more security groups for Application Load Balancer. If you skip this parameter, the default security group will be used. At least one of the security groups must allow an outgoing TCP connection to port 10501 in the node group subnet or its security group.

      • ingress.alb.yc.io/external-ipv4-address: To get a new IP address or provide public access to Application Load Balancer from the internet, specify the previously obtained IP address or set the value to auto.

        With auto setting, deleting the Ingress controller will also remove its associated IP address from the cloud. To avoid this, use a reserved IP address.

      • ingress.alb.yc.io/group-name: Specify the group name. It groups Kubernetes Ingress resources served by a separate Application Load Balancer instance.

   2. Create a load balancer:

      kubectl apply -f ingress.yaml
      

   3. Wait until the load balancer is created and gets a public IP address. This may take several minutes:

      kubectl get ingress alb-demo-tls
      

      The expected result is a non-empty value in the ADDRESS field for the new load balancer:

      NAME          CLASS   HOSTS           ADDRESS     PORTS    AGE
      alb-demo-tls  <none>  <domain_name>  <IP_address>  80,443  15h
      

      Based on the load balancer configuration, an L7 load balancer will be automatically deployed.

2. Follow the https://<domain_name> link and make sure that your application is successfully published.

   Note

   If the resource is unavailable at the specified URL, make sure that the security groups for the Managed Service for Kubernetes cluster and its node groups are configured correctly. If any rule is missing, add it.

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need:

1. Delete the Managed Service for Kubernetes cluster:

   yc managed-kubernetes cluster delete --name k8s-demo
   

2. Delete the service accounts.

   • Delete the service account created for resources:

     yc iam service-account delete --id $RES_SA_ID
     

   • Delete the service account created for nodes:

     yc iam service-account delete --id $NODE_SA_ID
     

   • Delete the service account created for the load balancer:

     yc iam service-account delete --id $IC_SA_ID
     

3. Delete resources Container Registry.

   1. Find out the ID of the Docker image pushed to the registry:

      Bash

      PowerShell

      IMAGE_ID=$(yc container image list --format json | jq .[0].id -r)
      

      $IMAGE_ID = (yc container image list --format json | ConvertFrom-Json).id
      

   2. Delete the Docker image:

      yc container image delete --id $IMAGE_ID
      

   3. Delete the registry:

      yc container registry delete --name yc-auto-cr
      

4. Delete the L7 load balancer.