Creating a new Kubernetes project in Yandex Cloud

This article describes how to run a new Kubernetes project in Yandex Cloud. An application from Yandex Container Registry is deployed in a Managed Service for Kubernetes cluster and published on the internet via the Yandex Application Load Balancer Ingress controller.

If you need to create a Managed Service for Kubernetes cluster with no internet access, see the Creating and configuring a Managed Service for Kubernetes cluster with no internet access section.

To launch an app:

1. Create service accounts
2. Create security groups
3. Prepare Kubernetes resources
4. Connect to the Managed Service for Kubernetes cluster
5. Create Container Registry resources
6. Install Application Load Balancer
7. Create a load balancer

If you no longer need the resources you created, delete them.

Getting startedGetting started

1. If you do not have the Yandex Cloud command line interface yet, install and initialize it.

   The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

2. Install the jq JSON stream processor:

   sudo apt update && sudo apt install jq
   

Create a network and subnetCreate a network and subnet

1. Create a network named yc-auto-network:

   yc vpc network create --name yc-auto-network
   

2. Create a subnet in the ru-central1-a availability zone:

   yc vpc subnet create \
     --name yc-auto-subnet-0 \
     --network-name yc-auto-network \
     --range 192.168.1.0/24 \
     --zone ru-central1-a
   

Register a domain zone and add a certificateRegister a domain zone and add a certificate

1. Register a public domain zone and delegate your domain.

2. If you already have a certificate for the domain zone, add its details to Yandex Certificate Manager.

   If you have no certificate, issue a new Let's Encrypt® certificate and add it to Certificate Manager.

3. Get the certificate ID:

   yc certificate-manager certificate list
   

   Result:

   +------+--------+---------------+---------------------+----------+--------+
   |  ID  |  NAME  |    DOMAINS    |      NOT AFTER      |   TYPE   | STATUS |
   +------+--------+---------------+---------------------+----------+--------+
   | <ID> | <name> | <domain_name> | 2022-04-06 17:19:37 | IMPORTED | ISSUED |
   +------+--------+---------------+---------------------+----------+--------+
   

Create service accountsCreate service accounts

For a Managed Service for Kubernetes cluster and load balancer to run, the following service accounts are required:

• Service account with the k8s.clusters.agent and vpc.publicAdmin roles for the folder where the Managed Service for Kubernetes cluster is created. This service account will be used to create the resources required for the Managed Service for Kubernetes cluster.
• Service account with the container-registry.images.puller role for the folder containing the Docker image registry. Nodes will pull the required Docker images from the registry on behalf of this service account.
• For the Application Load Balancer Ingress controller to run, you need service accounts with the following roles:
  • alb.editor: To create the required resources.
  • vpc.publicAdmin: To manage external connectivity.
  • certificate-manager.certificates.downloader: To use certificates registered in Certificate Manager.
  • compute.viewer: To use Managed Service for Kubernetes cluster nodes in load balancer target groups.

Service account for resourcesService account for resources

To create a service account which will create the resources for the Managed Service for Kubernetes cluster:

1. Write the folder ID from your Yandex Cloud CLI profile configuration to the variable:

   Bash

   PowerShell

   FOLDER_ID=$(yc config get folder-id)
   

   $FOLDER_ID = yc config get folder-id
   

2. Create a service account:

   Bash

   PowerShell

   yc iam service-account create --name k8s-res-sa-$FOLDER_ID
   

   yc iam service-account create --name k8s-res-sa-$FOLDER_ID
   

3. Write the service account ID to the variable:

   Bash

   PowerShell

   RES_SA_ID=$(yc iam service-account get --name k8s-res-sa-$FOLDER_ID --format json | jq .id -r)
   

   $RES_SA_ID = (yc iam service-account get --name k8s-res-sa-$FOLDER_ID --format json | ConvertFrom-Json).id
   

4. Assign the service account the editor role for the folder.

   yc resource-manager folder add-access-binding \
     --id $FOLDER_ID \
     --role editor \
     --subject serviceAccount:$RES_SA_ID
   

Service account for nodesService account for nodes

To create a service account that lets nodes download the necessary Docker images from the registry:

1. Write the folder ID from your Yandex Cloud CLI profile configuration to the variable:

   Bash

   PowerShell

   FOLDER_ID=$(yc config get folder-id)
   

   $FOLDER_ID = yc config get folder-id
   

2. Create a service account:

   Bash

   PowerShell

   yc iam service-account create --name k8s-node-sa-$FOLDER_ID
   

   yc iam service-account create --name k8s-node-sa-$FOLDER_ID
   

3. Write the service account ID to the variable:

   Bash

   PowerShell

   NODE_SA_ID=$(yc iam service-account get --name k8s-node-sa-$FOLDER_ID --format json | jq .id -r)
   

   $NODE_SA_ID = (yc iam service-account get --name k8s-node-sa-$FOLDER_ID --format json | ConvertFrom-Json).id
   

4. Assign the service account the container-registry.images.puller role for the folder.

   yc resource-manager folder add-access-binding \
     --id $FOLDER_ID \
     --role container-registry.images.puller \
     --subject serviceAccount:$NODE_SA_ID
   

Service account required for the Application Load Balancer Ingress controller to runService account required for the Application Load Balancer Ingress controller to run

1. Write the folder ID from your Yandex Cloud CLI profile configuration to the variable:

   Bash

   PowerShell

   FOLDER_ID=$(yc config get folder-id)
   

   $FOLDER_ID = yc config get folder-id
   

2. Create a service account:

   Bash

   PowerShell

   yc iam service-account create --name k8s-ic-sa-$FOLDER_ID
   

   yc iam service-account create --name k8s-ic-sa-$FOLDER_ID
   

3. Write the service account ID to the variable:

   Bash

   PowerShell

   IC_SA_ID=$(yc iam service-account get --name k8s-ic-sa-$FOLDER_ID --format json | jq .id -r)
   

   $RES_SA_ID = (yc iam service-account get --name k8s-ic-sa-$FOLDER_ID --format json | ConvertFrom-Json).id
   

4. Assign the service account the following roles for the folder:

   • alb.editor​.
   • vpc.publicAdmin​.
   • certificate-manager.certificates.downloader.
   • compute.viewer.

   yc resource-manager folder add-access-binding \
     --id $FOLDER_ID \
     --role alb.editor \
     --role vpc.publicAdmin \
     --role certificate-manager.certificates.downloader \
     --role compute.viewer \
     --subject serviceAccount:$IC_SA_ID
   

5. Create an authorized key for the service account and save it to a file named sa-key.json:

   yc iam key create \
     --service-account-id $IC_SA_ID \
     --output sa-key.json
   

Create security groupsCreate security groups

Create security groups for the Managed Service for Kubernetes cluster and its node groups.

Also configure the security groups required for Application Load Balancer.

Prepare Kubernetes resourcesPrepare Kubernetes resources

Create a Managed Service for Kubernetes clusterCreate a Managed Service for Kubernetes cluster

Create a Managed Service for Kubernetes cluster and specify the previously created service accounts in the --service-account-id and --node-service-account-id flags and security groups in the --security-group-ids flag.

yc managed-kubernetes cluster create \
  --name k8s-demo \
  --network-name yc-auto-network \
  --zone ru-central1-a \
  --subnet-name yc-auto-subnet-0 \
  --public-ip \
  --service-account-id $RES_SA_ID \
  --node-service-account-id $NODE_SA_ID \
  --security-group-ids <security_group_IDs>

yc managed-kubernetes cluster create `
  --name k8s-demo `
  --network-name yc-auto-network `
  --zone ru-central1-a `
  --subnet-name yc-auto-subnet-0 `
  --public-ip `
  --service-account-id $RES_SA_ID `
  --node-service-account-id $NODE_SA_ID `
  --security-group-ids <security_group_IDs>

Create a Managed Service for Kubernetes node groupCreate a Managed Service for Kubernetes node group

1. Make sure the Managed Service for Kubernetes cluster was created.

   1. In the management console, select the folder where the Managed Service for Kubernetes cluster was created.
   2. In the list of services, select Managed Service for Kubernetes.
   3. Check that your Managed Service for Kubernetes cluster was created successfully:
      • Look for Running in the Status column.
      • Look for Healthy in the State column.

2. Create a Managed Service for Kubernetes node group and specify the previously created security groups in the --network-interface security-group-groups-ids flag:

   Bash

   PowerShell

   yc managed-kubernetes node-group create \
     --name k8s-demo-ng \
     --cluster-name k8s-demo \
     --platform standard-v3 \
     --cores 2 \
     --memory 4 \
     --core-fraction 50 \
     --disk-type network-ssd \
     --fixed-size 2 \
     --network-interface subnets=yc-auto-subnet-0,ipv4-address=nat,security-group-ids=[<security_group_IDs>] \
     --async
   

   yc managed-kubernetes node-group create `
     --name k8s-demo-ng `
     --cluster-name k8s-demo `
     --platform standard-v3 `
     --cores 2 `
     --memory 4 `
     --core-fraction 50 `
     --disk-type network-ssd `
     --fixed-size 2 `
     --network-interface subnets=yc-auto-subnet-0,ipv4-address=nat,security-group-ids=[<security_group_IDs>] `
     --async
   

Connect to the Managed Service for Kubernetes clusterConnect to the Managed Service for Kubernetes cluster

Install kubectl and configure it to work with the created cluster.

Create Container Registry resourcesCreate Container Registry resources

Create a registryCreate a registry

Create a container registry:

yc container registry create --name yc-auto-cr

Configure Docker credential helperConfigure Docker credential helper

To facilitate authentication in Container Registry, configure a Docker credential helper. It enables you to use private Yandex Cloud registries without running the docker logincommand.

To configure a credential helper, run the following command:

yc container registry configure-docker

Prepare a Docker imagePrepare a Docker image

Build a Docker image and push it to the registry.

1. Create a Dockerfile named hello.dockerfile and add the following lines to it:

   FROM nginx
   CMD echo "Hi, I'm inside"
   

2. Assemble the Docker image.

   1. Get the ID of the previously created registry and write it to the variable:

      Bash

      PowerShell

      REGISTRY_ID=$(yc container registry get --name yc-auto-cr --format json | jq .id -r)
      

      $REGISTRY_ID = (yc container registry get --name yc-auto-cr --format json | ConvertFrom-Json).id
      

   2. Build the Docker image:

      docker build . -f hello.dockerfile -t cr.yandex/$REGISTRY_ID/nginx:hello
      

   3. Push the Docker image to the registry:

      docker push cr.yandex/$REGISTRY_ID/nginx:hello
      

3. Make sure the Docker image was pushed to the registry:

   yc container image list
   

   Result:

   +----------------------+---------------------+----------------------------+-------+-----------------+
   |          ID          |       CREATED       |            NAME            | TAGS  | COMPRESSED SIZE |
   +----------------------+---------------------+----------------------------+-------+-----------------+
   | crpa2mf008mp******** | 2019-11-20 11:52:17 | crp71hkgiolp********/nginx | hello | 27.5 MB         |
   +----------------------+---------------------+----------------------------+-------+-----------------+
   

Run the test appRun the test app

Create a pod with the app from the Docker image and make sure that no additional authentication in Container Registry was required to push the Docker image.

1. Run the pod with the app from the Docker image:

   kubectl run --attach hello-nginx --image cr.yandex/$REGISTRY_ID/nginx:hello
   

2. Check that the pod's state is Running and get its full name:

   kubectl get pods
   

   Result:

   NAME                          READY  STATUS   RESTARTS  AGE
   hello-nginx-5847fb96**-*****  1/1    Running  0         1h
   

3. Check the logs of the container running on this pod:

   kubectl logs hello-nginx-5847fb96**-*****
   

   Result:

   Hi, I'm inside
   

   The pod pushed the Docker image with no additional authentication on the Container Registry side.

Install Application Load BalancerInstall Application Load Balancer

To install Application Load Balancer, follow this guide.

Create a load balancerCreate a load balancer

1. Create a load balancer for Kubernetes services.

   1. Create an ingress.yaml file with the Ingress controller manifest:

      ---
      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: alb-demo-tls
        annotations:
          ingress.alb.yc.io/subnets: <list_of_subnet_IDs>
          ingress.alb.yc.io/security-groups: <list_of_security_group_IDs>
          ingress.alb.yc.io/external-ipv4-address: <IP_address_assignment_method>
          ingress.alb.yc.io/group-name: <Ingress_group_name>
      spec:
        tls:
          - hosts:
              - <domain_name>
            secretName: yc-certmgr-cert-id-<TLS_certificate_ID>
        rules:
          - host: <domain_name>
            http:
              paths:
                - pathType: Prefix
                  path: "/"
                  backend:
                    service:
                      name: service-hello
                      port:
                        name: http
      ---
      apiVersion: v1
        kind: Service
        metadata:
          name: service-hello
        spec:
          selector:
            run: hello-nginx
          type: NodePort
          ports:
            - name: http
              port: 80
              targetPort: 80
              protocol: TCP
              nodePort: 30080
      

      Where:

      • ingress.alb.yc.io/subnets: Specify one or more subnets that Application Load Balancer is going to work with.

      • ingress.alb.yc.io/security-groups: Specify one or more security groups for Application Load Balancer. If you skip this parameter, the default security group will be used. At least one of the security groups must allow an outgoing TCP connection to port 10501 in the node group subnet or its security group.

      • ingress.alb.yc.io/external-ipv4-address: To get a new IP or provide public access to Application Load Balancer from the internet, specify the previously obtained IP address or set the value to auto.

        If you set auto, deleting the Ingress controller will also delete the IP address from the cloud. To avoid this, use an existing reserved IP address.

      • ingress.alb.yc.io/group-name: Specify the group name. It groups Kubernetes Ingress resources served by a separate Application Load Balancer instance.

   2. Create a load balancer:

      kubectl apply -f ingress.yaml
      

   3. Wait until the load balancer is created and assigned a public IP address. This may take several minutes:

      kubectl get ingress alb-demo-tls
      

      The expected result is a non-empty value in the ADDRESS field for the created load balancer:

      NAME          CLASS   HOSTS           ADDRESS      PORTS    AGE
      alb-demo-tls  <none>  <domain_name>   <IP_address> 80, 443  15h
      

      Based on the load balancer configuration, an L7 load balancer will be automatically deployed.

2. Follow the https://<domain_name> link and make sure that your application is published.

   Note

   If the resource is unavailable at the specified URL, make sure that the security groups for the Managed Service for Kubernetes cluster and its node groups are configured correctly. If any rule is missing, add it.

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need:

1. Delete the Managed Service for Kubernetes cluster:

   yc managed-kubernetes cluster delete --name k8s-demo
   

2. Delete the service accounts.

   • Delete the service account created for resources:

     yc iam service-account delete --id $RES_SA_ID
     

   • Delete the service account created for nodes:

     yc iam service-account delete --id $NODE_SA_ID
     

   • Delete the service account created for the load balancer:

     yc iam service-account delete --id $IC_SA_ID
     

3. Delete resources Container Registry.

   1. Find out the ID of the Docker image pushed to the registry:

      Bash

      PowerShell

      IMAGE_ID=$(yc container image list --format json | jq .[0].id -r)
      

      $IMAGE_ID = (yc container image list --format json | ConvertFrom-Json).id
      

   2. Delete the Docker image:

      yc container image delete --id $IMAGE_ID
      

   3. Delete the registry:

      yc container registry delete --name yc-auto-cr
      

4. Delete the L7 load balancer.