Granting access to an application running in a Kubernetes cluster

The example below uses a Kubernetes application that responds to HTTP requests on port 8080. To provide access to the application, use public or internal services. Their IP addresses do not change, unlike the addresses of pods and cluster nodes.

To publish an application, use a LoadBalancer type service. You can set up two types of access:

• Public IP access with an external Yandex Network Load Balancer.

• Access from internal networks by IP address with an internal network load balancer.

  The application will be available:

  • From Yandex Virtual Private Cloud subnets.
  • From your corporate internal subnets connected to Yandex Cloud via Yandex Cloud Interconnect.
  • Via VPN.

When using an external load balancer, you can specify a static public IP address in the loadBalancerIP field. You need to reserve such an address in advance. When reserving a public IP address, you can enable DDoS protection. If you do not specify a static public IP address, the network load balancer will get a dynamic public IP address.

When using an internal load balancer, you can specify an internal IP address. Make sure the specified internal IP address is not assigned to some other resource in the same cloud network.

To ensure access to the Kubernetes application:

1. Get ready.
2. Create a Kubernetes app.
3. Create a LoadBalancer type service.
4. Check application availability.
5. Optionally, create a NetworkPolicy object.

If you no longer need the resources you created, delete them.

Getting startedGetting started

1. Install kubect and configure it to work with the new cluster.

2. Set up your infrastructure:

   Manually

   Terraform

   1. Create a cloud network and subnet.

   2. Create a service account with the k8s.clusters.agent, vpc.publicAdmin, and load-balancer.admin role. It needs the load-balancer.admin role to create a network load balancer.

   3. Create security groups for the Managed Service for Kubernetes cluster and its node groups.

      Warning

      The configuration of security groups determines performance and availability of the cluster and the services and applications running in it.

   4. Create a Managed Service for Kubernetes cluster and node group with public internet access and preconfigured security groups.

   1. If you do not have Terraform yet, install it.

   2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

   3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

   4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

   5. Download the k8s-load-balancer.tf Managed Service for Kubernetes cluster configuration file to the same working directory. This file describes:

      • Network.

      • Subnet.

      • Managed Service for Kubernetes cluster.

      • Service account for the Managed Service for Kubernetes cluster and node group.

      • Security groups which contain rules required for the Managed Service for Kubernetes cluster and its node groups.

        Warning

        The configuration of security groups determines performance and availability of the cluster and the services and applications running in it.

   6. Specify the following in the configuration file:

      • Folder ID.
      • Kubernetes version for the Managed Service for Kubernetes cluster and node groups.
      • Name of the Managed Service for Kubernetes cluster service account.

   7. Validate your Terraform configuration files using this command:

      terraform validate
      

      Terraform will display any configuration errors detected in your files.

   8. Create the required infrastructure:

      1. Run this command to view the planned changes:

         terraform plan
         

         If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

      2. If everything looks correct, apply the changes:

         1. Run this command:

            terraform apply
            

         2. Confirm updating the resources.

         3. Wait for the operation to complete.

      All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

      Timeouts

      The Terraform provider sets time limits for operations with Managed Service for Kubernetes cluster and node group:

      • Creating and editing a cluster: 30 minutes.
      • Creating and updating a node group: 60 minutes.
      • Deleting a node group: 20 minutes.

      Operations in excess of this time will be interrupted.

      How do I modify these limits?

      Add the timeouts sections (the yandex_kubernetes_cluster and yandex_kubernetes_node_group resources, respectively) to the cluster and node group description.

      Here is an example:

      resource "yandex_kubernetes_node_group" "<node_group_name>" {
        ...
        timeouts {
          create = "1h30m"
          update = "1h30m"
          delete = "30m"
        }
      }
      

Create a Kubernetes appCreate a Kubernetes app

1. Create a file named hello.yaml and add the Deployment resource specification to it to create the application:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: hello
   spec:
     replicas: 2
     selector:
       matchLabels:
         app: hello
     template:
       metadata:
         labels:
           app: hello
       spec:
         containers:
         - name: hello-app
           image: cr.yandex/crpjd37scfv653nl11i9/hello:1.1
   

2. Create an application:

   kubectl apply -f hello.yaml
   

3. Make sure the application was created:

   kubectl get deployment 
   

   Result:

   NAME    READY   UP-TO-DATE   AVAILABLE   AGE
   hello   2/2     2            2           17h
   

Create a LoadBalancer type serviceCreate a LoadBalancer type service

When you create a LoadBalancer type service, the Yandex Cloud controller installs a network load balancer in your folder. It is charged based on the Network Load Balancer pricing policy.

To create a LoadBalancer type service:

1. Select and prepare the service specification based on the required load balancer type:

   External load balancer

   Internal load balancer

   1. Create a file named load-balancer.yaml and add the following service specification to it:

      apiVersion: v1
      kind: Service
      metadata:
        name: hello
      spec:
        type: LoadBalancer
        ports:
        - port: <application_port>
          name: plaintext
          targetPort: 8080
        selector:
          <Kubernetes_labels>
      

      In the specification, indicate:

      • spec.ports.port: Application port.

        The example assumes the Kubernetes application is available over HTTP, so specify 80 as the value. If you need to access the application over HTTPS, set the value to 443.

      • spec.selector: Kubernetes labels specified in the spec.selector.matchLabels field of the Deployment resource.

        Specify the app: hello label as it is used in the Deployment resource you created previously.

      For details on the specification, see this reference.

   2. Optionally, reserve a static public IP address and add it to the specification:

      ...
      spec:
        loadBalancerIP: <static_IP_address>
        ...
      

      Note

      If you do not specify a static IP address, the network load balancer will get a dynamic IP address.

   1. Create a file named load-balancer.yaml and add the following service specification to it:

      apiVersion: v1
      kind: Service
      metadata:
        name: hello
        annotations:
          yandex.cloud/load-balancer-type: internal
          yandex.cloud/subnet-id: <cluster_subnet_ID>
      spec:
        type: LoadBalancer
        ports:
        - port: <application_port>
          name: plaintext
          targetPort: 8080
        selector:
          <Kubernetes_labels>
      

      In the specification, indicate:

      • yandex.cloud/subnet-id: ID of the subnet hosting the cluster. You can get the ID together with the subnet information.

      • spec.ports.port: Application port.

        The example assumes the Kubernetes application is available over HTTP, so specify 80 as the value. If you need to access the application over HTTPS, set the value to 443.

      • spec.selector: Kubernetes labels specified in the spec.selector.matchLabels field of the Deployment resource.

        Specify the app: hello label as it is used in the Deployment resource you created previously.

      For details on the specification, see this reference.

   2. Optionally, reserve a static private IP address and add it to the specification:

      ...
      spec:
        loadBalancerIP: <static_IP_address>
        ...
      

      Note

      If you do not specify a static IP address, the network load balancer will get a dynamic IP address.

2. Optionally, add a traffic management policy:

   ...
   spec:
     externalTrafficPolicy: <Cluster_or_Local>
     ...
   

   The possible values are:

   • Cluster: Traffic is routed to different Kubernetes nodes (default). As a result, traffic is distributed evenly; however, there are certain drawbacks:

     • The packet may come to one node's proxy and get rerouted to another node leading to delays in performing operations and sending packets.
     • The pod that receives the packet sees the IP address of the proxying node rather than the one of the client. As a result, the client IP address is not preserved.

   • Local: Traffic is proxied and distributed across pods on the same node. The traffic is routed to the node via the port specified in the service of the LoadBalancer or NodePort type.

     The traffic comes to a specific node, so it is distributed unevenly across nodes; however, the client IP address is preserved.

   For more information about policies for external traffic management, see this Kubernetes guide.

3. Optionally, enable node health checks.

   LoadBalancer type services in Managed Service for Kubernetes can run health check requests for a target group. Based on the received metrics, Managed Service for Kubernetes decides if the nodes are available.

   To enable node availability checks, specify the following annotations in the service specification:

   ...
   metadata:
     ...
     annotations:
       yandex.cloud/load-balancer-healthcheck-healthy-threshold: "2"
       yandex.cloud/load-balancer-healthcheck-interval: "2s"
   

   Annotations used:

   • yandex.cloud/load-balancer-healthcheck-healthy-threshold: Number of consecutive successful checks to consider a cluster node available.
   • yandex.cloud/load-balancer-healthcheck-interval: Check interval in seconds.

4. Create a network load balancer:

   kubectl apply -f load-balancer.yaml
   

Check application availabilityCheck application availability

1. View information about the network load balancer you created and get its IP address:

   Management console

   kubectl

   1. In the management console, select your default folder.

   2. Go to Network Load Balancer.

   3. The Load balancers tab shows the network load balancer with the k8s prefix in the name and the unique ID of your Kubernetes cluster in the description.

      Copy the balancer address in the IP address column.

   kubectl describe service hello
   

   Result:

   Name:                     hello
   Namespace:                default
   Labels:                   <none>
   Annotations:              <none>
   Selector:                 app=hello
   Type:                     LoadBalancer
   IP:                       172.20.169.7
   LoadBalancer Ingress:     130.193.50.111
   Port:                     plaintext 80/TCP
   TargetPort:               8080/TCP
   NodePort:                 plaintext 32302/TCP
   Endpoints:                10.1.130.4:8080
   Session Affinity:         None
   External Traffic Policy:  Cluster
   Events:
     Type    Reason                Age    From                Message
     ----    ------                ----   ----                -------
     Normal  EnsuringLoadBalancer  2m43s  service-controller  Ensuring load balancer
     Normal  EnsuredLoadBalancer   2m17s  service-controller  Ensured load balancer
   

   Copy the balancer address from the LoadBalancer Ingress field.

2. Make sure the application is available. The verification steps depend on your load balancer type:

   External load balancer

   Internal load balancer

   Run this command:

   curl http://<load_balancer_IP_address>
   

   Result:

   Hello, world!
   Running in 'hello-********'
   

   1. In the Managed Service for Kubernetes cluster subnet, create a Linux VM.

      Since you have deployed an internal network load balancer, you can only test access to the Kubernetes application from the cluster subnet.

   2. Connect to the VM over SSH.

   3. Check availability of the Kubernetes application:

      curl http://<load_balancer_IP_address>
      

      Result:

      Hello, world!
      Running in 'hello-********'
      

   Note

   If you cannot access the resource at the specified URL, make sure the security groups for the Managed Service for Kubernetes cluster and its node groups are configured correctly. If a rule is missing, add it.

Optionally, create a NetworkPolicy objectOptionally, create a NetworkPolicy object

To connect to services published via Network Load Balancer from specific IP addresses, enable network policies in the cluster. To set up access via the load balancer, create a NetworkPolicy object with an Ingress type policy.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: whitelist-netpol
  namespace: ns-example
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
  - Ingress
  ingress:
  - from:
    # Address ranges used by the load balancer to health check nodes.
    - ipBlock:
        cidr: 198.18.235.0/24
    - ipBlock:
        cidr: 198.18.248.0/24
    # Pod address ranges.
    - ipBlock:
        cidr: 172.16.1.0/12
    - ipBlock:
        cidr: 172.16.2.0/12

For more information, see the NetworkPolicy resource reference.

Delete the resources you createdDelete the resources you created

Delete the resources you no longer need to avoid paying for them:

1. Delete the resources depending on how you created them:

   Manually

   Terraform

   Delete the Managed Service for Kubernetes cluster.

   1. In the terminal window, go to the directory containing the infrastructure plan.

      Warning

      Make sure the directory has no Terraform manifests with the resources you want to keep. Terraform deletes all resources that were created using the manifests in the current directory.

   2. Delete resources:

      1. Run this command:

         terraform destroy
         

      2. Confirm deleting the resources and wait for the operation to complete.

      All the resources described in the Terraform manifests will be deleted.

2. If you used static public IP addresses to access your Managed Service for Kubernetes cluster or nodes, release and delete them.