Routing Application Load Balancer traffic directly to Yandex Managed Service for Kubernetes cluster pods with Gwin

By default, Application Load Balancer routes traffic to the service’s NodePort, while Yandex Managed Service for Kubernetes redirects it to one of the pods using kube-proxy. The workflow is as follows:

When routing directly into the pods, Gwin registers the IP addresses of the pods in the Application Load Balancer target group and the balancer sends the request directly to the pod:

This removes the intermediate node hop, reduces latency, offloads nodes, and allows Application Load Balancer to balance requests between pods directly.

Comparison of routing to pods and to NodePortComparison of routing to pods and to NodePort

If a pod crashes or a node becomes unavailable while using direct pod routing, Application Load Balancer may still route some traffic to that target. Until health check marks the target as unhealthy, 5xx errors or timeouts may occur.

With NodePort routing, the connection to the crashed pod is terminated, and Kubernetes routes new connections to the remaining pods. If the app tends to crash often and cannot handle that window, consider NodePort routing or implementing retry logic on the client side to smooth out short-lived failures.

Specifics of working with Cilium in tunnel modeSpecifics of working with Cilium in tunnel mode

In Managed Service for Kubernetes, in clusters with the Cilium network controller in tunnel mode, pod IP addresses are not directly accessible from the network. Pod CIDR ranges live only in a logical network and are not routed by Virtual Private Cloud. Therefore, Application Load Balancer cannot directly access pod IP addresses.

To link routes to pod networks through cluster nodes and the Application Load Balancer subnet, Gwin creates a route table. Each owner gets one table, marked with the gwin-owner: <owner> label:

• If the Application Load Balancer subnet already contains this owner’s table, it is taken under management.
• If no Application Load Balancer subnet contains the table, a new one is created and attached to all subnets.
• If the Application Load Balancer subnet contains a table without the owner label, Gwin stops with the PodRouteSubnetConflict event.
• A table that you no longer need is automatically detached and deleted. This occurs, for example, when the last service that uses pods as a target is deleted.

For proper routing to the pods in a cluster with Cilium, the following conditions must be met:

• For Application Load Balancer, the subnet is separated from cluster nodes. Do not create Application Load Balancer in cluster node subnets.
• The Gwin service account is assigned the following roles:
  • vpc.privateAdmin: For managing route tables and subnet associations.
  • k8s.viewer: For finding a cluster.
• There are quotas for creating static routes (vpc.staticRoutes.count). Each node has its own pod-CIDR, so you need one static route per node. The default quota is 256 routes per cloud.
• The pod CIDR does not overlap with subnet ranges. Two conflicting routes will appear for the same IP address.

Even if all conditions are met, Gwin does not begin to manage the routes until it is satisfied that:

1. The Gateway API or Ingress resources have a link to a service that uses pods as its target.
2. The cluster does need internal routes. Gwin checks that Cilium is in tunnel mode. In clusters with initially routable CIDR ranges of pods, such as Calico or Cilium with routing_mode=direct, Gwin remains on hold.
3. Each Application Load Balancer subnet is dedicated to the load balancer and not used by any cluster node group. If the subnet is shared, Gwin generates the PodRouteSubnetConflict event and does not create or update route tables.

Configuring routing to podsConfiguring routing to pods

An HTTP application is used as an example for the minimal routing setup to pods. The endpoint of the GET /healthz app returns 200 when the pod is ready to accept traffic.

If you have not installed Gwin yet, follow this guide. Next, set up the application using the guide provided below. Multiple Gwin installations or clusters that share a network are not supported.

Create a gatewayCreate a gateway

To determine how Application Load Balancer will receive incoming traffic, create a Gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: example-gateway
  annotations:
    gwin.yandex.cloud/subnets: <subnet_ID>
    gwin.yandex.cloud/securityGroups: <security_group_ID>
spec:
  gatewayClassName: gwin-default
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: Same

Where:

• gwin.yandex.cloud/subnets: Subnet in which you will create Application Load Balancer. It is important that this subnet is not occupied by cluster nodes. This is especially critical when working with Cilium in tunnel mode.
• gwin.yandex.cloud/securityGroups: Security group configured on the network with the load balancer.
• gatewayClassName: gwin-default: Instructing the Gwin controller to process this gateway and create a load balancer for it.
• listeners: Description of exactly how the load balancer will listen for traffic: protocol, port, and allowed routes.

Set up deploymentSet up deployment

Deployment manages the lifecycle of pods: their creation, updating, and restarting. Routing to pods critically depends on settings that synchronize the pod’s state in Kubernetes with its registration in Application Load Balancer.

Configure Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
spec:
  selector:
    matchLabels:
      app: example-app
  template:
    metadata:
      labels:
        app: example-app
    spec:
      readinessGates:
        - conditionType: gwin.yandex.cloud/load-balanced
      terminationGracePeriodSeconds: 30
      containers:
        - name: app
          image: example-app:latest
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
          lifecycle:
            preStop:
              exec:
                command: ["sh", "-c", "sleep 30"]

Where:

• readinessGates: Condition for the pod to be considered ready. The pod will not switch to Ready until Gwin registers it in the Application Load Balancer target group. This prevents traffic from being sent to a pod that is not yet visible to the load balancer.
• terminationGracePeriodSeconds: 30 and preStop: sleep 30: Time for which the pod remains alive after receiving the signal of completion. This is needed so that Gwin can remove the pod from the Application Load Balancer target group before the application stops. Without this, the pod may continue receiving requests even after completion has started.
• readinessProbe: Checking the application’s readiness to receive traffic. An equivalent health check will be configured separately for Application Load Balancer to avoid sending requests to unready pods.

Configure serviceConfigure service

Service groups the pods and provides them with a stable network address. Clearly indicate that you want to direct traffic to the pods instead of NodePort:

apiVersion: v1
kind: Service
metadata:
  name: example-app
  annotations:
    gwin.yandex.cloud/targets.type: Pod
    gwin.yandex.cloud/targets.cidrs: <pod-CIDR>
    gwin.yandex.cloud/targets.albZoneMatch: "false"
spec:
  type: ClusterIP
  selector:
    app: example-app
  ports:
    - name: http
      port: 80
      targetPort: 8080

Where:

• gwin.yandex.cloud/targets.type: Pod: Key annotation that instructs Gwin to route traffic directly to the pods.
• gwin.yandex.cloud/targets.cidrs: This annotation ensures that if a cluster has pods with IP addresses from different networks, only pods from the specified CIDR will be included in the Application Load Balancer target group.
• gwin.yandex.cloud/targets.albZoneMatch: "false": This annotation allows using pods located in zones where no Application Load Balancer instances are present. We recommend placing Application Load Balancer in all zones with pods; however, this setting provides flexibility for special cases.
• type: ClusterIP: Default type of service that does not create external IPs but allows Gwin to extract pod IPs for registration in Application Load Balancer.

Create an HTTPRouteCreate an HTTPRoute

HTTPRoute determines routing at the application level. Create an HTTPRoute:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: example-route
spec:
  parentRefs:
    - name: example-gateway
  hostnames:
    - app.example.com
  rules:
    - backendRefs:
        - name: example-app
          port: 80

Where:

• parentRefs: Links the route to a specific gateway. Traffic coming to this gateway will be handled in line with the rules of this HTTPRoute.
• hostnames: Defines which domain names this route applies to.
• backendRefs: Indicates where to route the traffic.

Set up health check Application Load BalancerSet up health check Application Load Balancer

Application Load Balancer must independently check whether the pod is ready to accept traffic, and exclude from load balancing those pods that do not respond. Gwin does not create these checks automatically, so you need to set them up explicitly. The Health Check must match the application’s readinessProbe so that statuses in Kubernetes and Application Load Balancer stay synchronized.

Configure the Application Load Balancer health check using RoutePolicy or annotations on HTTPRoute. We recommend to use the first option.

apiVersion: gwin.yandex.cloud/v1
kind: RoutePolicy
metadata:
  name: example-app
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: example-route
  policy:
    rules:
      backends:
        hc:
          interval: 3s
          timeout: 3s
          unhealthyThreshold: 1
          http:
            path: /healthz

metadata:
  annotations:
    gwin.yandex.cloud/rules.backends.hc.interval: "3s"
    gwin.yandex.cloud/rules.backends.hc.timeout: "3s"
    gwin.yandex.cloud/rules.backends.hc.unhealthyThreshold: "1"
    gwin.yandex.cloud/rules.backends.hc.http.path: /healthz

For gRPC and TCP backends, instead of http, configure the appropriate grpc or stream check in RoutePolicy.

Verification and diagnosticsVerification and diagnostics

After configuring routing to the pods, ensure that the Gateway, route, and policy have been accepted by the controller:

kubectl get gateway example-gateway
kubectl get httproute example-route
kubectl get routepolicy example-app

The gateway and HTTPRoute must be Ready, and the RoutePolicy must be applied.

The new pod will not become Ready until Gwin registers it in the Application Load Balancer target group. To check the status of the pod, use this command:

kubectl describe pod <pod-name>

Under Conditions, find the condition named gwin.yandex.cloud/load-balanced with the True value.

If the resource does not switch to Ready, check the events:

kubectl get events --sort-by=.lastTimestamp

Description of key events:

Useful metricsUseful metrics