Setting up the Yandex Application Load Balancer Ingress controller
The Yandex Application Load Balancer service is designed for load balancing and traffic distribution across applications. To use it for managing ingress traffic of applications running in a Managed Service for Kubernetes cluster, you need an Ingress controller
To set up access to the applications running in your Managed Service for Kubernetes cluster via Application Load Balancer:
- Set up the Ingress controller and test applications.
- (Optional) Configure the Ingress resource group.
- Make sure the Managed Service for Kubernetes cluster applications are accessible via Application Load Balancer.
For full configuration of the resources for the Application Load Balancer Ingress controller, see the following sections:
- Ingress: Backend traffic distribution and Ingress controller configuration rules.
- HttpBackendGroup: Combining backends into groups.
- IngressClass: Managing multiple Ingress controllers in a Kubernetes cluster.
- Service: Description of Kubernetes services used as backends.
Getting started
-
If you already have a certificate for the domain zone, add its details to the Yandex Certificate Manager service. Alternatively, you can add a new Let's Encrypt® certificate.
-
Create security groups for the Managed Service for Kubernetes cluster and its node groups.
Also configure the security groups required for Application Load Balancer.
Warning
The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.
-
Create a Managed Service for Kubernetes cluster. When creating it, specify the security groups prepared in advance.
If you intend to use your cluster within the Yandex Cloud network, there is no need to allocate a public IP address to it. To allow connections from outside the network, assign a public IP address to the cluster.
-
Create a node group. Allocate it a public IP address to provide internet access and allow pulling Docker images and components. Specify the security groups prepared in advance.
-
(Optional) Install ExternalDNS with the plugin for Yandex Cloud DNS to automatically create a DNS record in Yandex Cloud DNS when creating an Ingress controller.
-
Install kubectl
and configure it to work with the created cluster.If a cluster has no public IP address assigned and
kubectl
is configured via the cluster's private IP address, runkubectl
commands on a Yandex Cloud VM that is in the same network as the cluster.
Set up the Ingress controller and test applications
The Ingress controller's workload can include Kubernetes services or backend groups, such as Application Load Balancer target groups or Yandex Object Storage buckets.
Before getting started, get the ID of the previously added TLS certificate:
yc certificate-manager certificate list
Command result:
+----------------------+-----------+----------------+---------------------+----------+--------+
| ID | NAME | DOMAINS | NOT AFTER | TYPE | STATUS |
+----------------------+-----------+----------------+---------------------+----------+--------+
| fpq8diorouhp******** | sert-test | test.ru | 2022-01-06 17:19:37 | IMPORTED | ISSUED |
+----------------------+-----------+----------------+---------------------+----------+--------+
-
In a separate folder, create
demo-app-1.yaml
anddemo-app-2.yaml
application files:demo-app-1.yaml
apiVersion: v1 kind: ConfigMap metadata: name: alb-demo-1 data: nginx.conf: | worker_processes auto; events { } http { server { listen 80 ; location = /_healthz { add_header Content-Type text/plain; return 200 'ok'; } location / { add_header Content-Type text/plain; return 200 'Index'; } location = /app1 { add_header Content-Type text/plain; return 200 'This is APP#1'; } } } --- apiVersion: apps/v1 kind: Deployment metadata: name: alb-demo-1 labels: app: alb-demo-1 version: v1 spec: replicas: 2 selector: matchLabels: app: alb-demo-1 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: metadata: labels: app: alb-demo-1 version: v1 spec: terminationGracePeriodSeconds: 5 volumes: - name: alb-demo-1 configMap: name: alb-demo-1 containers: - name: alb-demo-1 image: nginx:latest ports: - name: http containerPort: 80 livenessProbe: httpGet: path: /_healthz port: 80 initialDelaySeconds: 3 timeoutSeconds: 2 failureThreshold: 2 volumeMounts: - name: alb-demo-1 mountPath: /etc/nginx readOnly: true resources: limits: cpu: 250m memory: 128Mi requests: cpu: 100m memory: 64Mi --- apiVersion: v1 kind: Service metadata: name: alb-demo-1 spec: selector: app: alb-demo-1 type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30081
demo-app-2.yaml
apiVersion: v1 kind: ConfigMap metadata: name: alb-demo-2 data: nginx.conf: | worker_processes auto; events { } http { server { listen 80 ; location = /_healthz { add_header Content-Type text/plain; return 200 'ok'; } location / { add_header Content-Type text/plain; return 200 'Add app#'; } location = /app2 { add_header Content-Type text/plain; return 200 'This is APP#2'; } } } --- apiVersion: apps/v1 kind: Deployment metadata: name: alb-demo-2 labels: app: alb-demo-2 version: v1 spec: replicas: 2 selector: matchLabels: app: alb-demo-2 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: metadata: labels: app: alb-demo-2 version: v1 spec: terminationGracePeriodSeconds: 5 volumes: - name: alb-demo-2 configMap: name: alb-demo-2 containers: - name: alb-demo-2 image: nginx:latest ports: - name: http containerPort: 80 livenessProbe: httpGet: path: /_healthz port: 80 initialDelaySeconds: 3 timeoutSeconds: 2 failureThreshold: 2 volumeMounts: - name: alb-demo-2 mountPath: /etc/nginx readOnly: true resources: limits: cpu: 250m memory: 128Mi requests: cpu: 100m memory: 64Mi --- apiVersion: v1 kind: Service metadata: name: alb-demo-2 spec: selector: app: alb-demo-2 type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30082
-
In the same folder, create a file named
ingress.yaml
and specify the previously delegated domain name, certificate ID, and settings for Application Load Balancer in it:apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: alb-demo-tls annotations: ingress.alb.yc.io/subnets: <list_of_subnet_IDs> ingress.alb.yc.io/security-groups: <list_of_security_group_IDs> ingress.alb.yc.io/external-ipv4-address: <IP_address_assignment_method> ingress.alb.yc.io/group-name: my-ingress-group spec: tls: - hosts: - <domain_name> secretName: yc-certmgr-cert-id-<TLS_certificate_ID> rules: - host: <domain_name> http: paths: - path: /app1 pathType: Prefix backend: service: name: alb-demo-1 port: number: 80 - path: /app2 pathType: Prefix backend: service: name: alb-demo-2 port: number: 80 - pathType: Prefix path: "/" backend: service: name: alb-demo-2 port: name: http
Where:
-
ingress.alb.yc.io/subnets
: One or more subnets that Application Load Balancer is going to work with. -
ingress.alb.yc.io/security-groups
: One or more security groups for Application Load Balancer. If you skip this parameter, the default security group will be used. At least one of the security groups must allow an outgoing TCP connection to port 10501 in the Managed Service for Kubernetes node group subnet or its security group. -
ingress.alb.yc.io/external-ipv4-address
: Providing public online access to Application Load Balancer. Enter the previously obtained IP address or setauto
to obtain a new IP address automatically.If you set
auto
, deleting the Ingress controller will also delete the IP address from the cloud. To avoid this, use an existing reserved IP address. -
ingress.alb.yc.io/group-name
: Group name. Kubernetes Ingress resources are grouped together, each group served by a separate Application Load Balancer instance.You can replace
my-ingress-group
with any group name you like. Make sure it meets the naming requirements .
In ALB Ingress Controller versions prior to 0.2.0, each backend group corresponds to a bundle of
host
,http.paths.path
, andhttp.paths.pathType
parameters. In versions 0.2.0 and later, the backend group corresponds to thebackend.service
parameter. This may cause collisions when updating the ALB Ingress Controller. To avoid them, find out whether upgrade restrictions apply to your infrastructure.(Optional) Enter the advanced settings for the controller:
Additional settings
-
ingress.alb.yc.io/group-settings-name
: Name for the Ingress resource group settings to be described in the optionalIngressGroupSettings
resource. For more information, see Configure the Ingress resource group. -
ingress.alb.yc.io/internal-ipv4-address
: Provide internal access to Application Load Balancer. Enter the internal IP address or useauto
to obtain the IP address automatically.Note
You can only use one type of access to Application Load Balancer at a time:
ingress.alb.yc.io/external-ipv4-address
oringress.alb.yc.io/internal-ipv4-address
. -
ingress.alb.yc.io/internal-alb-subnet
: Subnet for hosting the Application Load Balancer internal IP address. This parameter is required if theingress.alb.yc.io/internal-ipv4-address
parameter is selected. -
ingress.alb.yc.io/protocol
: Connection protocol used by the load balancer and the backends:http
: HTTP/1.1, defaulthttp2
: HTTP/2grpc
: gRPC
-
ingress.alb.yc.io/transport-security
: Encryption protocol for connections between the load balancer and backends.Warning
In ALB Ingress Controller version 0.2.0 and later, you can only use an annotation in the Service object.
If you annotate
Ingress
resources that use a single service with the same settings for backend groups, such annotation will apply correctly. However, this mechanism is obsolete and will not be supported going forward.The acceptable value is
tls
: TLS with no certificate challenge.If no annotation is specified, the load balancer connects to the backends with no encryption.
-
ingress.alb.yc.io/prefix-rewrite
: Replace the path for the specified value. -
ingress.alb.yc.io/upgrade-types
: Valid values for theUpgrade
HTTP header, e.g.,websocket
. -
ingress.alb.yc.io/request-timeout
: Maximum period for which the connection can be established. -
ingress.alb.yc.io/idle-timeout
: Maximum connection keep-alive time with zero data transmission.Values for
request-timeout
andidle-timeout
must be specified with units of measurement, e.g.,300ms
,1.5h
. Acceptable units of measurement include:ns
: Nanosecondsus
: Microsecondsms
: Millisecondss
: Secondsm
: Minutesh
: Hours
-
ingress.alb.yc.io/security-profile-id
: Support for Yandex Smart Web Security that allows you to get protected against DDoS attacks and bots, as well as activate WAF and limit the load to the resource being protected.To enable support for Yandex Smart Web Security, specify the previously created Smart Web Security security profile in the Ingress annotation:
ingress.alb.yc.io/security-profile-id: <security_profile_ID>
-
ingress.alb.yc.io/use-regex
: Support for RE2 regular expressions when matching the request path. If thetrue
string is provided, the support is enabled. Only applies if thepathType
parameter is set toExact
.
Note
The settings only apply to the hosts of the given controller rather than the entire Ingress group.
If you use several Ingress controllers, create an IngressClass resource for each of them. In the
Ingress
configuration, specify theIngressClass
you need in thespec.ingressClassName
field.For more information about the
Ingress
resource settings, see Ingress resource fields and annotations. -
-
Create an Ingress controller and applications:
kubectl apply -f .
-
Wait until the Ingress controller is created and assigned a public IP address. This may take several minutes.
To track the progress of controller creation and check that it is error-free, open the logs of the pod where the controller is being created:
-
In the management console
, go to the folder page and select Managed Service for Kubernetes. -
Click your cluster's name and select Workload in the left-hand panel.
-
Select one of the
alb-demo-***
pods the Ingress controller is being created on. -
Go to the Logs tab on the pod page.
You will see the Ingress controller creation logged, with the logs displayed in real time. Any errors that occur will also be logged.
-
-
Make sure the Ingress controller has been created. To do this, run the appropriate command and check that the command output shows the following value in the
ADDRESS
field:kubectl get ingress alb-demo-tls
Result:
NAME CLASS HOSTS ADDRESS PORTS AGE alb-demo-tls <none> <domain_name> <IP_address> 80, 443 15h
Based on the Ingress controller configuration, an L7 load balancer will be automatically deployed.
To set up a backend group use the HttpBackendGroup
CustomResourceDefinition
To configure Application Load Balancer to work with a backend group:
-
Create a backend group with a bucket:
-
Create a configuration file named
demo-app-1.yaml
for your application:demo-app-1.yaml
apiVersion: v1 kind: ConfigMap metadata: name: alb-demo-1 data: nginx.conf: | worker_processes auto; events { } http { server { listen 80 ; location = /_healthz { add_header Content-Type text/plain; return 200 'ok'; } location / { add_header Content-Type text/plain; return 200 'Index'; } location = /app1 { add_header Content-Type text/plain; return 200 'This is APP#1'; } } } --- apiVersion: apps/v1 kind: Deployment metadata: name: alb-demo-1 labels: app: alb-demo-1 version: v1 spec: replicas: 2 selector: matchLabels: app: alb-demo-1 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: metadata: labels: app: alb-demo-1 version: v1 spec: terminationGracePeriodSeconds: 5 volumes: - name: alb-demo-1 configMap: name: alb-demo-1 containers: - name: alb-demo-1 image: nginx:latest ports: - name: http containerPort: 80 livenessProbe: httpGet: path: /_healthz port: 80 initialDelaySeconds: 3 timeoutSeconds: 2 failureThreshold: 2 volumeMounts: - name: alb-demo-1 mountPath: /etc/nginx readOnly: true resources: limits: cpu: 250m memory: 128Mi requests: cpu: 100m memory: 64Mi --- apiVersion: v1 kind: Service metadata: name: alb-demo-1 spec: selector: app: alb-demo-1 type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30081
-
In a separate directory, create a file named
http-group.yaml
with theHttpBackendGroup
object settings:apiVersion: alb.yc.io/v1alpha1 kind: HttpBackendGroup metadata: name: example-backend-group spec: backends: # List of backends. - name: alb-demo-1 weight: 70 # Relative weight of the backend when distributing traffic. The load will be distributed in proportion to the weights of the group's other backends. Specify the weight even if you have only one backend in the group. service: name: alb-demo-1 port: number: 80 - name: bucket-backend weight: 30 storageBucket: name: <bucket_name>
(Optional) Enter the advanced settings for the controller:
spec.backends.useHttp2
: To useHTTP/2
.spec.backends.tls
: Certificate from the certificate authority that the load balancer will trust when establishing a secure connection with backend endpoints. Specify the certificate contents in thetrustedCa
field as open text.
For more information, see Backend groups.
-
Create a file named
ingress-http.yaml
and specify the previously delegated domain name, certificate ID, and settings for Application Load Balancer in it:apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: alb-demo-tls annotations: ingress.alb.yc.io/subnets: <list_of_subnet_IDs> ingress.alb.yc.io/security-groups: <list_of_security_group_IDs> ingress.alb.yc.io/external-ipv4-address: <IP_address_assignment_method> ingress.alb.yc.io/group-name: my-ingress-group spec: tls: - hosts: - <domain_name> secretName: yc-certmgr-cert-id-<TLS_certificate_ID> rules: - host: <domain_name> http: paths: - path: /app1 pathType: Exact backend: resource: apiGroup: alb.yc.io kind: HttpBackendGroup name: example-backend-group
Where:
-
ingress.alb.yc.io/subnets
: One or more subnets that Application Load Balancer is going to work with. -
ingress.alb.yc.io/security-groups
: One or more security groups for Application Load Balancer. If you skip this parameter, the default security group will be used. At least one of the security groups must allow an outgoing TCP connection to port 10501 in the Managed Service for Kubernetes node group subnet or its security group. -
ingress.alb.yc.io/external-ipv4-address
: Providing public online access to Application Load Balancer. Enter the previously obtained IP address or setauto
to obtain a new IP address automatically.If you set
auto
, deleting the Ingress controller will also delete the IP address from the cloud. To avoid this, use an existing reserved IP address. -
ingress.alb.yc.io/group-name
: Group name. Kubernetes Ingress resources are grouped together, each group served by a separate Application Load Balancer instance.You can replace
my-ingress-group
with any group name you like. Make sure it meets the naming requirements .
(Optional) Enter the advanced settings for the controller:
-
ingress.alb.yc.io/group-settings-name
: Name for the Ingress resource group settings to be described in the optionalIngressGroupSettings
resource. For more information, see Configure the Ingress resource group. -
ingress.alb.yc.io/internal-ipv4-address
: Provide internal access to Application Load Balancer. Enter the internal IP address or useauto
to obtain the IP address automatically.Note
You can only use one type of access to Application Load Balancer at a time:
ingress.alb.yc.io/external-ipv4-address
oringress.alb.yc.io/internal-ipv4-address
. -
ingress.alb.yc.io/internal-alb-subnet
: Subnet for hosting the Application Load Balancer internal IP address. This parameter is required if theingress.alb.yc.io/internal-ipv4-address
parameter is selected. -
ingress.alb.yc.io/protocol
: Connection protocol used by the load balancer and the backends:http
: HTTP/1.1, defaulthttp2
: HTTP/2grpc
: gRPC
-
ingress.alb.yc.io/prefix-rewrite
: Replace the path for the specified value. -
ingress.alb.yc.io/upgrade-types
: Valid values for theUpgrade
HTTP header, e.g.,websocket
. -
ingress.alb.yc.io/request-timeout
: Maximum period for which the connection can be established. -
ingress.alb.yc.io/idle-timeout
: Maximum connection keep-alive time with zero data transmission.Values for
request-timeout
andidle-timeout
must be specified with units of measurement, e.g.,300ms
,1.5h
. Acceptable units of measurement include:ns
: Nanosecondsus
: Microsecondsms
: Millisecondss
: Secondsm
: Minutesh
: Hours
-
ingress.alb.yc.io/security-profile-id
: Support for Yandex Smart Web Security that allows you to get protected against DDoS attacks and bots, as well as activate WAF and limit the load to the resource being protected.To enable support for Yandex Smart Web Security, specify the previously created Smart Web Security security profile in the Ingress annotation:
ingress.alb.yc.io/security-profile-id: <security_profile_ID>
-
ingress.alb.yc.io/use-regex
: Support for RE2 regular expressions when matching the request path. If thetrue
string is provided, the support is enabled. Only applies if thepathType
parameter is set toExact
.
Note
The settings only apply to the hosts of the given controller rather than the entire Ingress group.
For more information about the Ingress resource settings, see Ingress resource fields and annotations.
-
-
Create an Ingress controller, an
HttpBackendGroup
object, and a Kubernetes app:kubectl apply -f .
-
Wait until the Ingress controller is created and assigned a public IP address. This may take several minutes.
To track the progress of controller creation and check that it is error-free, open the logs of the pod where the controller is being created:
-
In the management console
, go to the folder page and select Managed Service for Kubernetes. -
Click your cluster's name and select Workload in the left-hand panel.
-
Select one of the
alb-demo-***
pods the Ingress controller is being created on. -
Go to the Logs tab on the pod page.
You will see the Ingress controller creation logged, with the logs displayed in real time. Any errors that occur will also be logged.
-
-
Make sure the Ingress controller has been created. To do this, run the appropriate command and check that the command output shows the following value in the
ADDRESS
field:kubectl get ingress alb-demo-tls
Result:
NAME CLASS HOSTS ADDRESS PORTS AGE alb-demo-tls <none> <domain_name> <IP_address> 80, 443 15h
Based on the Ingress controller configuration, an L7 load balancer will be automatically deployed.
By default the Application Load Balancer Ingress controller receives application health check requests from the L7 load balancer on TCP port 10501
and checks if the kube-proxy
You can use the HttpBackendGroup resource parameters to customize health checks. For more information, see Health checking your applications in a Yandex Managed Service for Kubernetes cluster with the Yandex Application Load Balancer Ingress controller.
(Optional) Configure the Ingress resource group
If you specified a name for the Ingress resource group settings in the ingress.alb.yc.io/group-settings-name
annotation when you installed the Ingress controller, you can specify logging settings for the L7 load balancer. To do this, create a custom log group and specify the Ingress resource group settings in the optional IngressGroupSettings
resource.
-
Create a
settings.yaml
file with your logging settings and the custom log group ID. For example:apiVersion: alb.yc.io/v1alpha1 kind: IngressGroupSettings metadata: name: <name_for_Ingress_resource_group_settings> logOptions: logGroupID: <custom_log_group_ID> discardRules: - discardPercent: 50 grpcCodes: - OK - CANCELLED - UNKNOWN - discardPercent: 67 httpCodeIntervals: - HTTP_1XX - discardPercent: 20 httpCodes: - 200 - 404
Where
name
is the name for the Ingress resource group settings in theingress.alb.yc.io/group-settings-name
annotation. -
Apply the settings for the Ingress resource group:
kubectl apply -f settings.yaml
Make sure the Managed Service for Kubernetes cluster applications are accessible through Application Load Balancer
-
If you have no ExternalDNS with a plugin for Cloud DNS installed, add an A record to your domain zone. In the Value field, specify the public IP address of the Ingress controller. If you are using ExternalDNS with a plugin for Yandex Cloud DNS, this record will be created automatically.
-
Test Application Load Balancer:
Kubernetes servicesBackend groupOpen the application URIs in your browser:
https://<your_domain>/app1 https://<your_domain>/app2
Make sure the applications are accessible via Application Load Balancer and return pages with the
This is APP#1
andThis is APP#2
lines, respectively.Open the application's URI in your browser:
https://<your_domain>/app1
Make sure that the target resources are accessible via Application Load Balancer.
Delete the resources you created
Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need: