Configuring an Yandex Application Load Balancer L7 load balancer using an Ingress controller
The Yandex Application Load Balancer service is designed for load balancing and traffic distribution across applications. To use it for managing ingress traffic of applications running in a Managed Service for Kubernetes cluster, you need an Ingress controller
To set up access to the applications running in your Managed Service for Kubernetes cluster via an Application Load Balancer L7 load balancer:
- Set up the Ingress resource and test applications.
- (Optional) Configure the Ingress resource group.
- Make sure the Managed Service for Kubernetes cluster applications are accessible via Application Load Balancer.
For full configuration of the resources for the Application Load Balancer Ingress controller, see the following sections:
- Ingress: Backend traffic distribution and load balancer configuration rules.
- HttpBackendGroup: Combining backends into groups.
- IngressClass: Managing multiple Ingress controllers in a Kubernetes cluster.
- Service: Description of Kubernetes services used as backends.
Getting started
-
If you already have a certificate for the domain zone, add its details to the Yandex Certificate Manager service. Alternatively, you can add a new Let's Encrypt® certificate.
-
Get the ID of the certificate you added:
yc certificate-manager certificate list
The result will be as follows:
+----------------------+-----------+----------------+---------------------+----------+--------+ | ID | NAME | DOMAINS | NOT AFTER | TYPE | STATUS | +----------------------+-----------+----------------+---------------------+----------+--------+ | fpq8diorouhp******** | sert-test | test.ru | 2022-01-06 17:19:37 | IMPORTED | ISSUED | +----------------------+-----------+----------------+---------------------+----------+--------+
-
Create security groups for the Managed Service for Kubernetes cluster and its node groups.
Also configure the security groups required for Application Load Balancer.
Warning
The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.
-
Create a Managed Service for Kubernetes cluster. When creating it, specify the security groups prepared in advance.
If you intend to use your cluster within the Yandex Cloud network, there is no need to allocate a public IP address to it. To allow connections from outside the network, assign a public IP address to the cluster.
-
Create a node group. Allocate it a public IP address to provide internet access and allow pulling Docker images and components. Specify the security groups prepared in advance.
-
Optionally, install ExternalDNS with a plugin for Yandex Cloud DNS to automatically create a DNS record in Yandex Cloud DNS when creating an Ingress controller.
-
Install kubectl
and configure it to work with the created cluster.If a cluster has no public IP address assigned and
kubectl
is configured via the cluster's private IP address, runkubectl
commands on a Yandex Cloud VM that is in the same network as the cluster.
Set up the Ingress resource and test applications
The Ingress resource defines:
-
L7 load balancer parameters set using annotations.
-
Rules for distribution of incoming traffic between Kubernetes services.
Services representing Application Load Balancer backends may be specified in the Ingress resource either directly or as part of HttpBackendGroup backend groups.
Create test applications and an Ingress resource:
-
In a separate directory, create the
demo-app-1.yaml
anddemo-app-2.yaml
application configuration files:demo-app-1.yaml
apiVersion: v1 kind: ConfigMap metadata: name: alb-demo-1 data: nginx.conf: | worker_processes auto; events { } http { server { listen 80 ; location = /_healthz { add_header Content-Type text/plain; return 200 'ok'; } location / { add_header Content-Type text/plain; return 200 'Index'; } location = /app1 { add_header Content-Type text/plain; return 200 'This is APP#1'; } } } --- apiVersion: apps/v1 kind: Deployment metadata: name: alb-demo-1 labels: app: alb-demo-1 version: v1 spec: replicas: 2 selector: matchLabels: app: alb-demo-1 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: metadata: labels: app: alb-demo-1 version: v1 spec: terminationGracePeriodSeconds: 5 volumes: - name: alb-demo-1 configMap: name: alb-demo-1 containers: - name: alb-demo-1 image: nginx:latest ports: - name: http containerPort: 80 livenessProbe: httpGet: path: /_healthz port: 80 initialDelaySeconds: 3 timeoutSeconds: 2 failureThreshold: 2 volumeMounts: - name: alb-demo-1 mountPath: /etc/nginx readOnly: true resources: limits: cpu: 250m memory: 128Mi requests: cpu: 100m memory: 64Mi --- apiVersion: v1 kind: Service metadata: name: alb-demo-1 spec: selector: app: alb-demo-1 type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30081
demo-app-2.yaml
apiVersion: v1 kind: ConfigMap metadata: name: alb-demo-2 data: nginx.conf: | worker_processes auto; events { } http { server { listen 80 ; location = /_healthz { add_header Content-Type text/plain; return 200 'ok'; } location / { add_header Content-Type text/plain; return 200 'Add app#'; } location = /app2 { add_header Content-Type text/plain; return 200 'This is APP#2'; } } } --- apiVersion: apps/v1 kind: Deployment metadata: name: alb-demo-2 labels: app: alb-demo-2 version: v1 spec: replicas: 2 selector: matchLabels: app: alb-demo-2 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: metadata: labels: app: alb-demo-2 version: v1 spec: terminationGracePeriodSeconds: 5 volumes: - name: alb-demo-2 configMap: name: alb-demo-2 containers: - name: alb-demo-2 image: nginx:latest ports: - name: http containerPort: 80 livenessProbe: httpGet: path: /_healthz port: 80 initialDelaySeconds: 3 timeoutSeconds: 2 failureThreshold: 2 volumeMounts: - name: alb-demo-2 mountPath: /etc/nginx readOnly: true resources: limits: cpu: 250m memory: 128Mi requests: cpu: 100m memory: 64Mi --- apiVersion: v1 kind: Service metadata: name: alb-demo-2 spec: selector: app: alb-demo-2 type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30082
-
In the same directory, create a file named
ingress.yaml
and specify in it the previously delegated domain name, ID of the certificate obtained earlier, and settings for the Application Load Balancer L7 load balancer:apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: alb-demo-tls annotations: ingress.alb.yc.io/subnets: <list_of_subnet_IDs> ingress.alb.yc.io/security-groups: <list_of_security_group_IDs> ingress.alb.yc.io/external-ipv4-address: <IP_address_assignment_method> ingress.alb.yc.io/group-name: my-ingress-group spec: tls: - hosts: - <domain_name> secretName: yc-certmgr-cert-id-<TLS_certificate_ID> rules: - host: <domain_name> http: paths: - path: /app1 pathType: Prefix backend: service: name: alb-demo-1 port: number: 80 - path: /app2 pathType: Prefix backend: service: name: alb-demo-2 port: number: 80 - pathType: Prefix path: "/" backend: service: name: alb-demo-2 port: name: http
Where:
-
ingress.alb.yc.io/subnets
: One or more subnets to host the Application Load Balancer L7 load balancer. -
ingress.alb.yc.io/security-groups
: One or more security groups for the load balancer. If you skip this parameter, the default security group will be used. At least one of the security groups must allow an outgoing TCP connection to port10501
in the Managed Service for Kubernetes node group subnet or to its security group. -
ingress.alb.yc.io/external-ipv4-address
: Public access to the load balancer from the internet. Enter the previously obtained IP address or setauto
to get a new IP address automatically.If you set
auto
, deleting the load balancer will also delete its IP address from the cloud. To avoid this, use an existing reserved IP address. -
ingress.alb.yc.io/group-name
: Group name. Ingress resources are grouped together, each group served by a separate load balancer.You can replace
my-ingress-group
with any group name you like. Make sure it meets the naming requirements .
In ALB Ingress Controller versions prior to 0.2.0, each backend group corresponds to a bundle of
host
,http.paths.path
, andhttp.paths.pathType
parameters. In versions 0.2.0 and later, the backend group corresponds to thebackend.service
parameter. This may cause collisions when updating the ALB Ingress Controller. To avoid them, find out whether upgrade restrictions apply to your infrastructure.(Optional) Enter the advanced settings for the load balancer:
Additional settings
Note
The settings listed below will only apply to the virtual hosts of the Ingress resource in which the corresponding annotations are configured.
They will not apply to the virtual hosts of the group's other Ingress resources.
Available settings:
-
ingress.alb.yc.io/group-settings-name
: Name for the Ingress resource group settings to be described in the optionalIngressGroupSettings
resource. For more information, see Configure the Ingress resource group. -
ingress.alb.yc.io/internal-ipv4-address
: Provide internal access to the load balancer. Enter the internal IP address or useauto
to obtain the IP address automatically.Note
You can only use one type of access to the load balancer at a time:
ingress.alb.yc.io/external-ipv4-address
oringress.alb.yc.io/internal-ipv4-address
. -
ingress.alb.yc.io/internal-alb-subnet
: Subnet to host the load balancer. This parameter is required if theingress.alb.yc.io/internal-ipv4-address
parameter is selected. -
ingress.alb.yc.io/protocol
: Connection protocol used between the load balancer and backends:http
: HTTP/1.1. Default.http2
: HTTP/2.grpc
: gRPC.
-
ingress.alb.yc.io/transport-security
: Encryption protocol for connections between the load balancer and backends.Warning
In ALB Ingress Controller version 0.2.0 and later, you can only use an annotation in the Service object.
If you annotate
Ingress
resources that use a single service with the same settings for backend groups, such annotation will apply correctly. However, this mechanism is obsolete and will not be supported going forward.The valid value is
tls
: TLS with no certificate challenge.If no annotation is specified, the load balancer connects to the backends with no encryption.
-
ingress.alb.yc.io/prefix-rewrite
: Replace the path for the specified value. -
ingress.alb.yc.io/upgrade-types
: Valid values of theUpgrade
HTTP header, e.g.,websocket
. -
ingress.alb.yc.io/request-timeout
: Maximum period for which a connection can be established. -
ingress.alb.yc.io/idle-timeout
: Maximum connection keep-alive time without data transmission.The
request-timeout
andidle-timeout
values must be specified with units of measurement, e.g.,300ms
or1.5h
. Valid units of measurement:ns
, nanosecondsus
, microsecondsms
, millisecondss
, secondsm
, minutesh
, hours
-
ingress.alb.yc.io/security-profile-id
: Support for Yandex Smart Web Security that allows you to get protected against DDoS attacks and bots, plus enable WAF and limit the load on the resource you are protecting.To enable support for Yandex Smart Web Security, specify the previously created Smart Web Security security profile in the Ingress annotation:
ingress.alb.yc.io/security-profile-id: <security_profile_ID>
Note
To connect your security profile to an Application Load Balancer virtual host, the service account used to operate the Ingress controller must have the smart-web-security.editor role for the folder hosting Application Load Balancer and Smart Web Security resources. For more information, see Assigning a role to a service account.
-
ingress.alb.yc.io/use-regex
: Support for RE2 regular expressions when matching the request path. If thetrue
string is provided, the support is enabled. Only applies if thepathType
parameter is set toExact
.
If you use several Ingress controllers, create an IngressClass resource for each of them. In the
Ingress
configuration, specify theIngressClass
you need in thespec.ingressClassName
field.For more information about the
Ingress
resource settings, see Ingress resource fields and annotations. -
-
Create Kubernetes applications and the Ingress resource:
kubectl apply -f .
ALB Ingress Controller will automatically deploy the L7 load balancer using Ingress resource configuration.
-
Wait until the Application Load Balancer L7 load balancer is created and gets a public IP address. This may take several minutes.
To follow the load balancer's creation and make sure it is error-free, open the logs of the pod the creation process was run in:
-
In the management console
, go to the folder page and select Managed Service for Kubernetes. -
Click your cluster's name and select Workload in the left-hand panel.
-
Select one of the
alb-demo-***
pods the load balancer's creation was run in. -
Go to the Logs tab on the pod page.
The load balancer's creation logs are generated and displayed in real time. Any errors that occur will also be logged.
-
-
Make sure the load balancer was created. To do this, run the appropriate command and check that the command output shows the following value in the
ADDRESS
field:kubectl get ingress alb-demo-tls
Result:
NAME CLASS HOSTS ADDRESS PORTS AGE alb-demo-tls <none> <domain_name> <IP_address> 80,443 15h
-
Create a backend group with a bucket:
-
Create a configuration file named
demo-app-1.yaml
for your application:demo-app-1.yaml
apiVersion: v1 kind: ConfigMap metadata: name: alb-demo-1 data: nginx.conf: | worker_processes auto; events { } http { server { listen 80 ; location = /_healthz { add_header Content-Type text/plain; return 200 'ok'; } location / { add_header Content-Type text/plain; return 200 'Index'; } location = /app1 { add_header Content-Type text/plain; return 200 'This is APP#1'; } } } --- apiVersion: apps/v1 kind: Deployment metadata: name: alb-demo-1 labels: app: alb-demo-1 version: v1 spec: replicas: 2 selector: matchLabels: app: alb-demo-1 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: metadata: labels: app: alb-demo-1 version: v1 spec: terminationGracePeriodSeconds: 5 volumes: - name: alb-demo-1 configMap: name: alb-demo-1 containers: - name: alb-demo-1 image: nginx:latest ports: - name: http containerPort: 80 livenessProbe: httpGet: path: /_healthz port: 80 initialDelaySeconds: 3 timeoutSeconds: 2 failureThreshold: 2 volumeMounts: - name: alb-demo-1 mountPath: /etc/nginx readOnly: true resources: limits: cpu: 250m memory: 128Mi requests: cpu: 100m memory: 64Mi --- apiVersion: v1 kind: Service metadata: name: alb-demo-1 spec: selector: app: alb-demo-1 type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30081
-
In a separate directory, create a file named
http-group.yaml
with theHttpBackendGroup
resource settings:apiVersion: alb.yc.io/v1alpha1 kind: HttpBackendGroup metadata: name: example-backend-group spec: backends: # List of backends. - name: alb-demo-1 weight: 70 # Backend relative weight when distributing traffic. The load will be distributed in proportion to the weights of the group's other backends. Specify the weight even if you have only one backend in the group. service: name: alb-demo-1 port: number: 80 - name: bucket-backend weight: 30 storageBucket: name: <bucket_name>
(Optional) Enter the advanced settings for the backend group:
spec.backends.useHttp2
:HTTP/2
mode.spec.backends.tls
: Certificate from the certificate authority the load balancer will trust when establishing a secure connection with backend endpoints. Specify the certificate contents in thetrustedCa
field in plain text.
For more information, see Backend groups.
-
In the same directory, create a file named
ingress-http.yaml
and specify in it the previously delegated domain name, ID of the certificate obtained earlier, and settings for the Application Load Balancer L7 load balancer:apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: alb-demo-tls annotations: ingress.alb.yc.io/subnets: <list_of_subnet_IDs> ingress.alb.yc.io/security-groups: <list_of_security_group_IDs> ingress.alb.yc.io/external-ipv4-address: <IP_address_assignment_method> ingress.alb.yc.io/group-name: my-ingress-group spec: tls: - hosts: - <domain_name> secretName: yc-certmgr-cert-id-<TLS_certificate_ID> rules: - host: <domain_name> http: paths: - path: /app1 pathType: Exact backend: resource: apiGroup: alb.yc.io kind: HttpBackendGroup name: example-backend-group
Where:
-
ingress.alb.yc.io/subnets
: One or more subnets to host the Application Load Balancer L7 load balancer. -
ingress.alb.yc.io/security-groups
: One or more security groups for the load balancer. If you skip this parameter, the default security group will be used. At least one of the security groups must allow an outgoing TCP connection to port10501
in the Managed Service for Kubernetes node group subnet or to its security group. -
ingress.alb.yc.io/external-ipv4-address
: Public access to the load balancer from the internet. Enter the previously obtained IP address or setauto
to get a new IP address automatically.If you set
auto
, deleting the load balancer will also delete its IP address from the cloud. To avoid this, use an existing reserved IP address. -
ingress.alb.yc.io/group-name
: Group name. Ingress resources are grouped together, each group served by a separate load balancer.You can replace
my-ingress-group
with any group name you like. Make sure it meets the naming requirements .
(Optional) Enter the advanced settings for the load balancer.
Note
The settings listed below will only apply to the virtual hosts of the Ingress resource in which the corresponding annotations are configured.
They will not apply to the virtual hosts of the group's other Ingress resources.
Available settings:
-
ingress.alb.yc.io/group-settings-name
: Name for the Ingress resource group settings to be described in the optionalIngressGroupSettings
resource. For more information, see Configure the Ingress resource group. -
ingress.alb.yc.io/internal-ipv4-address
: Provide internal access to the load balancer. Enter the internal IP address or useauto
to obtain the IP address automatically.Note
You can only use one type of access to the load balancer at a time:
ingress.alb.yc.io/external-ipv4-address
oringress.alb.yc.io/internal-ipv4-address
. -
ingress.alb.yc.io/internal-alb-subnet
: Subnet to host the load balancer. This parameter is required if theingress.alb.yc.io/internal-ipv4-address
parameter is selected. -
ingress.alb.yc.io/protocol
: Connection protocol used between the load balancer and backends:http
: HTTP/1.1. Default.http2
: HTTP/2.grpc
: gRPC.
-
ingress.alb.yc.io/prefix-rewrite
: Replace the path for the specified value. -
ingress.alb.yc.io/upgrade-types
: Valid values of theUpgrade
HTTP header, e.g.,websocket
. -
ingress.alb.yc.io/request-timeout
: Maximum period for which a connection can be established. -
ingress.alb.yc.io/idle-timeout
: Maximum connection keep-alive time without data transmission.The
request-timeout
andidle-timeout
values must be specified with units of measurement, e.g.,300ms
or1.5h
. Valid units of measurement:ns
, nanosecondsus
, microsecondsms
, millisecondss
, secondsm
, minutesh
, hours
-
ingress.alb.yc.io/security-profile-id
: Support for Yandex Smart Web Security that allows you to get protected against DDoS attacks and bots, plus enable WAF and limit the load on the resource you are protecting.To enable support for Yandex Smart Web Security, specify the previously created Smart Web Security security profile in the Ingress annotation:
ingress.alb.yc.io/security-profile-id: <security_profile_ID>
Note
To connect your security profile to an Application Load Balancer virtual host, the service account used to operate the Ingress controller must have the smart-web-security.editor role for the folder hosting Application Load Balancer and Smart Web Security resources. For more information, see Assigning a role to a service account.
-
ingress.alb.yc.io/use-regex
: Support for RE2 regular expressions when matching the request path. If thetrue
string is provided, the support is enabled. Only applies if thepathType
parameter is set toExact
.
For more information about the Ingress resource settings, see Ingress resource fields and annotations.
-
-
Create the Kubernetes app,
HttpBackendGroup
resource, and Ingress resource:kubectl apply -f .
ALB Ingress Controller will automatically deploy the L7 load balancer using Ingress resource configuration.
-
Wait until the Application Load Balancer L7 load balancer is created and gets a public IP address. This may take several minutes.
To follow the load balancer's creation and make sure it is error-free, open the logs of the pod the creation process was run in:
-
In the management console
, go to the folder page and select Managed Service for Kubernetes. -
Click your cluster's name and select Workload in the left-hand panel.
-
Select one of the
alb-demo-***
pods the load balancer's creation was run in. -
Go to the Logs tab on the pod page.
The load balancer's creation logs are generated and displayed in real time. Any errors that occur will also be logged.
-
-
Make sure the load balancer was created. To do this, run the appropriate command and check that the command output shows the following value in the
ADDRESS
field:kubectl get ingress alb-demo-tls
Result:
NAME CLASS HOSTS ADDRESS PORTS AGE alb-demo-tls <none> <domain_name> <IP_address> 80,443 15h
By default, the Application Load Balancer Ingress controller receives application health check requests from the L7 load balancer at TCP port 10501
and health checks the kube-proxykube-proxy
is healthy, then, even though an application does not respond in a particular pod, Kubernetes will redirect traffic to a different pod with that application or to a different node.
You can use the HttpBackendGroup resource parameters to customize health checks. For more information, see Health checking your apps in a Yandex Managed Service for Kubernetes cluster using a Yandex Application Load Balancer L7 load balancer.
(Optional) Configure the Ingress resource group
If you specified a name for the Ingress resource group settings in the ingress.alb.yc.io/group-settings-name
annotation when setting up the Ingress resource, you can specify logging settings for the L7 load balancer. To do this, create a custom log group and specify the Ingress resource group settings in the optional IngressGroupSettings
resource.
-
Create a
settings.yaml
file with your logging settings and the custom log group ID, e.g.:apiVersion: alb.yc.io/v1alpha1 kind: IngressGroupSettings metadata: name: <name_for_Ingress_resource_group_settings> logOptions: logGroupID: <user_log_group_ID> discardRules: - discardPercent: 50 grpcCodes: - OK - CANCELLED - UNKNOWN - discardPercent: 67 httpCodeIntervals: - HTTP_1XX - discardPercent: 20 httpCodes: - 200 - 404
Where
name
is the name for the Ingress resource group settings in theingress.alb.yc.io/group-settings-name
annotation. -
Apply the settings for the Ingress resource group:
kubectl apply -f settings.yaml
Make sure the applications are accessible via the L7 load balancer
-
If you have no ExternalDNS with a plugin for Cloud DNS installed, add an A record to your domain zone. In the Value field, specify the public IP address of the Application Load Balancer L7 load balancer. If you are using ExternalDNS with a plugin for Yandex Cloud DNS, this record will be created automatically.
-
Test the load balancer:
Ingress resource for Kubernetes servicesIngress resource for a backend groupOpen the application URIs in your browser:
https://<your_domain>/app1 https://<your_domain>/app2
Make sure the applications are accessible via the Application Load Balancer L7 load balancer and return pages with the
This is APP#1
andThis is APP#2
text, respectively.Open the application's URI in your browser:
https://<your_domain>/app1
Make sure the target resources are accessible via the Application Load Balancer L7 load balancer.
Delete the resources you created
Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need: