Logging settings for Yandex Application Load Balancer Ingress controllers
You can configure the logging settings for L7 balancers created with Application Load Balancer Ingress controllers
This article explains how to create three L7 balancers with different logging settings:
- Save logs to the default group.
- Save logs to a custom group.
- No logging.
To configure L7 balancers:
- Create a test application.
- Set up the Ingress controllers.
- Set up the Ingress controller groups.
- Check the result.
If you no longer need the resources you created, delete them.
Getting started
Prepare the infrastructure
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
-
Create the following service accounts for the Managed Service for Kubernetes cluster:
-
Service account for resources with the
k8s.clusters.agent
andvpc.publicAdmin
roles for the folder where the Managed Service for Kubernetes cluster is created. -
Service account for nodes with the container-registry.images.puller role for the folder with the Docker image registry. Nodes will pull the required Docker images from the registry on behalf of this account.
-
Service account for the operation of the Application Load Balancer Ingress controller with the following roles:
- alb.editor: To create the required resources.
- vpc.publicAdmin: To manage external connectivity.
- certificate-manager.certificates.downloader: To use certificates registered in Yandex Certificate Manager.
- compute.viewer: To use Managed Service for Kubernetes cluster nodes in balancer target groups.
You can use the same service account for all operations.
-
-
Create an authorized key for the service account of the Ingress controller in JSON format and save it to
key.json
:yc iam key create \ --service-account-name <Ingress_controller_service_account_name> \ --output key.json
The key data is required for the installation of the ALB Ingress Controller application.
-
Create security groups for the Managed Service for Kubernetes cluster and its node groups.
Also configure the security groups required for Application Load Balancer.
Warning
The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.
-
Create a Managed Service for Kubernetes cluster. Configure the settings:
- Specify the previously created service account for resources as well as security groups.
- If you intend to use your cluster within the Yandex Cloud network, there is no need to allocate a public IP address to it. To allow connections from outside the network, assign a public IP address to the cluster.
-
Create a node group. Configure the settings:
- Specify the previously created service account for nodes as well as security groups.
- Allocate it a public IP address to grant internet access to the node group and allow pulling Docker images and components.
-
If you do not have Terraform yet, install it.
-
Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.
-
Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it
. -
Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.
-
Download the k8s-and-registry-for-alb.tf
configuration file to the same working directory.This file describes:
-
Managed Service for Kubernetes cluster.
-
Security groups which contain rules required for the Managed Service for Kubernetes cluster and its node groups.
Some rules are required for Application Load Balancer to work correctly.
Warning
The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.
-
Service account for Managed Service for Kubernetes resources and nodes.
-
Service account for the operation of the Application Load Balancer Ingress controller.
-
Authorized key for the service account of the Ingress controller.
-
Creating a local
key.json
file with authorized key data. The key data is required for the installation of the ALB Ingress Controller application.
-
In the
k8s-and-registry-for-alb.tf
file, specify:- Folder ID.
- Kubernetes version for the Managed Service for Kubernetes cluster and node groups.
- Name of the service account for Kubernetes resources and nodes.
- Name of the service account for the operation of the Application Load Balancer Ingress controller.
- Name of the custom Cloud Logging log group.
-
Make sure the Terraform configuration files are correct using this command:
terraform validate
If there are any errors in the configuration files, Terraform will point them out.
-
Create the required infrastructure:
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console
. -
Before you start working with the Managed Service for Kubernetes cluster
-
Install kubectl
and configure it to work with the created cluster.If a cluster has no public IP address assigned and
kubectl
is configured via the cluster's private IP address, runkubectl
commands on a Yandex Cloud VM that is in the same network as the cluster.
Register a domain zone
Register a public domain zone and delegate your domain.
Install the Application Load Balancer Ingress controller
Install the ALB Ingress Controller application by following this guide. During the installation, use the key.json
key data from Prepare the infrastructure.
Create a test application
Create ConfigMap
-
Create an
app.yaml
application file:app.yaml
apiVersion: v1 kind: ConfigMap metadata: name: alb-demo-1 data: nginx.conf: | worker_processes auto; events { } http { server { listen 80 ; location = /_healthz { add_header Content-Type text/plain; return 200 'ok'; } location / { add_header Content-Type text/plain; return 200 'Index'; } location = /app1 { add_header Content-Type text/plain; return 200 'This is APP#1'; } } } --- apiVersion: apps/v1 kind: Deployment metadata: name: alb-demo-1 labels: app: alb-demo-1 version: v1 spec: replicas: 2 selector: matchLabels: app: alb-demo-1 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: metadata: labels: app: alb-demo-1 version: v1 spec: terminationGracePeriodSeconds: 5 volumes: - name: alb-demo-1 configMap: name: alb-demo-1 containers: - name: alb-demo-1 image: nginx:latest ports: - name: http containerPort: 80 livenessProbe: httpGet: path: /_healthz port: 80 initialDelaySeconds: 3 timeoutSeconds: 2 failureThreshold: 2 volumeMounts: - name: alb-demo-1 mountPath: /etc/nginx readOnly: true resources: limits: cpu: 250m memory: 128Mi requests: cpu: 100m memory: 64Mi --- apiVersion: v1 kind: Service metadata: name: alb-demo-1 spec: selector: app: alb-demo-1 type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30081
-
Create an app:
kubectl apply -f app.yaml
Result:
configmap/alb-demo-1 created deployment.apps/alb-demo-1 created service/alb-demo-1 created
Configure the Ingress controllers
Create three Ingress
-
Create an
ingress.yaml
file with the balancer settings and domain name:apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: logs-demo-nondefault annotations: ingress.alb.yc.io/group-name: non-default ingress.alb.yc.io/subnets: <list_of_subnet_IDs> ingress.alb.yc.io/security-groups: <list_of_security_group_IDs> ingress.alb.yc.io/group-settings-name: non-default-settings ingress.alb.yc.io/external-ipv4-address: auto spec: rules: - host: <domain_name> http: paths: - pathType: Prefix path: "/" backend: service: name: alb-demo-1 port: name: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: logs-demo-disabled annotations: ingress.alb.yc.io/group-name: logs-disabled ingress.alb.yc.io/subnets: <list_of_subnet_IDs> ingress.alb.yc.io/security-groups: <list_of_security_group_IDs> ingress.alb.yc.io/group-settings-name: logs-disabled-settings ingress.alb.yc.io/external-ipv4-address: auto spec: rules: - host: <domain_name> http: paths: - pathType: Prefix path: "/" backend: service: name: alb-demo-1 port: name: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: logs-demo-default annotations: ingress.alb.yc.io/group-name: default ingress.alb.yc.io/subnets: <list_of_subnet_IDs> ingress.alb.yc.io/security-groups: <list_of_security_group_IDs> ingress.alb.yc.io/external-ipv4-address: auto spec: rules: - host: <domain_name> http: paths: - pathType: Prefix path: "/" backend: service: name: alb-demo-1 port: name: http
Where:
ingress.alb.yc.io/group-name
: Grouping of Kubernetes Ingress resources, with each group served by a separate Application Load Balancer instance.ingress.alb.yc.io/subnets
: One or more subnets that Application Load Balancer is going to work with.ingress.alb.yc.io/security-groups
: One or more security groups for Application Load Balancer. If you skip this parameter, the default security group will be used.ingress.alb.yc.io/external-ipv4-address
: Providing public online access to Application Load Balancer. When set toauto
, a new IP address is used. Deleting the Ingress controller also deletes the IP address from the cloud.ingress.alb.yc.io/group-settings-name
: Name for the group settings to be described in the optionalIngressGroupSettings
resource.
(Optional) Enter advanced controller settings:
-
ingress.alb.yc.io/internal-ipv4-address
: Provide internal access to Application Load Balancer. Enter the internal IP address or useauto
to obtain the IP address automatically.Note
You can only use one type of access to Application Load Balancer at a time:
ingress.alb.yc.io/external-ipv4-address
oringress.alb.yc.io/internal-ipv4-address
. -
ingress.alb.yc.io/internal-alb-subnet
: Subnet for hosting the Application Load Balancer internal IP address. This parameter is required if theingress.alb.yc.io/internal-ipv4-address
parameter is selected. -
ingress.alb.yc.io/protocol
: Connection protocol used by the load balancer and the backends:http
: HTTP/1.1, defaulthttp2
: HTTP/2grpc
: gRPC
-
ingress.alb.yc.io/transport-security
: Encryption protocol for connections between the load balancer and backends.Warning
In ALB Ingress Controller version 0.2.0 and later, you can only use an annotation in the Service object.
If you annotate
Ingress
resources that use a single service with the same settings for backend groups, such annotation will apply correctly. However, this mechanism is obsolete and will not be supported going forward.The acceptable value is
tls
: TLS with no certificate challenge.If no annotation is specified, the load balancer connects to the backends with no encryption.
-
ingress.alb.yc.io/prefix-rewrite
: Replace the path for the specified value. -
ingress.alb.yc.io/upgrade-types
: Valid values for theUpgrade
HTTP header, e.g.,websocket
. -
ingress.alb.yc.io/request-timeout
: Maximum period for which the connection can be established. -
ingress.alb.yc.io/idle-timeout
: Maximum connection keep-alive time with zero data transmission.Values for
request-timeout
andidle-timeout
must be specified with units of measurement, e.g.,300ms
,1.5h
. Acceptable units of measurement include:ns
: Nanosecondsus
: Microsecondsms
: Millisecondss
: Secondsm
: Minutesh
: Hours
Note
The settings only apply to the hosts of the given controller rather than the entire Ingress group.
For more information about the Ingress resource settings, see Ingress resource fields and annotations.
-
Create Ingress controllers:
kubectl apply -f ingress.yaml
Result:
ingress.networking.k8s.io/logs-demo-nondefault created ingress.networking.k8s.io/logs-demo-disabled created ingress.networking.k8s.io/logs-demo-default created
Three L7 load balancers will be automatically deployed based on the Ingress controller configurations.
Configure the Ingress controller groups
Create an IngressGroupSettings
resource with the logging settings for the Ingress controller groups:
non-default-settings
: Log to the previously created custom log group with defined rules.logs-disabled-settings
: Disable logging.
No settings need to be specified to save logs to the default log group.
-
Create a
settings.yaml
file with the log group ID:apiVersion: alb.yc.io/v1alpha1 kind: IngressGroupSettings metadata: name: non-default-settings logOptions: logGroupID: <custom_log_group_ID> discardRules: - discardPercent: 50 grpcCodes: - OK - CANCELLED - UNKNOWN - discardPercent: 67 httpCodeIntervals: - HTTP_1XX - discardPercent: 20 httpCodes: - 200 - 404 --- apiVersion: alb.yc.io/v1alpha1 kind: IngressGroupSettings metadata: name: logs-disabled-settings logOptions: disable: true
-
Apply the settings for the Ingress controller groups:
kubectl apply -f settings.yaml
Result:
ingressgroupsettings.alb.yc.io/non-default-settings created ingressgroupsettings.alb.yc.io/logs-disabled-settings created
Check the result
Get the log group IDs for the created L7 balancers and make sure they match the settings in the settings.yaml
file:
- One of the balancers should use your custom log group with defined rules.
- Another should use the default log group.
- For the third balancer, logging should be disabled.
Delete the resources you created
Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need:
- Delete the Managed Service for Kubernetes cluster.
- If you reserved a public static IP address for the cluster, delete it.
- Delete the service accounts.
- Delete the log group.
-
In the terminal window, go to the directory containing the infrastructure plan.
-
Delete the
k8s-and-registry-for-alb.tf
configuration file. -
Make sure the Terraform configuration files are correct using this command:
terraform validate
If there are any errors in the configuration files, Terraform will point them out.
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
All resources described in the
k8s-and-registry-for-alb.tf
configuration file will be deleted. -