Transferring the Yandex Managed Service for Kubernetes cluster logs to Cloud Logging
You can send Managed Service for Kubernetes cluster logs to Cloud Logging:
- Use the
master logging
setting when creating or updating a cluster to enable Managed Service for Kubernetes master log sending. The setting is only available in the API, CLI, and Terraform. - To send pod and service logs, use the Fluent Bit application with the Cloud Logging plugin in the Managed Service for Kubernetes cluster.
Sending master logs to Managed Service for Kubernetes in Cloud Logging using master logging
To set up the transfer of Managed Service for Kubernetes master logs to Cloud Logging:
If you no longer need the resources you created, delete them.
Getting started
Prepare the infrastructure:
-
If you do not have a network yet, create one.
-
If you do not have any subnets yet, create them in the availability zones where your Managed Service for Kubernetes cluster and node group will be created.
-
-
Service account for Managed Service for Kubernetes resources with the editor role for the folder where the Managed Service for Kubernetes cluster is created.
-
Service account for Managed Service for Kubernetes nodes with the container-registry.images.puller role for the folder with the Docker image registry. Managed Service for Kubernetes nodes will pull the required Docker images from the registry on behalf of this account.
Tip
You can use the same service account for both operations.
-
-
Assign the logging.writer role to the service account for resources. This role is required by the Managed Service for Kubernetes cluster to send logs to Cloud Logging.
-
Create a Managed Service for Kubernetes cluster and a node group. When creating a Managed Service for Kubernetes cluster, specify the previously created service accounts for the resources and nodes.
-
Configure security groups for the Managed Service for Kubernetes cluster.
-
If you do not have Terraform yet, install it.
-
Download the file with provider settings
. Place it in a separate working directory and specify the parameter values. -
Download the k8s-cluster-with-master-logging.tf
configuration file of the Managed Service for Kubernetes cluster to the same working directory.This file describes:
- Network.
- Subnet.
- Security groups and rules required for a Managed Service for Kubernetes cluster and node group:
- Rules for service traffic.
- Rules for accessing the Kubernetes API and managing a Managed Service for Kubernetes cluster with
kubectl
through ports 443 and 6443. - Rules for connecting to services from the internet.
- Service account for resources and Managed Service for Kubernetes nodes and sending cluster logs to Cloud Logging.
- Managed Service for Kubernetes cluster.
- Managed Service for Kubernetes node group.
- Cloud Logging log group.
-
Specify the following in the configuration file:
- Folder ID.
- Kubernetes version for the Managed Service for Kubernetes cluster and node groups.
- Name of the service account for resources and Managed Service for Kubernetes nodes and for sending cluster logs to Cloud Logging.
- Cloud Logging log group name.
-
Run the
terraform init
command in the directory with the configuration files. This command initializes the provider specified in the configuration files and enables you to use the provider resources and data sources. -
Make sure the Terraform configuration files are correct using this command:
terraform validate
If there are any errors in the configuration files, Terraform will point them out.
-
Create the required infrastructure:
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console
. -
Enable the master logging setting
If you manually created the infrastructure, enable the master logging
setting:
-
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the
--folder-name
or--folder-id
parameter. -
Run this command:
yc managed-kubernetes cluster update <cluster_name> \ --master-logging enabled=true,` `log-group-id=<log_group_ID>,` `kube-apiserver-enabled=<sending_kube-apiserver_logs:_true_or_false>,` `cluster-autoscaler-enabled=<sending_cluster-autoscaler_logs:_true_or_false>,` `events-enabled=<sending_Kubernetes_events:_true_or_false>` `audit-enabled=<sending_audit_events:_true_or_false>
Where:
enabled
: Flag that enables log sending.log-group-id
: ID of the previously created log group to send the logs to.kube-apiserver-enabled
: Flag that enables kube-apiserver log sending.cluster-autoscaler-enabled
: Flag that enablescluster-autoscaler
log sending.events-enabled
: Flag that enables Kubernetes event sending.audit-enabled
: Flag for sending audit events.
Check the result
Check the transfer of Managed Service for Kubernetes master logs to the Cloud Logging log group.
Delete the resources you created
Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need:
- Delete the Managed Service for Kubernetes cluster.
- If you reserved a static public IP address for your Managed Service for Kubernetes cluster, release and delete it.
- Delete the created subnets and networks.
- Delete the created service accounts.
- Delete the log group.
-
In the command line, go to the directory with the current Terraform configuration file with an infrastructure plan.
-
Delete the
k8s-cluster-with-master-logging.tf
configuration file. -
Make sure the Terraform configuration files are correct using this command:
terraform validate
If there are any errors in the configuration files, Terraform will point them out.
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
All the resources described in the
k8s-cluster-with-master-logging.tf
configuration file will be deleted. -
Sending pod logs and service logs via Fluent Bit
To set up the transfer of Yandex Managed Service for Kubernetes pod and service logs to Yandex Cloud Logging:
If you no longer need the resources you created, delete them.
Getting started
Prepare the infrastructure:
- If you do not have a network yet, create one.
- If you do not have any subnets, create them in the availability zones where your Managed Service for Kubernetes cluster and node group will be created.
- Create service accounts:
- Service account for Managed Service for Kubernetes resources with the editor role for the folder where the Managed Service for Kubernetes cluster is created.
- Service account for Managed Service for Kubernetes nodes with the container-registry.images.puller role for the folder with the Docker image registry. Managed Service for Kubernetes nodes will pull the required Docker images from the registry on behalf of this account.
Tip
You can use the same service account for both operations.
-
Create security groups for the Managed Service for Kubernetes cluster and its node groups.
Warning
The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.
-
Create a Managed Service for Kubernetes cluster and a node group. When creating a Managed Service for Kubernetes cluster, specify the previously created service accounts for the resources and nodes as well as the security group.
-
If you do not have Terraform yet, install it.
-
Download the file with provider settings
. Place it in a separate working directory and specify the parameter values. -
Download the k8s-cluster-with-log-group.tf
configuration file of the Managed Service for Kubernetes cluster to the same working directory.This file describes:
-
Cloud Logging log group.
-
Managed Service for Kubernetes cluster.
-
Managed Service for Kubernetes node group.
-
Service account for Managed Service for Kubernetes resources and nodes.
-
Security groups which contain rules required for the Managed Service for Kubernetes cluster and its node groups.
Warning
The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.
-
Specify the following in the configuration file:
- Folder ID.
- Kubernetes version for the Managed Service for Kubernetes cluster and node groups.
- Name of the service account for Managed Service for Kubernetes resources and nodes.
- Cloud Logging log group name.
-
Run the
terraform init
command in the directory with the configuration files. This command initializes the provider specified in the configuration files and enables you to use the provider resources and data sources. -
Check that the Terraform configuration files are correct using this command:
terraform validate
If there are any errors in the configuration files, Terraform will point them out.
-
Create the required infrastructure:
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console
. -
Install and configure Fluent Bit
Install Fluent Bit by following this guide. In the application settings, specify the ID of the log group you created earlier. You can request the log group ID with a list of log groups in the folder.
-
Install kubectl
and configure it to work with the created cluster. -
Create the objects required for Fluent Bit to run:
kubectl create namespace logging && \ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-service-account.yaml && \ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-role-1.22.yaml && \ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-role-binding-1.22.yaml
-
Create a secret containing the key of the service account:
kubectl create secret generic secret-key-json \ --from-file=key.json \ --namespace logging
-
Download the
config.yaml
configuration file:wget https://raw.githubusercontent.com/knpsh/yc-logging-fluent-bit-example/main/config.yaml
-
Specify the log output parameters in the
data.output-elasticsearch.conf
section inconfig.yaml
:... output-elasticsearch.conf: | [OUTPUT] Name yc-logging Match * group_id <log_group_ID> resource_id <optional_cluster_ID> message_key log authorization iam-key-file:/etc/secret/key.json ...
You can request the log group ID with a list of log groups in the folder.
Specify additional settings
for Fluent Bit, if required. -
Create Fluent Bit objects:
kubectl apply -f config.yaml
Result:
configmap/fluent-bit-config created daemonset.apps/fluent-bit created
-
Make sure the Fluent Bit pod has entered the
Running
state:kubectl get pods -n logging
Check the result
Test the transfer of logs of Managed Service for Kubernetes pods and services to your Cloud Logging log group.
Delete the resources you created
Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need:
- Delete the Managed Service for Kubernetes cluster.
- If you reserved a static public IP address for your Managed Service for Kubernetes cluster, release and delete it.
- Delete the created subnets and networks.
- Delete the created service accounts.
- Delete the log group.
-
In the command line, go to the directory with the current Terraform configuration file with an infrastructure plan.
-
Delete the
k8s-cluster-with-log-group.tf
configuration file. -
Check that the Terraform configuration files are correct using this command:
terraform validate
If there are any errors in the configuration files, Terraform will point them out.
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
All the resources described in the
k8s-cluster-with-log-group.tf
configuration file will be deleted. -