If you do not have Terraform yet, install it.

Download the file with provider settings. Place it in a separate working directory and specify the parameter values.

Download the k8s-cluster-with-log-group.tf configuration file of the Managed Service for Kubernetes cluster to the same working directory.

This file describes:

• Network.

• Subnet.

• Cloud Logging log group.

• Managed Service for Kubernetes cluster.

• Managed Service for Kubernetes node group.

• Service account for Managed Service for Kubernetes resources and nodes.

• Security groups which contain rules required for the Managed Service for Kubernetes cluster and its node groups.

  Warning

  The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.

Specify the following in the configuration file:

• Folder ID.
• Kubernetes version for the Managed Service for Kubernetes cluster and node groups.
• Name of the service account for Managed Service for Kubernetes resources and nodes.
• Cloud Logging log group name.

Run the terraform init command in the directory with the configuration files. This command initializes the provider specified in the configuration files and enables you to use the provider resources and data sources.

Check that the Terraform configuration files are correct using this command:

terraform validate

If there are any errors in the configuration files, Terraform will point them out.

Create the required infrastructure:

1. Run the command to view planned changes:

   terraform plan
   

   If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

2. If you are happy with the planned changes, apply them:

   1. Run the command:

      terraform apply
      

   2. Confirm the update of resources.

   3. Wait for the operation to complete.

All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

Install kubectl and configure it to work with the created cluster.

Create the objects required for Fluent Bit to run:

kubectl create namespace logging && \
kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-service-account.yaml && \
kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-role-1.22.yaml && \
kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-role-binding-1.22.yaml

Create a secret containing the key of the service account:

kubectl create secret generic secret-key-json \
  --from-file=key.json \
  --namespace logging

Download the config.yaml configuration file:

wget https://raw.githubusercontent.com/knpsh/yc-logging-fluent-bit-example/main/config.yaml

Specify the log output parameters in the data.output-elasticsearch.conf section in config.yaml:

...
  output-elasticsearch.conf: |
    [OUTPUT]
      Name            yc-logging
      Match           *
      group_id        <log_group_ID>
      resource_id     <optional_cluster_ID>
      message_key     log
      authorization   iam-key-file:/etc/secret/key.json
...

You can request the log group ID with a list of log groups in the folder.

Specify additional settings for Fluent Bit, if required.

Create Fluent Bit objects:

kubectl apply -f config.yaml

Result:

configmap/fluent-bit-config created
daemonset.apps/fluent-bit created

Make sure the Fluent Bit pod has entered the Running state:

kubectl get pods -n logging

In the command line, go to the directory with the current Terraform configuration file with an infrastructure plan.

Delete the k8s-cluster-with-log-group.tf configuration file.

Check that the Terraform configuration files are correct using this command:

terraform validate

If there are any errors in the configuration files, Terraform will point them out.

Confirm updating the resources.

1. Run the command to view planned changes:

   terraform plan
   

   If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

2. If you are happy with the planned changes, apply them:

   1. Run the command:

      terraform apply
      

   2. Confirm the update of resources.

   3. Wait for the operation to complete.

All the resources described in the k8s-cluster-with-log-group.tf configuration file will be deleted.