Managing Kubernetes resources in a Yandex Managed Service for Kubernetes cluster via the Terraform provider
You can use Terraform manifests to create Kubernetes resources. To do this, activate the kubernetes
Terraform provider. It supports Terraform resources that are mapped to YAML configuration files for various Kubernetes resources.
It is convenient to create Kubernetes resources with Terraform if you are already using Terraform to support the infrastructure for a Yandex Managed Service for Kubernetes cluster. This way, you will be able to describe all resources in the same markup language.
In addition, Terraform tracks dependencies between resources and prevents creation, changes, or deletion of a resource if its dependencies are not ready. Let’s assume you are creating a resource named PersistentVolumeClaim
. It needs space in the PersistentVolume
resource storage, but there is not enough free space there. Terraform will detect the lack of free space and prevent creation of the PersistentVolumeClaim
resource.
The example below illustrates how to create standard Kubernetes resources using Terraform.
To create Kubernetes resources with Terraform:
- Set up your infrastructure.
- Activate the
kubernetes
provider. - Create Kubernetes resources.
- Make sure the cluster application is available from the internet.
If you no longer need the resources you created, delete them.
Required paid resources
The support cost includes:
- Fee for using the master and outgoing traffic in a Managed Service for Kubernetes cluster (see Managed Service for Kubernetes pricing).
- Fee for using computing resources, OS, and storage in cluster nodes (VMs) (see Compute Cloud pricing).
- Fee for an NLB (see Network Load Balancer pricing).
- Fee for using public IP addresses for the VM and NLB (see Virtual Private Cloud pricing).
Prepare the infrastructure for Managed Service for Kubernetes
-
If you do not have Terraform yet, install it.
-
Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.
-
Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it
.At this step, the file should not contain
kubernetes
provider settings. You will add them at the next steps. -
Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.
-
Download the managed-k8s-infrastructure.tf
configuration file to the same working directory.This file describes:
- Network.
- Subnet.
- Two security groups: one for the cluster and one for the node group.
- Cloud service account with the
k8s.clusters.agent
,vpc.publicAdmin
,load-balancer.admin
, andcontainer-registry.images.puller
roles. - Managed Service for Kubernetes cluster.
- Kubernetes node group.
-
Specify the values of variables in the
managed-k8s-infrastructure.tf
file. -
Make sure the Terraform configuration files are correct using this command:
terraform validate
If there are any errors in the configuration files, Terraform will point them out.
-
Create an infrastructure:
-
Run this command to view the planned changes:
terraform plan
If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply
-
Confirm updating the resources.
-
Wait for the operation to complete.
-
All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console
. -
-
Install kubect
and configure it to work with the new cluster.
Activate the Kubernetes provider
-
In the working directory, open the file with
yandex
provider settings. It must have the following structure:terraform { required_providers { yandex = { source = "yandex-cloud/yandex" } } required_version = ">= 0.13" } provider "yandex" { token = "<IAM_token>" cloud_id = "<cloud_ID>" folder_id = "<folder_ID>" zone = "<default_availability_zone>" }
-
In the file, specify the parameters required for the
kubernetes
provider to operate:-
Under
required_providers
, add:kubernetes = { source = "hashicorp/kubernetes" }
-
Under
required_providers
, changerequired_version
to">= 0.14.8"
. -
Add a new section at the end of the file:
data "yandex_client_config" "client" {} provider "kubernetes" { host = yandex_kubernetes_cluster.k8s-cluster.master[0].external_v4_endpoint cluster_ca_certificate = yandex_kubernetes_cluster.k8s-cluster.master[0].cluster_ca_certificate token = data.yandex_client_config.client.iam_token }
-
-
Make sure the resulting file looks like this:
terraform { required_providers { yandex = { source = "yandex-cloud/yandex" } kubernetes = { source = "hashicorp/kubernetes" } } required_version = ">= 0.14.8" } provider "yandex" { token = "<IAM_token>" cloud_id = "<cloud_ID>" folder_id = "<folder_ID>" zone = "<default_availability_zone>" } data "yandex_client_config" "client" {} provider "kubernetes" { host = yandex_kubernetes_cluster.k8s-cluster.master[0].external_v4_endpoint cluster_ca_certificate = yandex_kubernetes_cluster.k8s-cluster.master[0].cluster_ca_certificate token = data.yandex_client_config.client.iam_token }
-
Initialize the
kubernetes
provider:terraform init
Create Kubernetes resources
Create a test application and service of the LoadBalancer
type:
-
In the working directory, create a file named
deployment.tf
describing theDeployment
resource:resource "kubernetes_deployment" "demo-app-deployment" { metadata { name = "hello" labels = { app = "hello" version = "v1" } } spec { replicas = 2 selector { match_labels = { app = "hello" } } template { metadata { labels = { app = "hello" version = "v1" } } spec { container { name = "hello-app" image = "cr.yandex/crpjd37scfv653nl11i9/hello:1.1" } } } } }
-
In the working directory, create a file named
service.tf
describing theService
resource:resource "kubernetes_service" "demo-lb-service" { metadata { name = "hello" } spec { selector = { app = kubernetes_deployment.demo-app-deployment.spec.0.template.0.metadata[0].labels.app } type = "LoadBalancer" port { port = 80 target_port = 8080 } } }
-
Create Kubernetes resources:
-
View the planned changes:
terraform plan
-
If the changes are acceptable, apply them:
terraform apply
After you run the
terraform apply
command, you may get this error:Error: Waiting for rollout to finish: 2 replicas wanted; 0 replicas Ready │ │ with kubernetes_deployment.demo-app-deployment, │ on deployment.tf line 1, in resource "kubernetes_deployment" "demo-app-deployment": │ 1: resource "kubernetes_deployment" "demo-app-deployment" { │
It means the
Deployment
resources are not ready yet. Check their readiness using thekubectl get deployment
command, which will return this result:NAME READY UP-TO-DATE AVAILABLE AGE hello 0/2 2 0 12m
When the
READY
column indicates2/2
, run theterraform apply
command again. -
You can also create other standard Kubernetes resources using Terraform manifests. Use the YAML configuration of the resource you need as the basis (here is an example for a podcontainerPort
parameter from the YAML file with the container_port
parameter in Terraform. For a full list of Terraform resources for Kubernetes, see the relevant provider documentation
For information about creating custom resources
Make sure the cluster application is available from the internet
-
View information about the created load balancer:
kubectl describe service hello
Result:
Name: hello Namespace: default Labels: <none> Annotations: <none> Selector: app=hello Type: LoadBalancer IP Family Policy: SingleStack IP Families: IPv4 IP: 10.96.228.81 IPs: 10.96.228.81 LoadBalancer Ingress: 84.201.148.8 Port: <unset> 80/TCP TargetPort: 8080/TCP NodePort: <unset> 32532/TCP Endpoints: 10.112.128.7:8080,10.112.128.8:8080 Session Affinity: None External Traffic Policy: Cluster Internal Traffic Policy: Cluster Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal EnsuringLoadBalancer 5m32s service-controller Ensuring load balancer Normal EnsuredLoadBalancer 5m25s service-controller Ensured load balancer
-
Copy the IP address from the
LoadBalancer Ingress
field. -
Open the application's URL in your browser:
http://<copied_IP_address>
Result:
Hello, world! Running in 'hello-5c46b*****-nc**'
Delete the resources you created
-
In the terminal window, go to the directory containing the infrastructure plan.
-
Run this command:
terraform destroy
Terraform will delete all resources you created in the current configuration.