Installing Kubernetes Node Remediation

Kubernetes Node Remediation is a tool comprising several Kubernetes operators that provide automatic recovery of failed Managed Service for Kubernetes cluster nodes and high availability for stateful workloads.

The solution consists of two controllers:

• Node Healthcheck Controller, which tracks failures.
• Self Node Remediation Controller: Transfers the workload from failed nodes and restores them.

Getting startedGetting started

1. If you do not have the Yandex Cloud CLI yet, install and initialize it.

   The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id options.

2. Make sure the security groups for the Managed Service for Kubernetes cluster and its node groups are configured correctly. If a rule is missing, add it.

   Warning

   The configuration of security groups determines performance and availability of the cluster and the services and applications running in it.

3. Install kubect and configure it to work with the new cluster.

Installation from Yandex Cloud MarketplaceInstallation from Yandex Cloud Marketplace

1. In the management console, select a folder.
2. Go to Managed Service for Kubernetes.
3. Click the name of your cluster and select the Marketplace tab.
4. Under Application available for installation, select Kubernetes Node Remediation and click Go to install.
5. Configure the application:
   • Namespace: Create a new namespace, e.g., remediation-space. If you leave the default namespace, Kubernetes Node Remediation may work incorrectly.
   • Application name: Specify the application name.
6. Click Install.
7. Wait for the application to change its status to Deployed.
8. Create the NodeHealthCheck resource.

Installation using a Helm chartInstallation using a Helm chart

1. Install Helm v3.8.0 or higher.

2. To install a Helm chart with Kubernetes Node Remediation, run this command:

   helm pull oci://cr.yandex/yc-marketplace/yandex-cloud/medik8s/kubernetes-node-remediation/chart/kubernetes-node-remediation \
     --version 1.0.1 \
     --untar && \
   helm install \
     --namespace <namespace> \
     --create-namespace \
   kubernetes-node-remediation ./kubernetes-node-remediation/
   

   If you specify the default namespace, Kubernetes Node Remediation may work incorrectly. We recommend specifying a value different from all the existing namespaces, e.g., remediation-space.

   Note

   If you are using a Helm version below 3.8.0, add the export HELM_EXPERIMENTAL_OCI=1 && \ string at the beginning of the command to enable Open Container Initiative (OCI) support in the Helm client.

3. Create the NodeHealthCheck resource.

Creating the NodeHealthCheck resourceCreating the NodeHealthCheck resource

1. Create a file named NodeHealthCheck with the resource description:

   apiVersion: remediation.medik8s.io/v1alpha1
   kind: NodeHealthCheck
   metadata:
     name: nodehc-sample
   spec:
     minHealthy: 51%
     remediationTemplate:
       apiVersion: self-node-remediation.medik8s.io/v1alpha1
       kind: SelfNodeRemediationTemplate
       name: self-node-remediation-automatic-strategy-template
       namespace: <application_namespace>
     selector:
       matchLabels:
         beta.kubernetes.io/os: linux
     unhealthyConditions:
     - duration: 60s
       status: "False"
       type: Ready
     - duration: 60s
       status: Unknown
       type: Ready
   

   Where:

   • spec.minHealthy: Minimum percentage of healthy nodes required to initiate recovery.

   • spec.unhealthyConditions: List of node status conditions the controller uses to determine if the node is unhealthy.

     • duration: Amount of time the condition must persist before the node recovery process begins.
     • type: Condition type.
     • status: Expected status for recognizing a node as unhealthy.

     In the example shown, the NodeHealthCheck controller will initiate recovery if the Ready condition type for a node indicates that it is unavailable or in a down state for 60 seconds.

   Learn more about resource fields.

2. Navigate to the directory with the file and run this command:

   kubectl apply -f <file_name>