Installing Chaos Mesh

Chaos Mesh is an open-source platform that you can use to simulate various failures and faults in your Kubernetes clusters. You can perform simulation at different development and testing stages, as well as after release. You can use Chaos Mesh to detect and eliminate potential threats and set up security policies, e.g., using Kyverno.

Getting startedGetting started

1. Create a node group with at least 14 GB of RAM.

2. Make sure that the security groups for the Managed Service for Kubernetes cluster and its node groups are configured correctly. If any rule is missing, add it.

   Warning

   The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.

Installation from Yandex Cloud MarketplaceInstallation from Yandex Cloud Marketplace

1. Navigate to the folder dashboard and select Managed Service for Kubernetes.

2. Click the name of the Managed Service for Kubernetes cluster you need and select the Marketplace tab.

3. Under Application available for installation, select Chaos Mesh and click Go to install.

4. Configure the application:

   • Namespace: Create a new namespace, e.g., chaos-mech-space. If you leave the default namespace, Chaos Mesh may work incorrectly.
   • Application name: Specify the application name.

5. Click Install.

6. Wait for the application status to change to Deployed.

Installation using a Helm chartInstallation using a Helm chart

1. Install Helm v3.8.0 or higher.

2. Install kubect and configure it to work with the new cluster.

3. To install a Helm chart with Chaos Mesh, run this command:

   helm pull oci://cr.yandex/yc-marketplace/yandex-cloud/chaos-mesh/chart/chaos-mesh \
     --version 2.8.0 \
     --untar && \
   helm install \
     --namespace <namespace> \
     --create-namespace \
     chaos-mesh ./chaos-mesh/
   

   If you set namespace to the default namespace, Chaos Mesh may work incorrectly. We recommend specifying a value different from all the existing namespaces, e.g., chaos-mesh-space.

   Note

   If you are using a Helm version below 3.8.0, append the export HELM_EXPERIMENTAL_OCI=1 && \ string to the command to enable Open Container Initiative (OCI) support in the Helm client.

You can check the current version of the Helm chart on the application page.

Chaos Mesh web interface and authorizationChaos Mesh web interface and authorization

Once you install Chaos Mesh, use its web interface to go through the authorization process. Proceed as follows:

1. Connect to your cluster.

2. Forward the http://localhost:8080 URL to the Chaos Mesh web interface:

   kubectl port-forward service/chaos-dashboard 8080:2333 \
      --namespace <namespace>
   

   In the command, specify the namespace you selected when installing the application.

   Result:

   Forwarding from 127.0.0.1:8080 -> 2333
   Forwarding from [::1]:8080 -> 2333
   

3. In your browser, go to http://localhost:8080. This will open the authorization window of the Chaos Mesh web interface.

4. To log in, you need a Kubernetes account and token. Get them using one of the two methods:

   • Create an account and token for Chaos Mesh. For the account, you can set up access permissions in a specific namespace.

   • For Kubernetes clusters 1.24 or higher, use a Kubernetes service account and create a token for it:

     1. View a list of available Kubernetes service accounts:

        kubectl get serviceAccounts
        

        Each account has its own access permissions.

     2. Select the account you need and create a token:

        kubectl create token <account_name>
        

Working with Yandex Cloud resourcesWorking with Yandex Cloud resources

The application features the YCChaos scenario for simulation of VM node failures, e.g., restarts or stops. You can use it to check fault tolerance of systems dependent on the cloud infrastructure.

For example, to use YCChaos to simulate a VM node restart:

1. Create a service account and assign it the compute.operator role.

2. Issue an authorized key and save it to a file named sa-key.json.

3. Create a namespace named chaos-testing.

4. Create a Kubernetes secret based on the authorized key you created:

   kubectl create secret generic yc-sa-secret \
       --from-file=sa-key.json=./sa-key.json \
       -n chaos-testing
   

5. Save the workflow example with the YCChaos scenario to the chaos.yaml file:

   apiVersion: chaos-mesh.org/v1alpha1
   kind: Workflow
   metadata:
     name: yc-random-batch
     namespace: chaos-testing
   spec:
     entry: parallel
     templates:
     - name: parallel
       templateType: Parallel
       children: [yc-1, yc-2]
   
     - name: yc-1
       templateType: YCChaos
       deadline: 5m
       ycChaos:
         action: compute-restart
         computeInstance: <VM_1_ID>
         secretName: yc-sa-secret
   
     - name: yc-2
       templateType: YCChaos
       deadline: 5m
       ycChaos:
         action: compute-restart
         computeInstance: <VM_2_ID>
         secretName: yc-sa-secret
   

6. Create a workflow using this command:

   kubectl apply -f chaos.yaml
   

7. Make sure the specified VMs restart after you create the workflow.

   You can view the workflow results in the Chaos Mesh web interface.

See alsoSee also

• Chaos Mesh documentation