Using HashiCorp Vault to store secrets

HashiCorp Vault is an open-source tool for securely storing and accessing different kinds of secrets, such as passwords, certificates, and tokens.

Configure secret storage and access within your Yandex Managed Service for Kubernetes cluster using HashiCorp Vault with Key Management Service support from Yandex Cloud Marketplace.

This tutorial shows how to mount a secret from HashiCorp Vault using a Container Storage Interface (CSI) volume.

To set up access to a secret in a Managed Service for Kubernetes cluster using HashiCorp Vault:

1. Get your cloud ready.
2. Install HashiCorp Vault.
3. Log in to HashiCorp Vault.
4. Create a secret.
5. Configure the Kubernetes authentication method.
6. Install the SCI driver for the secret vault.
7. Create a SecretProviderClass resource.
8. Create a pod with the secret mounted.

If you no longer need the resources you created, delete them.

Required paid resourcesRequired paid resources

The support cost for this solution includes:

• Fee for using the master and outgoing traffic in a Managed Service for Kubernetes cluster (see Managed Service for Kubernetes pricing).
• Fee for using computing resources, OS, and storage in cluster nodes (VMs) (see Compute Cloud pricing).
• Fee for a public IP address assigned to cluster nodes (see Virtual Private Cloud pricing).
• Key Management Service fee for the number of active key versions (with Active or Scheduled For Destruction for status) and completed cryptographic operations (see Key Management Service pricing).

Get your cloud readyGet your cloud ready

1. Create a Kubernetes cluster and node group.

   Manually

   Using Terraform

   1. If you do not have a network yet, create one.

   2. If you do not have any subnets yet, create them in the availability zones where the new Kubernetes cluster and node group will reside.

   3. Create these service accounts:

      • Service account with the k8s.clusters.agent and vpc.publicAdmin roles for the folder where you want to create a Kubernetes cluster. This service account will be used to create resources for your Kubernetes cluster.
      • Service account with the container-registry.images.puller role. The nodes will use this account to pull the required Docker images from the registry.

      Tip

      You can use the same service account to manage your Kubernetes cluster and its node groups.

   4. Create security groups for the Managed Service for Kubernetes cluster and its node groups.

      Warning

      The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.

   5. Create a Kubernetes cluster and node group with any suitable configuration. When creating, specify the preconfigured security groups.

   1. If you do not have Terraform yet, install it.

   2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

   3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

   4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

   5. Download the k8s-cluster.tf cluster configuration file to the same working directory. This file describes:

      • Network.

      • Subnet.

      • Kubernetes cluster.

      • Service account for the Managed Service for Kubernetes cluster and node group.

      • Security groups which contain rules required for the Managed Service for Kubernetes cluster and its node groups.

        Warning

        The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.

   6. Specify the following in k8s-cluster.tf:

      • Folder ID.
      • Kubernetes version for the Kubernetes cluster and node groups.
      • Kubernetes cluster CIDR.
      • Name of the Managed Service for Kubernetes cluster service account.

   7. Make sure the Terraform configuration files are correct using this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

   8. Create the required infrastructure:

      1. Run this command to view the planned changes:

         terraform plan
         

         If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

      2. If everything looks correct, apply the changes:

         1. Run this command:

            terraform apply
            

         2. Confirm updating the resources.

         3. Wait for the operation to complete.

      All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

2. Install kubect and configure it to work with the new cluster.

Install HashiCorp VaultInstall HashiCorp Vault

Install HashiCorp Vault using Helm and initialize the vault by following this guide. In the installation command, specify the hcv namespace and add extra parameters to set up the Vault CSI provider:

--namespace hcv \
--set "injector.enabled=false" \
--set "csi.enabled=true"

Log in to HashiCorp VaultLog in to HashiCorp Vault

1. Start an interactive HashiCorp Vault shell session for the hashicorp-vault-0 pod:

   kubectl exec -it hashicorp-vault-0 \
      --namespace hcv \
      -- /bin/sh
   

2. Unseal the vault:

   vault operator unseal
   

   Enter one of the recovery keys you got during vault initialization.

3. Log in to HashiCorp Vault with the root token:

   vault login
   

   Enter the root token (Initial Root Token) you got during vault initialization.

Create a secretCreate a secret

1. Enable the kv secrets engine at the secret path:

   vault secrets enable -path=secret kv
   

2. Create a secret at secret/db-pass with the password:

   vault kv put secret/db-pass password="12345678"
   

3. Make sure the secret is available for reading at secret/db-pass:

   vault kv get secret/db-pass
   

   Result:

   ====== Data ======
   Key         Value
   ---         -----
   password    12345678
   

Configure the Kubernetes authentication methodConfigure the Kubernetes authentication method

This method allows authentication with a Kubernetes service account token.

1. Enable the Kubernetes authentication method:

   vault auth enable kubernetes
   

2. Configure authentication using the Kubernetes API address:

   vault write auth/kubernetes/config \
      kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443"
   

   The KUBERNETES_PORT_443_TCP_ADDR environment variable refers to the internal network address of the Kubernetes node.

3. Create a policy named internal-app that will allow the Kubernetes service account to read the secret you created:

   vault policy write internal-app - <<EOF
   path "secret/db-pass" {
     capabilities = ["read"]
   }
   EOF
   

4. Create the database role that will associate the internal-app policy with the Kubernetes webapp-sa service account (you will create it later):

   vault write auth/kubernetes/role/database \
      bound_service_account_names=webapp-sa \
      bound_service_account_namespaces=hcv \
      policies=internal-app \
      ttl=20m
   

   Tokens returned upon authentication are valid for 20 minutes.

5. Exit HashiCorp Vault:

   exit
   

Install the SCI driver for the secret vaultInstall the SCI driver for the secret vault

1. Add the secrets-store-csi-driver Helm repository:

   helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
   

2. Install the SCI driver:

   helm install csi secrets-store-csi-driver/secrets-store-csi-driver \
      --namespace=hcv \
      --set syncSecret.enabled=true
   

3. Make sure the driver is running and ready:

   kubectl get pods -n hcv -l "app=secrets-store-csi-driver"
   

   Result:

   NAME                                 READY   STATUS    RESTARTS   AGE
   csi-secrets-store-csi-driver-nbxcd   3/3     Running   0          4m28s
   

Create a SecretProviderClass resourceCreate a SecretProviderClass resource

1. Create a file named spc-vault-database.yaml with settings for the CSI provider:

   spc-vault-database.yaml

   apiVersion: secrets-store.csi.x-k8s.io/v1
   kind: SecretProviderClass
   metadata:
     name: vault-database
   spec:
     provider: vault
     parameters:
       vaultAddress: "http://hashicorp-vault.hcv:8200"
       roleName: "database"
       objects: |
         - objectName: "db-password"
           secretPath: "secret/db-pass"
           secretKey: "password"
   

2. Create the SecretProviderClass resource:

   kubectl apply -f spc-vault-database.yaml -n hcv
   

Create a pod with the secret mountedCreate a pod with the secret mounted

1. Create a service account named webapp-sa for the Kubernetes cluster:

   kubectl create serviceaccount webapp-sa \
      --namespace hcv
   

2. Create a file named webapp-pod.yaml containing the webapp pod configuration:

   spc-vault-database.yaml

   kind: Pod
   apiVersion: v1
   metadata:
     name: webapp
   spec:
     serviceAccountName: webapp-sa
     containers:
     - image: jweissig/app:0.0.1
       name: webapp
       volumeMounts:
       - name: secrets-store-inline
         mountPath: "/mnt/secrets-store"
         readOnly: true
     volumes:
       - name: secrets-store-inline
         csi:
           driver: secrets-store.csi.k8s.io
           readOnly: true
           volumeAttributes:
             secretProviderClass: "vault-database"
   

3. Create a pod named webapp:

   kubectl apply -f webapp-pod.yaml -n hcv 
   

4. Make sure the webapp pod is running and ready:

   kubectl get pod webapp -n hcv
   

   Result:

   NAME     READY   STATUS    RESTARTS   AGE
   webapp   1/1     Running   0          5m25s
   

5. Display the secret password stored at /mnt/secrets-store/db-password in the file system:

   kubectl exec webapp -n hcv -- cat /mnt/secrets-store/db-password
   

   Result:

   12345678
   

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them.

See alsoSee also

• HashiCorp Vault documentation
• Installing HashiCorp Vault with Key Management Service support
• Installing the External Secrets Operator with Yandex Lockbox support
• Syncing with Yandex Lockbox secrets