Integration with Object Storage

Container Storage Interface allows you to dynamically reserve Yandex Object Storage buckets and mount them to Managed Service for Kubernetes cluster pods. You can mount existing buckets or create new ones.

To use the Container Storage Interface features:

1. Set up the runtime environment.
2. Configure Container Storage Interface.

See also:

• Using Container Storage Interface with a PersistentVolume.
• PersistentVolume creation examples.

Setting up a runtime environmentSetting up a runtime environment

1. Create a service account with the storage.editor role.
2. Create a static access key for your service account. Save the key ID and secret key; you will need them when installing Container Storage Interface.
3. Create an Object Storage bucket that will be mounted to a PersistentVolume. Save the bucket name; you will need it when installing Container Storage Interface.

4. Install kubect and configure it to work with the new cluster.

Configuring Container Storage InterfaceConfiguring Container Storage Interface

Using Container Storage InterfaceUsing Container Storage Interface

With Container Storage Interface configured, there are certain aspects to consider when creating static and dynamic PersistentVolumes objects.

Dynamic PersistentVolumeDynamic PersistentVolume

For a dynamic PersistentVolume:

• Specify the name of the storage class in the spec.storageClassName parameter when creating a PersistentVolumeClaim.
• If required, specify the bucket name in the bucket parameter (or the Shared S3 bucket for volumes field in the Yandex Cloud Marketplace settings) when creating a storage class. This affects Container Storage Interface behavior:

  • If you specify the bucket name when configuring your storage class, Container Storage Interface will create a separate folder in this bucket for each created PersistentVolume.

    Note

    This setting can be useful if the cloud enforces strict quotas on the number of Object Storage buckets.

  • If you skip the bucket name, Container Storage Interface will create a separate bucket for each created PersistentVolume.

See also the example of creating a dynamic PersistentVolume.

Static PersistentVolumeStatic PersistentVolume

For a static PersistentVolume:

• When creating a PersistentVolumeClaim, set an empty value for the spec.storageClassName parameter.

• Then, specify the name of the bucket or bucket directory in the spec.csi.volumeHandle parameter. If there is no such bucket, create one.

  Note

  Deleting such a PersistentVolume will not automatically delete its associated bucket.

• To update GeeseFS options for working with a bucket, specify them in the spec.csi.volumeAttributes.options parameter when creating a PersistentVolume. For example, in the --uid option, you can specify the ID of the owner of all the files in a storage. To get a list of GeeseFS options, run the geesefs -h command or refer to the relevant GitHub repository.

  The GeeseFS options specified under parameters.options (or the GeeseFS mount options field in the Yandex Cloud Marketplace settings) in StorageClass are ignored for a static PersistentVolume. For more information, see this Kubernetes guide.

See also the example of creating a static PersistentVolume.

Use casesUse cases

Dynamic PersistentVolumeDynamic PersistentVolume

To use Container Storage Interface with a dynamic PersistentVolume:

1. Configure Container Storage Interface.

2. Create a PersistentVolumeClaim:

   1. Create a file named pvc-dynamic.yaml with the PersistentVolumeClaim description:

      pvc-dynamic.yaml

      ---
      apiVersion: v1
      kind: PersistentVolumeClaim
      metadata:
        name: csi-s3-pvc-dynamic
        namespace: default
      spec:
        accessModes:
        - ReadWriteMany
        resources:
          requests:
            storage: 5Gi
        storageClassName: csi-s3
      

      You can change the requested storage size as you need in the spec.resources.requests.storage parameter value.

   2. Create a PersistentVolumeClaim:

      kubectl create -f pvc-dynamic.yaml
      

   3. Make sure the PersistentVolumeClaim status changed to Bound:

      kubectl get pvc csi-s3-pvc-dynamic
      

      Result:

      NAME                STATUS  VOLUME                    CAPACITY  ACCESS MODES  STORAGECLASS  AGE
      csi-s3-pvc-dynamic  Bound   pvc-<dynamic_bucket_name>  5Gi       RWX           csi-s3        73m
      

3. Create a pod to test your dynamic PersistentVolume.

   1. Create a file named pod-dynamic.yaml with the pod description:

      pod-dynamic.yaml

      ---
      apiVersion: v1
      kind: Pod
      metadata:
        name: csi-s3-test-nginx-dynamic
        namespace: default
      spec:
        containers:
        - name: csi-s3-test-nginx
          image: nginx
          volumeMounts:
            - mountPath: /usr/share/nginx/html/s3
              name: webroot
        volumes:
          - name: webroot
            persistentVolumeClaim:
              claimName: csi-s3-pvc-dynamic
              readOnly: false
      

   2. Create a pod to mount a bucket to for your dynamic PersistentVolume:

      kubectl create -f pod-dynamic.yaml
      

   3. Make sure the pod has switched to Running:

      kubectl get pods
      

4. Create the /usr/share/nginx/html/s3/hello_world file in the container by running the following command on the pod:

   kubectl exec -ti csi-s3-test-nginx-dynamic -- touch /usr/share/nginx/html/s3/hello_world
   

5. Make sure the file is in the bucket:

   1. Navigate to the folder dashboard and select Object Storage.
   2. Click the pvc-<dynamic_bucket_name> bucket. If you specified a bucket name when configuring your storage class, open the bucket and the pvc-<dynamic_bucket_name> folder located inside.

Static PersistentVolumeStatic PersistentVolume

To use Container Storage Interface with a static PersistentVolume:

1. Configure Container Storage Interface.

2. Create a PersistentVolumeClaim:

   1. Create a file named pvc-static.yaml with the PersistentVolumeClaim description:

      pvc-static.yaml

      ---
      apiVersion: v1
      kind: PersistentVolumeClaim
      metadata:
        name: csi-s3-pvc-static
        namespace: default
      spec:
        accessModes:
          - ReadWriteMany
        resources:
          requests:
            storage: 10Gi
        storageClassName: ""
      

      For a static PersistentVolume, you do not need to specify the storage class name in the spec.storageClassName parameter. You can change the requested storage size as you need in the spec.resources.requests.storage parameter value.

   2. Create a file named pv-static.yaml with the static PersistentVolume description, and specify the volumeHandle parameter value in it:

      pv-static.yaml

      ---
      apiVersion: v1
      kind: PersistentVolume
      metadata:
        name: s3-volume
      spec:
        storageClassName: csi-s3
        capacity:
          storage: 10Gi
        accessModes:
          - ReadWriteMany
        claimRef:
          namespace: default
          name: csi-s3-pvc-static
        csi:
          driver: ru.yandex.s3.csi
          volumeHandle: "<static_bucket_name>/<path_to_folder_in_bucket>"
          controllerPublishSecretRef:
            name: csi-s3-secret
            namespace: kube-system
          nodePublishSecretRef:
            name: csi-s3-secret
            namespace: kube-system
          nodeStageSecretRef:
            name: csi-s3-secret
            namespace: kube-system
          volumeAttributes:
            capacity: 10Gi
            mounter: geesefs
            options: "--memory-limit=1000 --dir-mode=0777 --file-mode=0666 --uid=1001"
      

      Here, the GeeseFS settings for working with a bucket are different compared to StorageClass. There is an additional --uid option which specifies the ID of the owner of all the files in the storage: 1001. See above for more information on setting up GeeseFS for a static PersistentVolume.

      Optionally, set the path to the folder in the bucket in the volumeHandle parameter.

   3. Create a PersistentVolumeClaim:

      kubectl create -f pvc-static.yaml
      

   4. Create a static PersistentVolume:

      kubectl create -f pv-static.yaml
      

   5. Make sure the PersistentVolumeClaim status changed to Bound:

      kubectl get pvc csi-s3-pvc-static
      

      Result:

      NAME               STATUS  VOLUME                  CAPACITY  ACCESS MODES  STORAGECLASS  AGE
      csi-s3-pvc-static  Bound   <PersistentVolume_name>  10Gi      RWX           csi-s3        73m
      

3. Create a pod to test your static PersistentVolume.

   1. Create a file named pod-static.yaml with the pod description:

      pod-static.yaml

      ---
      apiVersion: v1
      kind: Pod
      metadata:
        name: csi-s3-test-nginx-static
        namespace: default
      spec:
        containers:
        - name: csi-s3-test-nginx-static
          image: nginx
          volumeMounts:
            - mountPath: /usr/share/nginx/html/s3
              name: s3-volume
        volumes:
          - name: s3-volume
            persistentVolumeClaim:
              claimName: csi-s3-pvc-static
              readOnly: false
      

   2. Create a pod to mount a bucket to for your static PersistentVolume:

      kubectl create -f pod-static.yaml
      

   3. Make sure the pod has switched to Running:

      kubectl get pods
      

4. Create the /usr/share/nginx/html/s3/hello_world_static file in the container by running the following command on the pod:

   kubectl exec -ti csi-s3-test-nginx-static -- touch /usr/share/nginx/html/s3/hello_world_static
   

5. Make sure the file is in the bucket:

   1. Navigate to the folder dashboard and select Object Storage.
   2. Click <static_bucket_name>. If you specified the path to the folder located inside the bucket in the static PersistentVolume description, you need to open that folder first.