Deleting all objects in a bucket

Written by

Yandex Cloud

Updated at November 5, 2024

Note

To automatically delete partially loaded objects in Object Storage, configure a lifecycle rule.

To clear a bucket and not pay for storage:

AWS CLI

Python (boto3)

API

If you do not have the AWS CLI yet, install and configure it.

Create a variable containing the bucket name:
```
BUCKET_NAME=<bucket_name>
```

Delete all versions of objects from the bucket:

aws s3api delete-objects \
  --endpoint-url https://storage.yandexcloud.net \
  --bucket $BUCKET_NAME \
  --delete \
    "$(aws s3api list-object-versions \
      --endpoint-url https://storage.yandexcloud.net \
      --bucket $BUCKET_NAME \
      --query '{Objects: Versions[].{Key: Key, VersionId: VersionId}}' \
      --max-items 1000)"

You can also use --max-keys instead of the --max-items parameter.

Result:

{
    "Deleted": [
        {
            "Key": "object_000",
            "VersionId": "0005CDD3********"
        },
        {
            "Key": "object_001",
            "VersionId": "0005CDD3********"
        },
        ...
    ]
}

Using this command, you can delete up to 1,000 versions of objects: this is related to restrictions for the aws s3api delete-objects operation. If there are more versions in the bucket, run the command several times.

If the bucket has versioning enabled, delete all the delete markers:

aws s3api delete-objects \
  --endpoint-url https://storage.yandexcloud.net \
  --bucket $BUCKET_NAME \
  --delete \
    "$(aws s3api list-object-versions \
      --endpoint-url https://storage.yandexcloud.net \
      --bucket $BUCKET_NAME \
      --query '{Objects: DeleteMarkers[].{Key: Key, VersionId: VersionId}}' \
      --max-items 1000)"

You can also use --max-keys instead of the --max-items parameter.

Result:

{
    "Deleted": [
        {
            "Key": "object_034",
            "DeleteMarker": true,
            "DeleteMarkerVersionId": "0005CDD3********"
        },
        {
            "Key": "object_057",
            "DeleteMarker": true,
            "DeleteMarkerVersionId": "0005CDD3********"
        },
        ...
    ]
}

Using this command, you can delete up to 1,000 delete markers: this is related to restrictions for the aws s3api delete-objects operation. If there are more versions in the bucket, run the command several times.

Delete partially uploaded objects:

aws s3api list-multipart-uploads \
  --endpoint-url https://storage.yandexcloud.net \
  --bucket $BUCKET_NAME \
| jq -r '.Uploads[] | "--key \"\(.Key)\" --upload-id \(.UploadId)"' \
| while read -r line; do
  eval
    "aws s3api abort-multipart-upload \
      --endpoint-url https://storage.yandexcloud.net \
      --bucket $BUCKET_NAME \
      $line";
done

Get a list of object parts remaining in the bucket:

aws s3api list-multipart-uploads \
  --endpoint-url https://storage.yandexcloud.net \
  --bucket $BUCKET_NAME \
| jq -r '.Uploads[] | "--key \"\(.Key)\" --upload-id \(.UploadId)"' \
| while read -r line; do
  eval
    "aws s3api list-parts \
      --endpoint-url https://storage.yandexcloud.net \
      --bucket $BUCKET_NAME \
      $line";
done

The list may contain parts of objects that began uploading before and finished uploading after the previous step executed. If the list is not empty, repeat steps 4 and 5.

Run the following code:

import boto3

bucket_name = '<bucket_name>'
s3 = boto3.resource('s3',
    endpoint_url='https://storage.yandexcloud.net',
    aws_access_key_id='<key_ID>',
    aws_secret_access_key='<secret_key>')
bucket = s3.Bucket(bucket_name)

# Deleting all versions (works for non-versioned buckets too).
bucket.object_versions.delete()

# Aborting all multipart uploads, which also deletes all parts.
for multipart_upload in bucket.multipart_uploads.iterator():
    # Part uploads that are currently in progress may or may not succeed,
    # so it might be necessary to abort a multipart upload multiple times.
    while len(list(multipart_upload.parts.all())) > 0:
        multipart_upload.abort()

Use the deleteMultipleObjects S3 API method.

In the management console, information about the number of objects in a bucket and the used space is updated with a few minutes' delay.

Deleting all objects in a bucket

Was the article helpful?