Deleting all objects from a bucket
Warning
By default, the delete operation is irreversible. You can only restore the objects that were uploaded with versioning on.
Note
To automatically delete partially loaded objects in Object Storage, configure a lifecycle rule.
To clear a bucket and avoid storagecharges:
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
-
See the description of the CLI command for deleting objects:
yc storage s3api delete-objects --help -
Get a list of buckets in the default folder:
yc storage bucket listResult:
+------------------+----------------------+-------------+-----------------------+---------------------+ | NAME | FOLDER ID | MAX SIZE | DEFAULT STORAGE CLASS | CREATED AT | +------------------+----------------------+-------------+-----------------------+---------------------+ | first-bucket | b1gmit33ngp6******** | 53687091200 | STANDARD | 2022-12-16 13:58:18 | +------------------+----------------------+-------------+-----------------------+---------------------+ -
Prepare a JSON file listing the objects to delete. Here is an example of its contents:
{ "Objects": [ { "Key": "<object_key>" }, { "Key": "<object_key>" }, ... ] }Where:
Objects: Array of objects to delete.Key: Object key.
-
Run this command:
yc storage s3api delete-objects \ --bucket <bucket_name> \ --delete <path_to_JSON_file>Where:
--bucket: Name of your bucket.--delete: Path to the JSON file listing the objects to delete.
Result:
deleted: - key: file-1 version_id: "null" - key: file-2 version_id: "null" request_id: 4c35e7d4********Alternative command:
yc storage s3 rm \ s3://<bucket_name>/ \ --recursiveWhere
--recursiveis the parameter to delete all objects.Result:
delete: s3://my-bucket/object-1.txt delete: s3://my-bucket/object-2.txt ... delete: s3://my-bucket/object-n.txt
If you do not have the AWS CLI yet, install and configure it.
-
Create a variable containing the bucket name:
BUCKET_NAME=<bucket_name> -
Delete all object versions from the bucket:
aws s3api delete-objects \ --endpoint-url https://storage.yandexcloud.net \ --bucket $BUCKET_NAME \ --delete \ "$(aws s3api list-object-versions \ --endpoint-url https://storage.yandexcloud.net \ --bucket $BUCKET_NAME \ --query '{Objects: Versions[].{Key: Key, VersionId: VersionId}}' \ --max-items 1000)"You can also use
--max-keysinstead of the--max-itemsparameter.Result:
{ "Deleted": [ { "Key": "object_000", "VersionId": "0005CDD3********" }, { "Key": "object_001", "VersionId": "0005CDD3********" }, ... ] }Using this command, you can delete a maximum of 1,000 object versions at a time due to the
aws s3api delete-objectslimitations. If your bucket contains more versions, run the command multiple times. -
For buckets with enabled versioning, remove all delete markers:
aws s3api delete-objects \ --endpoint-url https://storage.yandexcloud.net \ --bucket $BUCKET_NAME \ --delete \ "$(aws s3api list-object-versions \ --endpoint-url https://storage.yandexcloud.net \ --bucket $BUCKET_NAME \ --query '{Objects: DeleteMarkers[].{Key: Key, VersionId: VersionId}}' \ --max-items 1000)"You can also use
--max-keysinstead of the--max-itemsparameter.Result:
{ "Deleted": [ { "Key": "object_034", "DeleteMarker": true, "DeleteMarkerVersionId": "0005CDD3********" }, { "Key": "object_057", "DeleteMarker": true, "DeleteMarkerVersionId": "0005CDD3********" }, ... ] }Using this command, you can delete a maximum of 1,000 delete markers at a time due to the
aws s3api delete-objectslimitations. If your bucket contains more delete markers, run the command multiple times. -
Delete partially uploaded objects:
aws s3api list-multipart-uploads \ --endpoint-url https://storage.yandexcloud.net \ --bucket $BUCKET_NAME \ | jq -r '.Uploads[] | "--key \"\(.Key)\" --upload-id \(.UploadId)"' \ | while read -r line; do eval "aws s3api abort-multipart-upload \ --endpoint-url https://storage.yandexcloud.net \ --bucket $BUCKET_NAME \ $line"; done -
Get a list of object parts remaining in the bucket:
aws s3api list-multipart-uploads \ --endpoint-url https://storage.yandexcloud.net \ --bucket $BUCKET_NAME \ | jq -r '.Uploads[] | "--key \"\(.Key)\" --upload-id \(.UploadId)"' \ | while read -r line; do eval "aws s3api list-parts \ --endpoint-url https://storage.yandexcloud.net \ --bucket $BUCKET_NAME \ $line"; doneThe list may include parts of objects whose upload started before and completed after the preceding step. If the list is not empty, repeat steps 4 and 5.
Run the following code:
import boto3
bucket_name = '<bucket_name>'
s3 = boto3.resource('s3',
endpoint_url='https://storage.yandexcloud.net',
aws_access_key_id='<key_ID>',
aws_secret_access_key='<secret_key>')
bucket = s3.Bucket(bucket_name)
# Deleting all versions (works for non-versioned buckets too).
bucket.object_versions.delete()
# Aborting all multipart uploads, which also deletes all parts.
for multipart_upload in bucket.multipart_uploads.iterator():
# Part uploads that are currently in progress may or may not succeed,
# so it might be necessary to abort a multipart upload multiple times.
while len(list(multipart_upload.parts.all())) > 0:
multipart_upload.abort()
Use the deleteMultipleObjects S3 API method.
In the management console, the information about the number of objects and storage space used in the bucket is updated with a few minutes' delay.