How to migrate data from an Object Storage bucket to ice storage
Case description
You need to migrate data from a standard or cold storage in an Object Storage bucket to ice storage.
Solution
You cannot use lifecycle rules to change the storage class of objects in an Object Storage bucket to ICE. There are two workarounds to do that:
- Copy the objects.
- Set a lifecycle using the AWS CLI.
To copy objects to a different bucket using the AWS CLI, follow these steps:
- Change the storage class for your bucket to
ICE. - Install and configure the AWS CLI.
- Run the
aws --endpoint-url=https://storage.yandexcloud.net s3 cp --recursive s3://*\$OLD_BUCKET_NAME\ s3://*\$NEW_BUCKET_NAME\command.
Tip
In the provided example, replace $OLD_BUCKET_NAME with the source bucket name and $NEW_BUCKET_NAME, with the target bucket name.
In this case, moving data between storage classes occurs directly within Object Storage, without downloading objects to a local host running the AWS CLI. You can also specify the storage class for an object using the --storage-class option in the AWS CLI:
aws --endpoint-url=https://storage.yandexcloud.net s3 cp --recursive s3://*\$OLD_BUCKET_NAME\ s3://*\$NEW_BUCKET_NAME\ --storage-class GLACIER
Any of the above options will only change storage type for objects in the bucket. No duplicate objects will be created in the bucket as a result of this operation.
To set up a lifecycle using the AWS CLI, follow these steps:
-
Create a
transition-type lifecycle via the management console web UI. -
Save the current lifecycle rule configuration to the
lifecycles.jsonfile with the AWS CLI:aws s3api get-bucket-lifecycle-configuration \ --endpoint-url=https://storage.yandexcloud.net \ --bucket <$BUCKET_NAME> > lifecycles.jsonIn the provided example, replace
$BUCKET_NAMEwith the name of your Object Storage bucket. -
Edit
lifecycles.json, changing the storage class fromCOLDtoICE. -
Apply the new configuration using the AWS CLI:
aws s3api put-bucket-lifecycle-configuration \ --bucket <$BUCKET_NAME> \ --endpoint-url=https://storage.yandexcloud.net \ --lifecycle-configuration file://lifecycles.jsonIn the provided example, replace
$BUCKET_NAMEwith the name of your Object Storage bucket.