Copying objects
Object Storage supports server-side copy.
If bucket encryption is disabled, objects will be copied to server-side buckets. The host running the copy command and Object Storage only exchange object keys. No fee is charged for the copying traffic in this case because the traffic is internal for the object storage. However, you will be billed for copy requests.
If bucket objects are encrypted, they will first be copied to the host running the command and then uploaded to the target bucket.
Large objects uploaded using multipart uploads are stored in parts within the bucket. To copy these objects, you need to use the copyPart call for each part. Therefore, multipart objects cost more to copy than regular ones.
You can copy either the entire bucket contents or an individual bucket object.
To work with objects in an encrypted bucket, a user or service account must have the following roles for the encryption key in addition to the storage.configurer role:
kms.keys.encrypter: To read the key, encrypt and upload objects.kms.keys.decrypter: To read the key, decrypt and download objects.kms.keys.encrypterDecrypter: This role includes thekms.keys.encrypterandkms.keys.decrypterpermissions.
For more information, see Key Management Service service roles.
Copying a single object
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
-
See the description of the CLI command to copy an object:
yc storage s3api copy-object --help -
Get a list of buckets in the default folder:
yc storage bucket listResult:
+------------------+----------------------+-------------+-----------------------+---------------------+ | NAME | FOLDER ID | MAX SIZE | DEFAULT STORAGE CLASS | CREATED AT | +------------------+----------------------+-------------+-----------------------+---------------------+ | first-bucket | b1gmit33ngp6******** | 53687091200 | STANDARD | 2022-12-16 13:58:18 | +------------------+----------------------+-------------+-----------------------+---------------------+ -
Run this command:
yc storage s3api copy-object \ --copy-source <source_bucket>/<object_key> \ --bucket <target_bucket> \ --key <object_key>Where:
--copy-source: Name of the source bucket and object key you need to copy.--bucket: Name of the bucket to copy the object to.--key: Key to use for storing the object in the bucket.
Result:
copy_object_result: etag: '"d41d8cd98f00b204e9800998********"' last_modified_at: "2024-10-08T14:21:41.628Z" request_id: 61523025******** copy_source_version_id: "null"Alternative command:
yc storage s3 cp \ s3://<source_bucket>/<original_object_key> \ s3://<target_bucket>/<copy_object_key>Result:
copy: s3://my-bucket/object.txt to s3://new-bucket/object-copy.txtConfiguring parameters for the
yc storage s3 cpcommandYou can set the following parameters for the
yc storage s3 cpcommand:s3.max-queue-size: Maximum number of issues per queue. The default value is1,000.s3.max-concurrent-requests: Maximum number of simultaneous requests. The default value is10.s3.multipart-threshold: Object threshold size to trigger a multipart upload if exceeded. The default value is8MB.s3.multipart-chunksize: Size of parts the object will be split into in a multipart upload. The default value is8MB.
To set these parameters, use the
yc config set <parameter> <value>command.
-
If you do not have the AWS CLI yet, install and configure it.
-
Run this command:
aws --endpoint-url=https://storage.yandexcloud.net/ \ s3 cp s3://<source_bucket>/<object_key> s3://<target_bucket>/<object_key>Where:
--endpoint-url: Object Storage endpoint.s3 cp: Copy object command.
Result:
copy: s3://<source_bucket>/<object_key> to s3://<target_bucket>/<object_key>For more information about the
aws s3 cpcommand, see this article in the AWS CLI command reference.
Copying all bucket objects to a different bucket
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
-
See the description of the CLI command to copy objects:
yc storage s3 cp --help -
Get a list of buckets in the default folder:
yc storage bucket listResult:
+------------------+----------------------+-------------+-----------------------+---------------------+ | NAME | FOLDER ID | MAX SIZE | DEFAULT STORAGE CLASS | CREATED AT | +------------------+----------------------+-------------+-----------------------+---------------------+ | first-bucket | b1gmit33ngp6******** | 53687091200 | STANDARD | 2022-12-16 13:58:18 | +------------------+----------------------+-------------+-----------------------+---------------------+ -
Run this command:
yc storage s3 cp \ s3://<source_bucket>/ \ s3://<target_bucket>/ \ --recursiveWhere
--recursiveis the parameter for copying all objects.Result:
copy: s3://my-bucket/object-1.txt to s3://new-bucket/object-1-copy.txt copy: s3://my-bucket/object-2.txt to s3://new-bucket/object-2-copy.txt ... copy: s3://my-bucket/object-n.txt to s3://new-bucket/object-n-copy.txtConfiguring parameters for the
yc storage s3 cpcommandYou can set the following parameters for the
yc storage s3 cpcommand:s3.max-queue-size: Maximum number of issues per queue. The default value is1,000.s3.max-concurrent-requests: Maximum number of simultaneous requests. The default value is10.s3.multipart-threshold: Object threshold size to trigger a multipart upload if exceeded. The default value is8MB.s3.multipart-chunksize: Size of parts the object will be split into in a multipart upload. The default value is8MB.
To set these parameters, use the
yc config set <parameter> <value>command.
-
If you do not have the AWS CLI yet, install and configure it.
-
Run this command:
aws --endpoint-url=https://storage.yandexcloud.net/ \ s3 cp --recursive s3:<source_bucket>/ s3:<target_bucket>/Where:
--endpoint-url: Object Storage endpoint.s3 cp: Command to copy objects.--recursive: Parameter for copying all objects from the source bucket.
For more information about the
aws s3 cpcommand, see this article in the AWS CLI command reference.
All objects from the source bucket will now appear in the target bucket.
Note
You can copy objects between buckets, both within the same cloud and across different clouds. To do this, make sure your service account has write permissions for both buckets.