Contact UsGet started
© 2025 Direct Cursus Technology L.L.C.

Multipart upload

Written by
Updated at May 5, 2025

A network failure while uploading an object to an Object Storage bucket will terminate the upload, so you will have to re-upload your file. The larger the file size, the longer the upload, thus increasing the risk of network errors.

Therefore, a multipart upload is a better choice when uploading large files to the bucket. If there is a network failure when uploading a file part, you will only need to re-upload that one part, not the whole file.

To upload an object to the bucket in parts:

Split your file into parts

On your PC, split your original file into parts of preferred size, e.g., using the split command:

split -b 100M ./video.mp4 part.

Where:

  • 100M: Size of file parts. In this example, the size is 100 MB.

    The M suffix means the size is in MB. You can also use the K and G suffixes for KB and GB, respectively.

  • ./video.mp4: Path to the original file you need to split into parts to upload to the bucket.

  • part.: File name prefix for the parts.

Result:

% ls -l
total 822368
-rw-r--r--@ 1 User  group  104857600 Dec 20 22:56 part.aa
-rw-r--r--@ 1 User  group  104857600 Dec 20 22:56 part.ab
-rw-r--r--@ 1 User  group     808601 Dec 20 22:56 part.ab
-rwx------@ 1 User  group  210523801 Feb 24  2023 video.mp4

The split command has split video.mp4 into three parts: part.aa, part.ab, and part.ab.

Create a multipart upload in a bucket

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

  1. See the description of the CLI command for creating a multipart upload to upload an object into a bucket:

    yc storage s3api create-multipart-upload --help

  2. Get a list of buckets in the default folder:

    yc storage bucket list

    Result:

    +------------------+----------------------+-------------+-----------------------+---------------------+
|       NAME       |      FOLDER ID       |  MAX SIZE   | DEFAULT STORAGE CLASS |     CREATED AT      |
+------------------+----------------------+-------------+-----------------------+---------------------+
| first-bucket     | b1gmit33ngp6******** | 53687091200 | STANDARD              | 2022-12-16 13:58:18 |
+------------------+----------------------+-------------+-----------------------+---------------------+

  3. Create a multipart upload:

    yc storage s3api create-multipart-upload \
  --bucket <bucket_name> \
  --key <object_key>

    Where:

    • --bucket: Name of your bucket.
    • --key: Object key you will use to store the object in the bucket.

    Result:

    request_id: 644e7c89********
bucket: first-bucket
key: video.mp4
upload_id: 000629B7********

    Save the multipart upload ID (upload_id), as you will need it for the following steps.

    For more information about the yc storage s3api create-multipart-upload command, see the CLI reference.

If you do not have the AWS CLI yet, install and configure it.

  1. Get a list of available buckets:

    aws s3api list-buckets \
  --endpoint-url=https://storage.yandexcloud.net

    Result:

    {
  "Buckets": [
    {
      "Name": "new-bucket",
      "CreationDate": "2024-12-20T17:48:17+00:00"
    },
    {
      "Name": "first-bucket",
      "CreationDate": "2023-12-28T12:20:49.538000+00:00"
    }
  ],
  "Owner": {
    "DisplayName": "",
    "ID": ""
  }
}

  2. Create a multipart upload:

    aws s3api create-multipart-upload \
  --endpoint-url=https://storage.yandexcloud.net \
  --bucket <bucket_name> \
  --key <object_key>

    Where:

    • --bucket: Name of your bucket.
    • --key: Object key you will use to store the object in the bucket.

    Result:

    {
  "Bucket": "first-bucket",
  "Key": "video.mp4",
  "UploadId": "000629C2********"
}

    Save the multipart upload ID (UploadId), as you will need it for the following steps.

Use the startUpload S3 API method to create a multipart upload in your bucket.

Upload parts of the object to the bucket

Upload the first part of the object:

  1. See the description of the CLI command for uploading a part of the object to a bucket:

    yc storage s3api upload-part --help

  2. Run this command:

    yc storage s3api upload-part \
  --bucket <bucket_name> \
  --key <object_key> \
  --upload-id <multipart_upload_ID> \
  --part-number <part_number> \
  --body <path_to_part_file>

    Where:

    • --bucket: Name of your bucket.

    • --key: Object key you will use to store the object in the bucket.

    • --upload-id: Multipart upload ID obtained in the previous step.

    • --part-number: Sequential number of the object part uploaded to the bucket.

      The number uniquely identifies the part and determines its position in the general sequence. The number is an integer from 1 to 10,000 inclusive.

      If multiple parts with the same number are uploaded, Object Storage saves the last one received.

      Each part, except the last one, should be at least 5 MB in size.

    • --body: Path to the file with the part on your PC.

    Result:

    request_id: 7ab2820f********
etag: '"a2c44573954874e1a62a3f59********"'

    Save the etag value: you will need it to assemble the object from parts on the Object Storage bucket side.

    For more information about the yc storage s3api upload-part command, see the CLI reference.

aws s3api upload-part \
  --endpoint-url=https://storage.yandexcloud.net \
  --bucket <bucket_name> \
  --key <object_key> \
  --upload-id <multipart_upload_ID> \
  --part-number <part_number> \
  --body <path_to_part_file>

Where:

  • --bucket: Name of your bucket.

  • --key: Object key you will use to store the object in the bucket.

  • --upload-id: Multipart upload ID obtained in the previous step.

  • --part-number: Sequential number of the object part uploaded to the bucket.

    The number uniquely identifies the part and determines its position in the general sequence. The number is an integer from 1 to 10,000 inclusive.

    If multiple parts with the same number are uploaded, Object Storage saves the last one received.

    Each part, except the last one, should be at least 5 MB in size.

  • --body: Path to the file with the part on your PC.

Result:

{
    "ETag": "\"a2c44573954874e1a62a3f59********\""
}

Save the ETag value: you will need it to assemble the object from parts on the Object Storage bucket side.

To upload a part of an object to a bucket, use the uploadPart S3 API method.

Similarly, upload the other object parts by successively replacing --part-number and the --body path to the part's file in your commands. Do not forget to save the unique etag value you will get for each part you upload.

Complete your multipart upload

Once all the parts are successfully uploaded, complete the multipart upload and assemble the parts into a whole object:

  1. See the description of the CLI command for completing a multipart upload:

    yc storage s3api complete-multipart-upload --help

  2. Run this command:

    yc storage s3api complete-multipart-upload \
  --bucket <bucket_name> \
  --key <object_key> \
  --upload-id <multipart_upload_ID> \
  --multipart-upload \
    '{"Parts": [{"PartNumber": <part_1_number>, "ETag": "<part_1_etag_value>"}, {"PartNumber": <part_2_number>, "ETag": "<part_2_etag_value>"}, ..., {"PartNumber": <part_n_number>, "ETag": "<part_n_etag_value>"}]}'

    Where:

    • --bucket: Name of your bucket.

    • --key: Object key you will use to store the object in the bucket.

    • --upload-id: Multipart upload ID obtained in the previous step.

    • --multipart-upload: Object containing the sequential numbers and etag values of all uploaded parts in proper order. Here is an example:

      --multipart-upload \
  '{"Parts": [{"PartNumber": 1, "ETag": "a2c44573954874e1a62a3f59********"}, {"PartNumber": 2, "ETag": "3c0e70e1d05bd4e2500f5a90********"}, {"PartNumber": 3, "ETag": "953ecfff0150fd0970008320********"}]}'

    Result:

    request_id: bde94875********
bucket: first-bucket
key: video.mp4
etag: '"040d1fe80bd1d4f12728b192********-3"'
location: https://storage.yandexcloud.net:443/first-bucket/video.mp4

    For more information about the yc storage s3api complete-multipart-upload command, see the CLI reference.

aws s3api complete-multipart-upload \
  --endpoint-url=https://storage.yandexcloud.net \
  --bucket <bucket_name> \
  --key <object_key> \
  --upload-id <multipart_upload_ID> \
  --multipart-upload \
    '{"Parts": [{"PartNumber": <part_1_number>, "ETag": "<part_1_etag_value>"}, {"PartNumber": <part_2_number>, "ETag": "<part_2_etag_value>"}, ..., {"PartNumber": <part_n_number>, "ETag": "<part_n_etag_value>"}]}'

Where:

  • --bucket: Name of your bucket.

  • --key: Object key you will use to store the object in the bucket.

  • --upload-id: Multipart upload ID obtained in the previous step.

  • --multipart-upload: Object containing the sequential numbers and etag values of all uploaded parts in proper order. Here is an example:

    --multipart-upload \
  '{"Parts": [{"PartNumber": 1, "ETag": "a2c44573954874e1a62a3f59********"}, {"PartNumber": 2, "ETag": "3c0e70e1d05bd4e2500f5a90********"}, {"PartNumber": 3, "ETag": "953ecfff0150fd0970008320********"}]}'

Result:

{
  "Location": "https://storage.yandexcloud.net/first-bucket/video.mp4",
  "Bucket": "first-bucket",
  "Key": "video.mp4",
  "ETag": "\"040d1fe80bd1d4f12728b192********-3\""
}

Use the completeUpload S3 API method to complete your multipart upload.

An object with the prefix you specified has thus been created from uploaded parts in an Object Storage bucket. View this guide to get the information about the new object.

Previous
Uploading an object
Next
Get a list of bucket objects