Downloading an object

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

1. See the description of the CLI command for downloading an object from a bucket:

   yc storage s3api get-object --help
   

2. Get a list of buckets in the default folder:

   yc storage bucket list
   

   Result:

   +------------------+----------------------+-------------+-----------------------+---------------------+
   |       NAME       |      FOLDER ID       |  MAX SIZE   | DEFAULT STORAGE CLASS |     CREATED AT      |
   +------------------+----------------------+-------------+-----------------------+---------------------+
   | first-bucket     | b1gmit33ngp6******** | 53687091200 | STANDARD              | 2022-12-16 13:58:18 |
   +------------------+----------------------+-------------+-----------------------+---------------------+
   

3. Run this command:

   yc storage s3api get-object \
     --bucket <bucket_name> \
     --key <object_key> \
     <download_path>
   

   Where:

   • --bucket: Name of your bucket.
   • --key: Object key.
   • <download_path>: Local download path.

   Result:

   etag: '"d41d8cd98f00b204e9800998********"'
   request_id: af194b83********
   accept_ranges: bytes
   content_type: application/octet-stream
   last_modified_at: "2024-10-08T12:36:36Z"
   server_side_encryption: aws:kms
   sse_kms_key_id: abj497vtg3h0********
   

   Alternative command:

   yc storage s3 cp \
     s3://<bucket_name>/<object_key> \
     <download_path>
   

   Result:

   download: s3://my-bucket/object.txt to object.txt
   

   Configuring parameters for the yc storage s3 cp command

   You can set the following parameters for the yc storage s3 cp command:

   • s3.max-queue-size: Maximum number of issues per queue. The default value is 1,000.
   • s3.max-concurrent-requests: Maximum number of simultaneous requests. The default value is 10.
   • s3.multipart-threshold: Object threshold size to trigger a multipart upload if exceeded. The default value is 8MB.
   • s3.multipart-chunksize: Size of parts the object will be split into in a multipart upload. The default value is 8MB.

   To set these parameters, use the yc config set <parameter> <value> command.

If you do not have the AWS CLI yet, install and configure it.

Downloading a single object

aws s3 cp \
  --endpoint-url=https://storage.yandexcloud.net \
  s3://<bucket_name>/<object_key> \
  <local_path>

Where:

• --endpoint-url: Object Storage endpoint.
• <bucket_name>: Name of the bucket you want to download the object from.
• <object_key>: Key of the object you want to download.
• <local_path>: Path to the directory to save the downloaded object to, e.g., ~/downloads/.

Downloading a folder (all objects with a specified prefix)

You can learn more about folders in Object Storage Directory here.

aws s3 cp \
  --endpoint-url=https://storage.yandexcloud.net \
  --recursive \
  s3://<bucket_name>/<prefix>/ \
  <local_path>

Where:

• --endpoint-url: Object Storage endpoint.
• --recursive: Parameter for downloading all objects with the specified prefix.
• <bucket_name>: Name of the bucket you want to download the objects from.
• <prefix>: Prefix (folder) of the objects you want to download, e.g., test/folder.
• <local_path>: Path to the directory to save the downloaded objects to, e.g., ~/downloads/.

Downloading all objects from a bucket

aws s3 cp \
  --endpoint-url=https://storage.yandexcloud.net \
  --recursive \
  s3://<bucket_name> \
  <local_path>

Where:

• --endpoint-url: Object Storage endpoint.
• --recursive: Parameter for downloading all objects from the bucket to a local directory.
• <bucket_name>: Name of the bucket you want to download the objects from.
• <local_path>: Path to the directory to save the downloaded objects to, e.g., ~/downloads/.

aws s3 cp is a high-level command providing limited features. For more information, see the AWS CLI reference.

You can download bucket objects selectively using the aws s3api command and a JMESPath query template. To download objects using a query template, run this command:

• Bash:

  aws s3api list-objects \
      --endpoint-url https://storage.yandexcloud.net \
      --bucket <bucket_name> \
      --query '<query>' \
      --output text | xargs -I {} aws s3api get-object --endpoint-url https://storage.yandexcloud.net --bucket <bucket_name> --key {} <local_path>{}
  

  Where:

  • --endpoint-url: Object Storage endpoint.
  • --bucket: Name of the bucket you want to download the objects from.
  • --query: Query in JMESPath format.
  • <local_path>: Path to the directory to save the downloaded objects to, e.g., ~/downloads/.

  Here is an example of a command that downloads all objects whose filenames start with date-20231002 from the bucket named sample-bucket to the ~/downloads/ local directory:

  aws s3api list-objects \
    --endpoint-url https://storage.yandexcloud.net \
    --bucket sample-bucket \
    --query 'Contents[?starts_with(Key, `date-20231002`) == `true`].[Key]' \
    --output text | xargs -I {} aws s3api get-object --endpoint-url https://storage.yandexcloud.net --bucket sample-bucket --key {} ~/downloads/{}
  

• PowerShell:

  Foreach($x in (aws s3api list-objects `
    --endpoint-url https://storage.yandexcloud.net `
    --bucket <bucket_name> `
    --query '<query>' `
    --output text)) `
    {aws s3api get-object --endpoint-url https://storage.yandexcloud.net --bucket <bucket_name> --key $x <local_path>$x}
  

  Where:

  • --endpoint-url: Object Storage endpoint.
  • --bucket: Name of the bucket you want to download the objects from.
  • --query: Query in JMESPath format.
  • <local_path>: Path to the directory to save the downloaded objects to, e.g., d:\downloads\.

  Here is an example of a command that downloads all objects whose filenames start with date-20231002 from the bucket named sample-bucket to the d:\downloads\ local directory:

  Foreach($x in (aws s3api list-objects `
    --endpoint-url https://storage.yandexcloud.net `
    --bucket sample-bucket `
    --query 'Contents[?starts_with(Key, `date-20231002`) == `true`].[Key]' `
    --output text)) `
    {aws s3api get-object --endpoint-url https://storage.yandexcloud.net --bucket sample-bucket --key $x d:\downloads\$x}
  

To download an object, use the get S3 API method.