S3cmd

S3cmd is a console client (Linux, Mac) for services that support the Amazon S3 HTTP API. The general procedure for running commands can be found in the official s3cmd documentation.

Getting startedGetting started

1. Create a service account.

2. Assign to the service account the roles required for your project, e.g., storage.editor for a bucket (to work with a particular bucket) or a folder (to work with all buckets in this folder). For more information about roles, see Access management with Yandex Identity and Access Management.

   To work with objects in an encrypted bucket, a user or service account must have the following roles for the encryption key in addition to the storage.configurer role:

   • kms.keys.encrypter: To read the key, encrypt and upload objects.
   • kms.keys.decrypter: To read the key, decrypt and download objects.
   • kms.keys.encrypterDecrypter: This role includes the kms.keys.encrypter and kms.keys.decrypter permissions.

   For more information, see Key Management Service service roles.

3. Create a static access key.

   As a result, you will get the static access key data. To authenticate in Object Storage, you will need the following:

   • key_id: Static access key ID
   • secret: Secret key

   Save key_id and secret: you will not be able to get the key value again.

InstallationInstallation

To install S3cmd, check out the installation section in the S3cmd repository on GitHub.

SetupSetup

To configure S3cmd, use the s3cmd --configure command. For a query, specify values for the following parameters:

• Access Key: Key ID you got previously.

• Secret Key: Static key contents you got previously.

• Default Region: ru-central1.

  To work with Object Storage, always specify ru-central1 as the region. A different region value may lead to an authorization error.

• S3 Endpoint: storage.yandexcloud.net.

• DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.storage.yandexcloud.net.

• Leave the other settings unchanged.

The program will try to establish a connection with Object Storage and get a list of buckets. If successful, it will display Success. Your access key and secret key worked fine :-).

The s3cmd --configure command will save the settings in a ~/.s3cfg file in the following format:

[default]
access_key = id
secret_key = secretKey
bucket_location = ru-central1
host_base = storage.yandexcloud.net
host_bucket = %(bucket)s.storage.yandexcloud.net

If necessary, you can change these settings directly in the file. You can also specify settings when launching the client by using the appropriate parameters.

For static site hosting commands to work correctly, manually add the following parameter to the configuration file:

website_endpoint = http://%(bucket)s.website.yandexcloud.net

FeaturesFeatures

• S3cmd treats Object Storage as a hierarchical file system and object keys look like file paths.

• S3cmd has no support for object versions. To work with object versions, use the AWS CLI.

• By default, S3cmd uploads objects to standard storage. To specify the storage class when uploading an object, use the --storage-class key.
• By default, when uploading an object, S3cmd can send the additional X-Amz-Meta-S3cmd-Attrs header with the attributes of your file (access permissions, file owners, timestamps). The header value is saved in the metadata of the object. You can disable the sending of attributes using the preserve_attrs = False parameter in the ~/.s3cfg configuration file or using the --no-preserve key.

Examples of operationsExamples of operations

Getting a list of bucketsGetting a list of buckets

s3cmd ls

Create a bucketCreate a bucket

s3cmd mb s3://bucket

When creating a bucket, follow the naming conventions.

Uploading an object to cold storageUploading an object to cold storage

s3cmd --storage-class COLD put local_file s3://bucket/object

Getting a list of objectsGetting a list of objects

s3cmd ls s3://bucket

Getting an objectGetting an object

s3cmd get s3://bucket/object local_file

Deleting an objectDeleting an object

s3cmd del s3://bucket/object

Uploading an object with MIME-types specifiedUploading an object with MIME-types specified

To specify MIME-types when uploading an object, use the put command with the following flags:

s3cmd put \
  --no-guess-mime-type \
  --no-mime-magic \
  --mime-type="application/javascript" \
  <local_file_path> \
  s3://<bucket_name>/

Where --mime-type is your object's MIME type.