S3cmd
S3cmd
Getting started
-
Assign to the service account the roles required for your project, e.g., storage.editor for a bucket (to work with a particular bucket) or a folder (to work with all buckets in this folder). For more information about roles, see Access management with Yandex Identity and Access Management.
To work with objects in an encrypted bucket, a user or service account must have the following roles for the encryption key in addition to the
storage.configurer
role:kms.keys.encrypter
: To read the key, encrypt and upload objects.kms.keys.decrypter
: To read the key, decrypt and download objects.kms.keys.encrypterDecrypter
: This role includes thekms.keys.encrypter
andkms.keys.decrypter
permissions.
For more information, see Key Management Service service roles.
-
As a result, you will get the static access key data. To authenticate in Object Storage, you will need the following:
key_id
: Static access key IDsecret
: Secret key
Save
key_id
andsecret
: you will not be able to get the key value again.
Note
A service account is only allowed to view a list of buckets in the folder it was created in.
A service account can perform actions with objects in buckets that are created in folders different from the service account folder. To enable this, assign the service account roles for the appropriate folder or its bucket.
Installation
To install S3cmd, check out the installation section
Configuration
To configure S3cmd, use the s3cmd --configure
command. Specify values for the following parameters in the request:
-
Access Key
: Enter the ID of the key you received when generating the static key. -
Secret Key
: Enter the secret key you received when generating the static key. -
Default Region
: Enterru-central1
.To work with Object Storage, always specify the
ru-central1
. A different region value may lead to an authorization error. -
S3 Endpoint
: Enterstorage.yandexcloud.net
. -
DNS-style bucket+hostname:port template for accessing a bucket
: Enter%(bucket)s.storage.yandexcloud.net
. -
Leave the other parameters unchanged.
The program will try to establish a connection with Object Storage and get a list of buckets. If successful, it will display Success. Your access key and secret key worked fine :-)
.
The s3cmd --configure
command will save the settings in the ~/.s3cfg
file in the following format:
[default]
access_key = id
secret_key = secretKey
bucket_location = ru-central1
host_base = storage.yandexcloud.net
host_bucket = %(bucket)s.storage.yandexcloud.net
You can change these settings directly in the file, if required. You can also specify settings when launching the program by using the appropriate parameters.
For the static website hosting control commands to work correctly, manually add the following parameter to the configuration file:
website_endpoint = http://%(bucket)s.website.yandexcloud.net
Features
- S3cmd treats Object Storage as a hierarchical file system and object keys look like file paths.
-
S3cmd has no support for object versions. To work with object versions, use the AWS CLI.
- By default, S3cmd uploads objects to a standard storage. To specify the storage class when uploading an object, use the
--storage-class
key. - By default, when uploading an object, S3cmd can send the additional
X-Amz-Meta-S3cmd-Attrs
header with the attributes of your file (access permissions, file owners, timestamps). The header value is saved in the metadata of the object. You can disable the sending of attributes using thepreserve_attrs = False
parameter in the~/.s3cfg
configuration file or using the--no-preserve
key.
Examples of operations
Note
To enable debug output in the console, use the --debug
key.
Getting a list of buckets
Note
You can authorize in the Amazon S3 HTTP API and tools that support it using static keys obtained for the service account.
You can only view the list of buckets in the folder in which your service account was created.
s3cmd ls
Create a bucket
s3cmd mb s3://bucket
When creating a bucket, follow the naming conventions.
Uploading an object to a cold storage
s3cmd --storage-class COLD put local_file s3://bucket/object
Getting a list of objects
s3cmd ls s3://bucket
Retrieving an object
s3cmd get s3://bucket/object local_file
Deleting an object
s3cmd del s3://bucket/object
Uploading an object with MIME-types specified
To specify MIME-typesput
command with the following flags:
s3cmd put \
--no-guess-mime-type \
--no-mime-magic \
--mime-type="application/javascript" \
<local_file_path> \
s3://<bucket_name>/
Where --mime-type
is your object's MIME type.