Uploading an object
You can create folders inside buckets and upload objects there. Keep in mind that in the SDK and HTTP API, an object key is the entire path to the object from the bucket root. For more information, see Objects.
Note
You cannot upload objects larger than 5 GB via the management console (see Quotas and limits in Object Storage). When uploading via the console, you also cannot set content-type
or other headers. To upload large objects or specify object headers, use other tools.
Regular uploads
To work with objects in an encrypted bucket, a user or service account must have the following roles for the encryption key in addition to the storage.configurer
role:
kms.keys.encrypter
: To read the key, encrypt, and upload objects.kms.keys.decrypter
: To read the key, decrypt, and download objects.kms.keys.encrypterDecrypter
: Includes thekms.keys.encrypter
andkms.keys.decrypter
permissions.
For more information, see Key Management Service service roles.
In the management console, you can work with Object Storage buckets like a hierarchical file system.
To upload an object:
- In the management console
, select the folder to upload an object to. - Select Object Storage.
- Click the name of the bucket you need.
- If you want to upload the object to a particular folder, go to that folder by clicking on its name. If you want to create a new folder, click Create folder.
- Once you navigate to the appropriate folder, click Upload.
- In the window that opens, select the required files and click Open.
- The management console displays all the objects selected for uploading and prompts you to select a storage class. The default storage class is defined in the bucket settings.
- Click Upload.
- Refresh the page.
In the management console, information about the number of objects in a bucket and the used space is updated with a few minutes' delay.
-
If you do not have the AWS CLI yet, install and configure it.
-
To upload a single object, run the command:
aws --endpoint-url=https://storage.yandexcloud.net/ \ s3 cp <path_to_local_file>/ s3://<bucket_name>/<object_key>
Where:
--endpoint-url
: Object Storage endpoint.s3 cp
: Command to upload an object. To upload an object, specify the path to the local file to upload in the first part of the command and the name of the bucket and the key to associate with the object in the bucket in the second part.
To load all objects from the local directory, use the following command:
aws --endpoint-url=https://storage.yandexcloud.net/ \ s3 cp --recursive <path_to_local_directory>/ s3://<bucket_name>/<prefix>/
Where:
--endpoint-url
: Object Storage endpoint.s3 cp --recursive
: Command to upload all objects contained in a local directory, including nested ones. To upload objects, specify the path to the folder to copy the files from in the first part of the command and the name of the bucket to copy the files to and the ID of the folder in storage in the second part.
The aws s3 cp
command is high-level, its functionality is limited. For more information, see the AWS CLI reference
Note
Terraform uses a service account to interact with Object Storage. Assign to the service account the required role, e.g., storage.admin
, for the folder where you are going to create resources.
Terraform
For more information about the provider resources, see the documentation on the Terraform
If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.
If you don't have Terraform, install it and configure the Yandex Cloud provider.
Before you start, retrieve the static access keys: a secret key and a key ID used for authentication in Object Storage.
To create an object in an existing bucket:
-
In the configuration file, describe the parameters of resources that you want to create:
resource "yandex_iam_service_account" "sa" { name = "<service_account_name>" } // Assigning a role to a service account resource "yandex_resourcemanager_folder_iam_member" "sa-admin" { folder_id = "<folder_ID>" role = "storage.admin" member = "serviceAccount:${yandex_iam_service_account.sa.id}" } // Creating a static access key resource "yandex_iam_service_account_static_access_key" "sa-static-key" { service_account_id = yandex_iam_service_account.sa.id description = "static access key for object storage" } resource "yandex_storage_object" "test-object" { access_key = yandex_iam_service_account_static_access_key.sa-static-key.access_key secret_key = yandex_iam_service_account_static_access_key.sa-static-key.secret_key bucket = "<bucket_name>" key = "<object_name>" source = "<path_to_file>" }
Where:
-
access_key
: ID of the static access key. -
secret_key
: Value of the secret access key. -
bucket
: Name of the bucket to add the object to. This is a required parameter. -
key
: Name of the object in the bucket. This is a required parameter. The name format is as follows:- The name must be from 3 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- The first character must be a letter and the last character cannot be a hyphen.
-
source
: Relative or absolute path to the files you need to upload to the bucket.
For more information about resources you can create with Terraform, see the provider documentation
. -
-
Make sure the configuration files are correct.
-
In the command line, go to the folder where you created the configuration file.
-
Run a check using this command:
terraform plan
If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.
-
-
Deploy cloud resources.
-
If the configuration does not contain any errors, run this command:
terraform apply
-
Confirm creating the resources: type
yes
in the terminal and press Enter.All the resources you need will then be created in the specified folder. You can check the new resources and their configuration using the management console
.
-
To upload an object, use the upload S3 API method.
Uploading an object version with an object lock
If a bucket has versioning and object lock enabled, you can specify object lock settings (disable deleting or overwriting) when uploading an object version.
-
If you do not have the AWS CLI yet, install and configure it.
-
Run this command:
aws --endpoint-url=https://storage.yandexcloud.net/ \ s3api put-object \ --body <path_to_local_file> \ --bucket <bucket_name> \ --key <object_key> \ --object-lock-mode <type_of_object_lock_with_retention_period> \ --object-lock-retain-until-date <object_lock_retain_until_date_and_time> \ --object-lock-legal-hold-status <status_of_legal_hold>
Where:
--endpoint-url
: Object Storage endpoint.s3api put-object
: Command to upload an object version. To upload object versions with an object lock, specify the following parameters:-
--body
: Path to the file to be uploaded to the bucket. -
--bucket
: Name of your bucket. -
--key
: Key to store the object in the bucket with. -
--object-lock-mode
: Type of object lock set for a certain period:GOVERNANCE
: Object lock with a predefined retention period that can be managed.COMPLIANCE
: Object lock with a predefined retention period with strict compliance.
-
--object-lock-retain-until-date
Date and time until which an object is to be locked, specified in any format described in the HTTP standard , e.g.,Mon, 12 Dec 2022 09:00:00 GMT
. Can only be set together with the--object-lock-mode
parameter. -
--object-lock-legal-hold-status
: Legal hold status:ON
: Enabled.OFF
: Disabled.
-
You can place an object version only under an object lock with a retention period (the
object-lock-mode
andobject-lock-retain-until-date
parameters), only under a legal hold (object-lock-legal-hold-status
), or under both. For more information about their combined use, see Object lock types.
To upload an object version with a lock, use the upload S3 API method with the X-Amz-Object-Lock-Mode
and X-Amz-Object-Lock-Retain-Until-Date
headers to apply a retention period and X-Amz-Object-Lock-Legal-Hold
to put a legal hold.
If a bucket already has the default object locks set for a certain period configured, you should upload any objects to it with their MD5 hash
-
Calculate a file's MD5 hash and encode it with Base64
:md5=($(md5sum <path_to_local_file>)) md5_base64=$(echo $md5 | base64)
-
If you do not have the AWS CLI yet, install and configure it.
-
Upload an object to the bucket:
aws --endpoint-url=https://storage.yandexcloud.net/ \ s3api put-object \ --body <path_to_local_file> \ --bucket <bucket_name> \ --key <object_key> \ --content-md5 $md5_base64
Where:
--endpoint-url
: Object Storage endpoint.s3api put-object
: Command to upload an object version. To upload object versions, specify the following parameters:--body
: Path to the file to be uploaded to the bucket.--bucket
: Name of your bucket.--key
: Key to store the object in the bucket with.--content-md5
: Object's encoded MD5 hash.
You can also add the following parameters to the command:
--object-lock-mode
and--object-lock-mode
to place an object version under an object lock for a certain period with a configuration different from the bucket's object lock default settings.--object-lock-legal-hold-status
to place an object version under a legal hold.
For more information about these parameters, see the guide above.
To upload an object version with a default retention period, use the upload S3 API method with the Content-MD5
header.