Uploading an object
You can create folders inside buckets and upload objects there. Keep in mind that in the SDK and HTTP API, an object key is the entire path to the object from the bucket root. For more information, see Objects.
Note
You can upload objects of up to 5 GB via the management console (see Quotas and limits in Object Storage). When uploading via the console, you cannot set content-type
or other headers. To upload large objects or specify object headers, use other tools.
You can use tools that support Object Storage and signed URLs to upload objects into the bucket.
Regular uploads
To work with objects in an encrypted bucket, a user or service account must have the following roles for the encryption key in addition to the storage.configurer
role:
kms.keys.encrypter
: To read the key, encrypt, and upload objects.kms.keys.decrypter
: To read the key, decrypt, and download objects.kms.keys.encrypterDecrypter
: Includes thekms.keys.encrypter
andkms.keys.decrypter
permissions.
For more information, see Key Management Service service roles.
- In the management console
, select Object Storage from the list of services and go to the bucket you want to upload your object to. - In the left-hand panel, select
Objects. - If you want to upload an object to the bucket for the first time, click Upload objects.
- If you want to upload the object to a particular folder, go to that folder by clicking on its name. If you need to create a new folder, click Create folder.
- Within the folder you need, click
Upload on the top panel. - In the window that opens, select the required files and click Open.
- The management console displays all the objects selected for uploading and prompts you to select a storage class. The default storage class is defined in the bucket settings.
- Click Upload.
- Refresh the page.
In the management console, the information about the number of objects in the bucket and used up space is updated with a few minutes delay.
-
If you do not have the AWS CLI yet, install and configure it.
-
To upload a single object, run the command:
aws --endpoint-url=https://storage.yandexcloud.net/ \ s3 cp <local_file_path> s3://<bucket_name>/<object_key>
Where:
--endpoint-url
: Object Storage endpoint.s3 cp
: Command to upload an object. To upload an object, in the first part of the command, specify the path to the local file to upload. In the second part, provide the name of your bucket and key you will use to store the object in the bucket.
To load all objects from the local directory, use the following command:
aws --endpoint-url=https://storage.yandexcloud.net/ \ s3 cp --recursive <path_to_local_directory>/ s3://<bucket_name>/<prefix>/
Where:
--endpoint-url
: Object Storage endpoint.s3 cp --recursive
: Command to upload all objects stored in a local directory, including the nested ones. To upload objects, specify the path to the folder to copy the files from in the first part of the command and the name of the bucket to copy the files to and the ID of the folder in storage in the second part.
aws s3 cp
is a high-level command with limited functionality. For more information, see the AWS CLI reference
Note
Terraform uses a service account to interact with Object Storage. Assign to the service account the required role, e.g., storage.admin
, for the folder where you are going to create resources.
Terraform
For more information about the provider resources, see the documentation on the Terraform
If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.
If you don't have Terraform, install it and configure the Yandex Cloud provider.
Before you start, retrieve the static access keys: a secret key and a key ID used for authentication in Object Storage.
To create an object in an existing bucket:
-
In the configuration file, describe the parameters of resources that you want to create:
resource "yandex_iam_service_account" "sa" { name = "<service_account_name>" } // Assigning a role to a service account resource "yandex_resourcemanager_folder_iam_member" "sa-admin" { folder_id = "<folder_ID>" role = "storage.admin" member = "serviceAccount:${yandex_iam_service_account.sa.id}" } // Creating a static access key resource "yandex_iam_service_account_static_access_key" "sa-static-key" { service_account_id = yandex_iam_service_account.sa.id description = "static access key for object storage" } resource "yandex_storage_object" "test-object" { access_key = yandex_iam_service_account_static_access_key.sa-static-key.access_key secret_key = yandex_iam_service_account_static_access_key.sa-static-key.secret_key bucket = "<bucket_name>" key = "<object_name>" source = "<path_to_file>" }
Where:
-
access_key
: Static access key ID. -
secret_key
: Secret access key value. -
bucket
: Name of the bucket where to add the object. This is a required parameter. -
key
: Name of the object in the bucket. This is a required parameter. The name format is as follows:- The name must be from 3 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- The first character must be a letter and the last character cannot be a hyphen.
-
source
: Relative or absolute path to the file you need to upload to the bucket.
For more information about the resources you can create with Terraform, see the provider documentation
. -
-
Make sure the configuration files are correct.
-
In the command line, go to the folder where you created the configuration file.
-
Run a check using this command:
terraform plan
If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.
-
-
Deploy cloud resources.
-
If the configuration does not contain any errors, run this command:
terraform apply
-
Confirm creating the resources: type
yes
in the terminal and press Enter.All the resources you need will then be created in the specified folder. You can check the new resources and their configuration using the management console
.
-
To upload an object, use the upload S3 API method.
Uploading an object version with an object lock
If a bucket has versioning and object lock enabled, you can specify object lock settings (disable deleting or overwriting) when uploading an object version.
- In the management console
, select Object Storage from the list of services and go to the bucket you want to upload your object to. - In the left-hand panel, select
Objects. - If you want to upload an object to the bucket for the first time, click Upload objects.
- If you want to upload the object to a particular folder, go to that folder by clicking on its name. If you want to create a new folder, click Create folder on the top panel.
- Within the folder you need, click
Upload on the top panel. - In the window that opens, select the required files and click Open.
- The management console displays all the objects selected for uploading and prompts you to select a storage class. The default storage class is defined in the bucket settings.
- To configure locks for uploaded objects, select the lock type from the Object version lock drop-down list:
- Legal hold: Indefinitely prohibits deleting or overwriting the object version, while you still can upload new versions of the object. A user with the
storage.uploader
role can set and remove the lock. This lock cannot be bypassed. Combined with a temporary lock, the indefinite one has priority. - Retention: Prohibits deleting or overwriting the object version for a specified period of time, while you still can upload new versions of the object. A user with the
storage.uploader
role can set the lock. Combined with an indefinite lock, the temporary one has no priority.
- Legal hold: Indefinitely prohibits deleting or overwriting the object version, while you still can upload new versions of the object. A user with the
- If you selected Retention, specify Default lock type:
- Governance: A user with the
storage.admin
role can bypass the lock, change its expiration date, or remove it. - Compliance: User with the
storage.admin
role can only extend the retention period. Such locks cannot be bypassed, shortened, or removed until they expire.
- Governance: A user with the
- Specify Default lock period in days or years. It starts from the moment the object version is uploaded to the bucket.
- Click Upload and reload the page.
In the management console, the information about the number of objects in the bucket and used up space is updated with a few minutes delay.
-
If you do not have the AWS CLI yet, install and configure it.
-
Run this command:
aws --endpoint-url=https://storage.yandexcloud.net/ \ s3api put-object \ --body <local_file_path> \ --bucket <bucket_name> \ --key <object_key> \ --object-lock-mode <temporary_lock_type> \ --object-lock-retain-until-date <temporary_lock_period_end_date_and_time> \ --object-lock-legal-hold-status <indefinite_lock_status>
Where:
--endpoint-url
: Object Storage endpoint.s3api put-object
: Command to upload an object version. To upload object versions with an object lock, specify the following parameters:-
--body
: Path to the file you need to upload to the bucket. -
--bucket
: Name of your bucket. -
--key
: Key to use for storing the object in the bucket. -
--object-lock-mode
: Type of object lock with retention:GOVERNANCE
: Temporary managed lock.COMPLIANCE
: Temporary strict lock.
-
--object-lock-retain-until-date
: Retention end date and time in any format described in the HTTP standard , e.g.,Mon, 12 Dec 2022 09:00:00 GMT
. You can only specify it together with the--object-lock-mode
parameter. -
--object-lock-legal-hold-status
: Legal hold status:ON
: Enabled.OFF
: Disabled.
-
For an object version, you can use only object lock with retention (
object-lock-mode
andobject-lock-retain-until-date
parameters), only legal hold (object-lock-legal-hold-status
), or both at the same time. For more information about their combined use, see Object lock types.
To upload an object version with a lock, use the upload S3 API method with the X-Amz-Object-Lock-Mode
and X-Amz-Object-Lock-Retain-Until-Date
headers to apply a retention period and X-Amz-Object-Lock-Legal-Hold
to put a legal hold.
If object locks with retention are configured for a bucket by default, when uploading objects to that bucket, you should specify their MD5 hashes
-
Calculate a file’s MD5 hash and encode it with Base64
:md5=($(md5sum <local_file_path>)) md5_base64=$(echo $md5 | base64)
-
If you do not have the AWS CLI yet, install and configure it.
-
Upload an object to the bucket:
aws --endpoint-url=https://storage.yandexcloud.net/ \ s3api put-object \ --body <local_file_path> \ --bucket <bucket_name> \ --key <object_key> \ --content-md5 $md5_base64
Where:
--endpoint-url
: Object Storage endpoint.s3api put-object
: Command to upload an object version. To upload object versions, specify the following parameters:--body
: Path to the file you need to upload to the bucket.--bucket
: Name of your bucket.--key
: Key to use for storing the object in the bucket.--content-md5
: Object's encoded MD5 hash.
You can also add the following parameters to the command:
--object-lock-mode
and--object-lock-retain-until-date
to place an object version under an object lock with retention that is different from the bucket's default settings.--object-lock-legal-hold-status
to place a legal hold on an object version.
For more information about these parameters, see the guide above.
To upload an object version with a default retention period, use the upload S3 API method with the Content-MD5
header.