Authentication with the Object Storage API
You can use the following types of APIs to work with Object Storage:
AWS S3 API
To authenticate with the AWS S3 API, you can use an IAM token or a static access key.
Warning
For AWS S3 API, IAM token authentication if the recommended method: it is more secure and, unlike static key authentication, is does not require creating a request signature.
An IAM token can be issued for either a user account or a service account, and any actions using the IAM token are performed on behalf of the account for which the token was issued. However, using a service account to manage buckets and objects is more secure.
If authenticating with the API via an IAM token, you do not have to additionally sign HTTP requests.
Amazon S3 tools, such as the AWS CLI and AWS SDK, support static access key authentication only and cannot be used at the same time with IAM token authentication.
To authenticate with the AWS S3 API and use Terraform and other supported tools, a static access key can be used. A static access key is issued for a specific service account, and all actions involving this key are performed on behalf of this service account. For more information, see How do I use the S3 API?.
You can use Yandex Lockbox to safely store the static key for access to Object Storage. For more information, see Using a Yandex Lockbox secret to store a static access key.
To use the AWS S3 API with authentication via a static access key directly (without an SDK or apps), you will need to sign requests yourself. You can test the request and signature generation using the AWS CLI in debug mode.
For the full list of S3 API methods, see the S3 API reference.
Note
A service account is only allowed to view a list of buckets in the folder it was created in.
A service account can perform actions with objects in buckets that are created in folders different from the service account folder. To enable this, assign the service account roles for the appropriate folder or its bucket.
AWS S3 API use case
Warning
Make sure the account you are using to make the request has the permissions to perform the requested action. For example, to upload an object to a bucket, assign the storage.uploader role for the bucket to the account. For more information, see Access management methods in Object Storage: Overview.
Below are examples of requests for uploading an object to a bucket:
IAM_TOKEN="<IAM_token_contents>"
BUCKET_NAME="<bucket_name>"
LOCAL_FILE="<local_file_path>"
OBJECT_PATH="<object_key>"
curl \
--request PUT \
--header "Authorization: Bearer ${IAM_TOKEN}" \
--upload-file "${LOCAL_FILE}" \
--verbose \
"https://storage.yandexcloud.net/${BUCKET_NAME}/${OBJECT_PATH}"
Where:
IAM_TOKEN: IAM token body.BUCKET_NAME: Name of the bucket to upload the file to.LOCAL_FILE: Path to the local file you want to upload to the bucket, e.g.,./sample.txt.OBJECT_PATH: Key to assign to the object in the bucket, e.g.,new-prefix/sample-object.txt.
In the same way, you can upload a file to the bucket without saving it locally. For example, archive the directory and send the archive to the bucket:
IAM_TOKEN="<IAM_token_contents>"
BUCKET_NAME="<bucket_name>"
OBJECT_PATH="<object_key>"
DIRECTORY_PATH="<path_to_directory>"
tar -cvzf - "${DIRECTORY_PATH}" | curl \
--request PUT \
--header "Authorization: Bearer ${IAM_TOKEN}" \
--upload-file - \
--verbose \
"https://storage.yandexcloud.net/${BUCKET_NAME}/${OBJECT_PATH}"
Where DIRECTORY_PATH is the path to the directory you want to archive.
Starting from version 8.3.0, the curl utility supports automatic generation of the signature string, request signing, and substitution of the required headers when working with the AWS S3 API.
You can also generate these headers and sign requests manually. For more information, see the example for curl 8.2.1 and lower.
curl 8.3.0 and higher
AWS_KEY_ID="<static_key_ID>"
AWS_SECRET_KEY="<secret_key>"
LOCAL_FILE="<local_file_path>"
BUCKET_NAME="<bucket_name>"
OBJECT_PATH="<object_key>"
curl \
--request PUT \
--user "${AWS_KEY_ID}:${AWS_SECRET_KEY}" \
--aws-sigv4 "aws:amz:ru-central1:s3" \
--upload-file "${LOCAL_FILE}" \
--verbose \
"https://storage.yandexcloud.net/${BUCKET_NAME}/${OBJECT_PATH}"
Where:
AWS_KEY_ID: Static access key ID.AWS_SECRET_KEY: Secret key.LOCAL_FILE: Path to the local file you want to upload, e.g.,./sample.txt.BUCKET_NAME: Name of the bucket to upload the file to.OBJECT_PATH: Key to assign to the object in the bucket, e.g.,new-prefix/sample-object.txt.
In the same way, you can upload a file to the bucket without saving it locally. For example, archive the directory and send the archive to the bucket:
AWS_KEY_ID="<static_key_ID>"
AWS_SECRET_KEY="<secret_key>"
BUCKET_NAME="<bucket_name>"
OBJECT_PATH="<object_key>"
DIRECTORY_PATH="<path_to_directory>"
tar -cvzf - "${DIRECTORY_PATH}" | curl \
--request PUT \
--user "${AWS_KEY_ID}:${AWS_SECRET_KEY}" \
--aws-sigv4 "aws:amz:ru-central1:s3" \
--upload-file - \
--verbose \
"https://storage.yandexcloud.net/${BUCKET_NAME}/${OBJECT_PATH}"
Where DIRECTORY_PATH is the path to the directory you want to archive.
curl 8.2.1 and lower
AWS_KEY_ID="<static_key_ID>"
AWS_SECRET_KEY="<secret_key>"
LOCAL_FILE="<local_file_path>"
BUCKET_NAME="<bucket_name>"
OBJECT_PATH="<object_key>"
CONTENT_TYPE="<object_MIME_type>"
DATE_VALUE=`date -R`
STRING_TO_SIGN="PUT\n\n${CONTENT_TYPE}\n${DATE_VALUE}\n/${BUCKET_NAME}/${OBJECT_PATH}"
SIGNATURE=`echo -en ${STRING_TO_SIGN} | openssl sha1 -hmac ${AWS_SECRET_KEY} -binary | base64`
curl \
--request PUT \
--upload-file "${LOCAL_FILE}" \
--verbose \
--header "Host: storage.yandexcloud.net" \
--header "Date: ${DATE_VALUE}" \
--header "Content-Type: ${CONTENT_TYPE}" \
--header "Authorization: AWS ${AWS_KEY_ID}:${SIGNATURE}" \
"https://storage.yandexcloud.net/${BUCKET_NAME}/${OBJECT_PATH}"
Where:
AWS_KEY_ID: Static access key ID.AWS_SECRET_KEY: Secret key.LOCAL_FILE: Path to the local file you want to upload, e.g.,./sample.txt.BUCKET_NAME: Name of the bucket to upload the file to.OBJECT_PATH: Key to assign to the object in the bucket, e.g.,new-prefix/sample-object.txt.CONTENT_TYPE: MIME type of the object being uploaded, e.g.,text/plain.
Yandex Cloud gRPC and REST APIs
For authentication in the Yandex Cloud gRPC and REST APIs, get an IAM token. Learn more about getting an IAM token for different account types:
Specify the received IAM token when accessing Yandex Cloud resources via the API. Provide the IAM token in the Authorization header in the following format:
Authorization: Bearer <IAM_token>
For the full list of Yandex Cloud API calls and methods, see the gRPC API and REST API references.
Yandex Cloud API use case
In this example, we will create a 50 GB bucket with a standard storage class.
export IAM_TOKEN="<IAM_token>"
grpcurl \
-H "Authorization: Bearer $IAM_TOKEN" \
-d '{
"name": "<bucket_name>",
"folder_id": "<folder_ID>",
"default_storage_class": "STANDARD",
"max_size": "53687091200",
"anonymous_access_flags": [{
"read": false,
"list": false,
"configRead": false
}]
}' \
storage.api.cloud.yandex.net:443 \
yandex.cloud.storage.v1.BucketService/Create
Where:
IAM_TOKEN: IAM token. See Getting an IAM token for details.name: Bucket name.folder_id: Folder ID.default_storage_class: Storage class.max_size: Bucket size.anonymous_access_flags: Bucket access settings:read: Public read access to objects.list: Public access to the list of objects.configRead: Public read access to settings.
Result:
{
"id": "e3ehmmasama1********",
"description": "create bucket",
"createdAt": "2023-08-10T06:32:19.836842Z",
"createdBy": "ajego134p5h1********",
"modifiedAt": "2023-08-10T06:32:19.836842Z",
"done": true,
"metadata": {"@type":"type.googleapis.com/yandex.cloud.storage.v1.CreateBucketMetadata","name":"<bucket_name>"},
"response": {"@type":"type.googleapis.com/yandex.cloud.storage.v1.Bucket","acl":{},"anonymousAccessFlags":{"read":false,"list":false},"createdAt":"2023-08-10T06:32:17.557756Z","defaultStorageClass":"STANDARD","folderId":"b1gmit33ngp3********","maxSize":"53687091200","name":"<bucket_name>","versioning":"VERSIONING_DISABLED"}
}
export IAM_TOKEN="<IAM_token>"
curl \
--request POST \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $IAM_TOKEN" \
--data '{
"name": "<bucket_name>",
"folderId": "<folder_ID>",
"defaultStorageClass": "STANDARD",
"maxSize": "53687091200",
"anonymousAccessFlags": {
"read": false,
"list": false,
"configRead": false
}
}' \
https://storage.api.cloud.yandex.net/storage/v1/buckets
Where:
IAM_TOKEN: IAM token. See Getting an IAM token for details.name: Bucket name.folderId: Folder ID.default_storage_class: Storage class.maxSize: Bucket size.anonymousAccessFlags: Bucket access settings:read: Public read access to objects.list: Public access to the list of objects.configRead: Public read access to settings.
Result:
{
"done": true,
"metadata": {
"@type": "type.googleapis.com/yandex.cloud.storage.v1.CreateBucketMetadata",
"name": "<bucket_name>"
},
"response": {
"@type": "type.googleapis.com/yandex.cloud.storage.v1.Bucket",
"anonymousAccessFlags": {
"read": false,
"list": false
},
"acl": {},
"name": "<bucket_name>",
"folderId": "b1gmit33ngp3********",
"defaultStorageClass": "STANDARD",
"versioning": "VERSIONING_DISABLED",
"maxSize": "53687091200",
"createdAt": "2023-08-08T12:54:29.321021Z"
},
"id": "e3enrkcct2pt********",
"description": "create bucket",
"createdAt": "2023-08-08T12:54:32.111022Z",
"createdBy": "ajego134p5h1********",
"modifiedAt": "2023-08-08T12:54:32.111022Z"
}