Authentication with the Object Storage API
You can use the following types of APIs to work with Object Storage:
AWS S3 API
To authenticate in the AWS S3 API and work with Terraform and other supported tools, use a static access key. A static access key is issued for a specific service account, and all actions involving this key are performed on behalf of this service account. For more information, see How do I use the S3 API?.
You can safely store an Object Storage static access key in Yandex Lockbox. For more information, see Using a Yandex Lockbox secret to store a static access key.
For a full list of S3 API methods, see S3 API reference.
Note
A service account is only allowed to view a list of buckets in the folder it was created in.
A service account can perform actions with objects in buckets that are created in folders different from the service account folder. To enable this, assign the service account roles for the appropriate folder or its bucket.
If you want to use the AWS S3 API directly (without an SDK or apps), you will need to sign requests yourself. You can test the request and signature generation process using the AWS CLI in debug mode.
AWS S3 API use case
Starting from version 8.3.0, the curl utility supports automatic generation of the signature string, request signing, and substitution of the required headers when working with the AWS S3 API.
You can also generate these headers and sign requests manually. For more information, see the example for curl 8.2.1 and lower.
Note
Make sure that the service account you are using to make the request has the permissions to perform the requested action. For example, to upload an object into a bucket, assign the storage.uploader role for the bucket to the service account. For more information, see Access management methods in Object Storage: Overview.
Below are examples of requests for uploading an object to a bucket.
AWS_KEY_ID="<static_key_ID>"
AWS_SECRET_KEY="<secret_key>"
LOCAL_FILE="<local_file_path>"
BUCKET_NAME="<bucket_name>"
OBJECT_PATH="<object_key>"
curl \
--request PUT \
--user "${AWS_KEY_ID}:${AWS_SECRET_KEY}" \
--aws-sigv4 "aws:amz:ru-central1:s3" \
--upload-file "${LOCAL_FILE}" \
--verbose \
"https://storage.yandexcloud.net/${BUCKET_NAME}/${OBJECT_PATH}"
Where:
AWS_KEY_ID: Static access key ID.AWS_SECRET_KEY: Secret key.LOCAL_FILE: Path to the local file you want to upload, e.g.,./sample.txt.BUCKET_NAME: Name of the bucket to upload the file to.OBJECT_PATH: Key to assign to the object in the bucket, e.g.,new-prefix/sample-object.txt.
In the same way, you can upload the file to the bucket without saving it locally. For example, archive the directory and send the archive to the bucket:
AWS_KEY_ID="<static_key_ID>"
AWS_SECRET_KEY="<secret_key>"
BUCKET_NAME="<bucket_name>"
OBJECT_PATH="<object_key>"
DIRECTORY_PATH="<path_to_directory>"
tar -cvzf - "${DIRECTORY_PATH}" | curl \
--request PUT \
--user "${AWS_KEY_ID}:${AWS_SECRET_KEY}" \
--aws-sigv4 "aws:amz:ru-central1:s3" \
--upload-file - \
--verbose \
"https://storage.yandexcloud.net/${BUCKET_NAME}/${OBJECT_PATH}"
Where DIRECTORY_PATH is the path to the directory you want to archive.
AWS_KEY_ID="<static_key_ID>"
AWS_SECRET_KEY="<secret_key>"
LOCAL_FILE="<local_file_path>"
BUCKET_NAME="<bucket_name>"
OBJECT_PATH="<object_key>"
CONTENT_TYPE="<object_MIME_type>"
DATE_VALUE=`date -R`
STRING_TO_SIGN="PUT\n\n${CONTENT_TYPE}\n${DATE_VALUE}\n/${BUCKET_NAME}/${OBJECT_PATH}"
SIGNATURE=`echo -en ${STRING_TO_SIGN} | openssl sha1 -hmac ${AWS_SECRET_KEY} -binary | base64`
curl \
--request PUT \
--upload-file "${LOCAL_FILE}" \
--verbose \
--header "Host: storage.yandexcloud.net" \
--header "Date: ${DATE_VALUE}" \
--header "Content-Type: ${CONTENT_TYPE}" \
--header "Authorization: AWS ${AWS_KEY_ID}:${SIGNATURE}" \
"https://storage.yandexcloud.net/${BUCKET_NAME}/${OBJECT_PATH}"
Where:
AWS_KEY_ID: Static access key ID.AWS_SECRET_KEY: Secret key.LOCAL_FILE: Path to the local file you want to upload, e.g.,./sample.txt.BUCKET_NAME: Name of the bucket to upload the file to.OBJECT_PATH: Key to assign to the object in the bucket, e.g.,new-prefix/sample-object.txt.CONTENT_TYPE: MIME type of the object being uploaded, e.g.,text/plain.
Yandex Cloud gRPC and REST APIs
For authentication in the Yandex Cloud gRPC and REST APIs, get an IAM token. Learn more about how to get an IAM token for different types of accounts:
Specify the received IAM token when accessing Yandex Cloud resources via the API. Provide the IAM token in the Authorization header in the following format:
Authorization: Bearer <IAM_token>
For a full list of Yandex Cloud API calls and methods, see gRPC API and REST API references.
Yandex Cloud API use case
In the example, a 50GB bucket is created with a standard storage class.
export IAM_TOKEN="<IAM_token>"
grpcurl \
-H "Authorization: Bearer $IAM_TOKEN" \
-d '{
"name": "<bucket_name>",
"folder_id": "<folder_ID>",
"default_storage_class": "STANDARD",
"max_size": "53687091200",
"anonymous_access_flags": [{
"read": false,
"list": false,
"configRead": false
}]
}' \
storage.api.cloud.yandex.net:443 \
yandex.cloud.storage.v1.BucketService/Create
Where:
IAM_TOKEN: IAM token. See Getting an IAM token for details.name: Bucket name.folder_id: Folder ID.default_storage_class: Storage class.max_size: Bucket size.anonymous_access_flags: Bucket access settings:read: Public read access to objects.list: Public access to the list of objects.configRead: Public read access to settings.
Result:
{
"id": "e3ehmmasama1********",
"description": "create bucket",
"createdAt": "2023-08-10T06:32:19.836842Z",
"createdBy": "ajego134p5h1********",
"modifiedAt": "2023-08-10T06:32:19.836842Z",
"done": true,
"metadata": {"@type":"type.googleapis.com/yandex.cloud.storage.v1.CreateBucketMetadata","name":"<bucket_name>"},
"response": {"@type":"type.googleapis.com/yandex.cloud.storage.v1.Bucket","acl":{},"anonymousAccessFlags":{"read":false,"list":false},"createdAt":"2023-08-10T06:32:17.557756Z","defaultStorageClass":"STANDARD","folderId":"b1gmit33ngp3********","maxSize":"53687091200","name":"<bucket_name>","versioning":"VERSIONING_DISABLED"}
}
export IAM_TOKEN="<IAM_token>"
curl \
--request POST \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $IAM_TOKEN" \
--data '{
"name": "<bucket_name>",
"folderId": "<folder_ID>",
"defaultStorageClass": "STANDARD",
"maxSize": "53687091200",
"anonymousAccessFlags": {
"read": false,
"list": false,
"configRead": false
}
}' \
https://storage.api.cloud.yandex.net/storage/v1/buckets
Where:
IAM_TOKEN: IAM token. See Getting an IAM token for details.name: Bucket name.folderId: Folder ID.default_storage_class: Storage class.maxSize: Bucket size.anonymousAccessFlags: Bucket access settings:read: Public read access to objects.list: Public access to the list of objects.configRead: Public read access to settings.
Result:
{
"done": true,
"metadata": {
"@type": "type.googleapis.com/yandex.cloud.storage.v1.CreateBucketMetadata",
"name": "<bucket_name>"
},
"response": {
"@type": "type.googleapis.com/yandex.cloud.storage.v1.Bucket",
"anonymousAccessFlags": {
"read": false,
"list": false
},
"acl": {},
"name": "<bucket_name>",
"folderId": "b1gmit33ngp3********",
"defaultStorageClass": "STANDARD",
"versioning": "VERSIONING_DISABLED",
"maxSize": "53687091200",
"createdAt": "2023-08-08T12:54:29.321021Z"
},
"id": "e3enrkcct2pt********",
"description": "create bucket",
"createdAt": "2023-08-08T12:54:32.111022Z",
"createdBy": "ajego134p5h1********",
"modifiedAt": "2023-08-08T12:54:32.111022Z"
}