boto3 and boto
boto3 and boto are software development kits (SDKs) for the Python 2.x and 3.x programming languages. The SDKs are designed for working with AWS services.
Getting started
-
Assign to the service account the roles required for your project, e.g., storage.editor for a bucket (to work with a particular bucket) or a folder (to work with all buckets in this folder). For more information about roles, see Access management with Yandex Identity and Access Management.
To work with objects in an encrypted bucket, a user or service account must have the following roles for the encryption key in addition to the
storage.configurerrole:kms.keys.encrypter: To read the key, encrypt and upload objects.kms.keys.decrypter: To read the key, decrypt and download objects.kms.keys.encrypterDecrypter: This role includes thekms.keys.encrypterandkms.keys.decrypterpermissions.
For more information, see Key Management Service service roles.
-
As a result, you will get the static access key data. To authenticate in Object Storage, you will need the following:
key_id: Static access key IDsecret: Secret key
Save
key_idandsecret: you will not be able to get the key value again.
To access the HTTP API directly, you need static key authentication, which is supported by the tools listed in Supported tools.
Note
You can disable using static keys for bucket access. Once disabled, access will be denied to all tools using this access option: the AWS CLI, SDK, and third-party applications. Access via ephemeral keys, temporary Security Token Service access keys, and pre-signed URLs will also be disabled. Only access with an IAM token or anonymous access (if enabled) will remain.
You can use Yandex Lockbox to safely store the static key for access to Object Storage. For more information, see Using a Yandex Lockbox secret to store a static access key.
Note
A service account is only allowed to view a list of buckets in the folder it was created in.
A service account can perform actions with objects in buckets that are created in folders different from the service account folder. To enable this, assign the service account roles for the appropriate folder or its bucket.
Installation
To install boto, follow the tutorial in the developer's repository: boto3, boto.
Configuration
-
Create a directory to store the authentication data in and navigate to it:
For macOS and Linux:
mkdir ~/.aws/For Windows:
mkdir C:\Users\<username>\.aws\ -
In the
.awsdirectory, create a file namedcredentialsand copy the credentials received earlier into it:[default] aws_access_key_id = <static_key_ID> aws_secret_access_key = <secret_key> -
Create a file named
configwith the default region settings and copy the following information to it:[default] region = ru-central1 endpoint_url = https://storage.yandexcloud.netNote
Some apps designed to work with Amazon S3 do not allow you to specify the region; this is why Object Storage may also accept the main AWS region value, which is the first row in the table of regions.
To access Object Storage, use the https://storage.yandexcloud.net endpoint.
Add environment variables to a function in Cloud Functions:
AWS_ACCESS_KEY_ID: Static key ID of the service account.AWS_SECRET_ACCESS_KEY: Secret key.AWS_DEFAULT_REGION: Region ID.
Use the storage.yandexcloud.net address to access Object Storage.
Example
boto3:
#!/usr/bin/env python
#-*- coding: utf-8 -*-
import boto3
session = boto3.session.Session()
s3 = session.client(
service_name='s3',
endpoint_url='https://storage.yandexcloud.net'
)
# Creating a new bucket
s3.create_bucket(Bucket='bucket-name')
# Uploading objects into a bucket
## From a string
s3.put_object(Bucket='bucket-name', Key='object_name', Body='TEST', StorageClass='COLD')
## From a file
s3.upload_file('this_script.py', 'bucket-name', 'py_script.py')
s3.upload_file('this_script.py', 'bucket-name', 'script/py_script.py')
# Getting a list of objects in a bucket
for key in s3.list_objects(Bucket='bucket-name')['Contents']:
print(key['Key'])
# Deleting multiple objects
forDeletion = [{'Key':'object_name'}, {'Key':'script/py_script.py'}]
response = s3.delete_objects(Bucket='bucket-name', Delete={'Objects': forDeletion})
# Getting an object
get_object_response = s3.get_object(Bucket='bucket-name',Key='py_script.py')
print(get_object_response['Body'].read())
Boto3 retrieves authentication credentials from the ~/.aws directory by default, but you can manually set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables.
...
session = boto3.session.Session()
s3 = session.client(
service_name='s3',
endpoint_url='https://storage.yandexcloud.net',
aws_access_key_id='<static_key_ID>',
aws_secret_access_key='<secret_key>'
)
Note
This method is not considered secure as it poses a risk of key leaks.
boto
#!/usr/bin/env python
#-*- coding: utf-8 -*-
import os
from boto.s3.key import Key
from boto.s3.connection import S3Connection
os.environ['S3_USE_SIGV4'] = 'True'
conn = S3Connection(
host='storage.yandexcloud.net'
)
conn.auth_region_name = 'ru-central1'
# Creating a new bucket
conn.create_bucket('bucket-name')
bucket = conn.get_bucket('bucket-name')
# Uploading objects into a bucket
## From a string
bucket.new_key('test-string').set_contents_from_string('TEST')
## From a file
file_key_1 = Key(bucket)
file_key_1.key = 'py_script.py'
file_key_1.set_contents_from_filename('this_script.py')
file_key_2 = Key(bucket)
file_key_2.key = 'script/py_script.py'
file_key_2.set_contents_from_filename('this_script.py')
# Getting a list of objects in a bucket
keys_list=bucket.list()
for key in keys_list:
print (key.key)
# Deleting multiple objects
response = bucket.delete_keys(['test-string', 'py_script.py'])
# Getting an object
key = bucket.get_key('script/py_script.py')
print (key.get_contents_as_string())
See the example in the video conversion guide.