boto3 and boto
boto3
Getting started
-
Assign to the service account the roles required for your project, e.g., storage.editor for a bucket (to work with a particular bucket) or a folder (to work with all buckets in this folder). For more information about roles, see Access management with Yandex Identity and Access Management.
To work with objects in an encrypted bucket, a user or service account must have the following roles for the encryption key in addition to the
storage.configurer
role:kms.keys.encrypter
: To read the key, encrypt and upload objects.kms.keys.decrypter
: To read the key, decrypt and download objects.kms.keys.encrypterDecrypter
: This role includes thekms.keys.encrypter
andkms.keys.decrypter
permissions.
For more information, see Key Management Service service roles.
-
As a result, you will get the static access key data. To authenticate in Object Storage, you will need the following:
key_id
: Static access key IDsecret
: Secret key
Save
key_id
andsecret
: you will not be able to get the key value again.
Note
A service account is only allowed to view a list of buckets in the folder it was created in.
A service account can perform actions with objects in buckets that are created in folders different from the service account folder. To enable this, assign the service account roles for the appropriate folder or its bucket.
Installation
Note
Object Storage supports boto3 1.35.99 or lower.
To install boto3 version 1.35.99, run the following command in the terminal:
pip3 install boto3==1.35.99
For more information on how to install boto
, see the developer's repository: boto3
Configuration
-
Create a directory to store the authentication data in and navigate to it:
For macOS and Linux:
mkdir ~/.aws/
For Windows:
mkdir C:\Users\<username>\.aws\
-
In the
.aws
directory, create a file namedcredentials
with credentials for Object Storage and copy the following data into it:[default] aws_access_key_id = <static_key_ID> aws_secret_access_key = <secret_key>
-
Create a file named
config
with the default region settings and copy the following information to it:[default] region = ru-central1 endpoint_url = https://storage.yandexcloud.net
Note
Some apps designed to work with Amazon S3 do not allow you to specify the region; this is why Object Storage may also accept the
us-east-1
value.
To access Object Storage, use the https://storage.yandexcloud.net
endpoint.
Add environment variables to a function in Cloud Functions:
AWS_ACCESS_KEY_ID
: Static key ID of the service account.AWS_SECRET_ACCESS_KEY
: Secret key.AWS_DEFAULT_REGION
: Region ID.
Use the storage.yandexcloud.net
address to access Object Storage.
Example
boto3:
#!/usr/bin/env python
#-*- coding: utf-8 -*-
import boto3
session = boto3.session.Session()
s3 = session.client(
service_name='s3',
endpoint_url='https://storage.yandexcloud.net'
)
# Creating a new bucket
s3.create_bucket(Bucket='bucket-name')
# Uploading objects into a bucket
## From a string
s3.put_object(Bucket='bucket-name', Key='object_name', Body='TEST', StorageClass='COLD')
## From a file
s3.upload_file('this_script.py', 'bucket-name', 'py_script.py')
s3.upload_file('this_script.py', 'bucket-name', 'script/py_script.py')
# Getting a list of objects in a bucket
for key in s3.list_objects(Bucket='bucket-name')['Contents']:
print(key['Key'])
# Deleting multiple objects
forDeletion = [{'Key':'object_name'}, {'Key':'script/py_script.py'}]
response = s3.delete_objects(Bucket='bucket-name', Delete={'Objects': forDeletion})
# Getting an object
get_object_response = s3.get_object(Bucket='bucket-name',Key='py_script.py')
print(get_object_response['Body'].read())
boto
#!/usr/bin/env python
#-*- coding: utf-8 -*-
import os
from boto.s3.key import Key
from boto.s3.connection import S3Connection
os.environ['S3_USE_SIGV4'] = 'True'
conn = S3Connection(
host='storage.yandexcloud.net'
)
conn.auth_region_name = 'ru-central1'
# Creating a new bucket
conn.create_bucket('bucket-name')
bucket = conn.get_bucket('bucket-name')
# Uploading objects into a bucket
## From a string
bucket.new_key('test-string').set_contents_from_string('TEST')
## From a file
file_key_1 = Key(bucket)
file_key_1.key = 'py_script.py'
file_key_1.set_contents_from_filename('this_script.py')
file_key_2 = Key(bucket)
file_key_2.key = 'script/py_script.py'
file_key_2.set_contents_from_filename('this_script.py')
# Getting a list of objects in a bucket
keys_list=bucket.list()
for key in keys_list:
print (key.key)
# Deleting multiple objects
response = bucket.delete_keys(['test-string', 'py_script.py'])
# Getting an object
key = bucket.get_key('script/py_script.py')
print (key.get_contents_as_string())
For an example, see this video conversion guide.