Connecting to S3 using the boto3 library

This guide describes how to connect to an S3 object storage in Jupyter Notebook using the boto3 library. To connect to an object storage, you can use an S3 connector:

To set up an S3 connection from the notebook code:

1. Create the secrets: token (with ID) and key_value (with a the secret part of the static access key for the service account).

2. Open the DataSphere project:

   1. Select the relevant project in your community or on the DataSphere homepage in the Recent projects tab.

   2. Click Open project in JupyterLab and wait for the loading to complete.
   3. Open the notebook tab.

3. Import the libraries:

   import boto3
   import os
   from os import path
   

4. Enter the name of your bucket in the storage.

   bucket_name = '<bucket_name>'
   

5. Establish a connection:

   session = boto3.session.Session()
   
   ENDPOINT = "https://storage.yandexcloud.net"
   
   session = boto3.Session(
       aws_access_key_id=(os.environ['token']),
       aws_secret_access_key=(os.environ['key_value']),
       region_name="ru-central1",
   )
   
   s3 = session.client(
       "s3", endpoint_url=ENDPOINT)
   

6. Enter the bucket name and retrieve a list of objects contained in it.

   for key in s3.list_objects(Bucket='<bucket_name>')['Contents']:
       print(key['Key'])