Connecting to S3 using boto3

Follow this guide to connect to an S3 object storage in Jupyter Notebook using boto3. To connect to an object storage, use an S3 connector:

To set up an S3 connection from the notebook code:

1. Create secrets: token (with ID) and key_value (with a secret part of the static access key for the service account).

2. Open the DataSphere project:

   1. Select the project in your community or on the DataSphere home page in the Recent projects tab.

   2. Click Open project in JupyterLab and wait for the loading to complete.
   3. Open the notebook tab.

3. Import the libraries:

   import boto3
   import os
   from os import path
   

4. Enter the name of your bucket in the storage:

   bucket_name = '<bucket_name>'
   

5. Establish a connection:

   session = boto3.session.Session()
   
   ENDPOINT = "https://storage.yandexcloud.net"
   
   session = boto3.Session(
       aws_access_key_id=(os.environ['token']),
       aws_secret_access_key=(os.environ['key_value']),
       region_name="ru-central1",
   )
   
   s3 = session.client(
       "s3", endpoint_url=ENDPOINT)
   

6. Get a list of objects in the bucket:

   paginator = s3.get_paginator('list_objects_v2')
   for page in paginator.paginate(Bucket='<bucket_name>'):
       for key in page.get('Contents', []):
           print(key['Key'])