S3 connector
DataSphere allows you to connect to an S3 object storage using a special type of resource, S3 connector.
An S3 connector is a resource that stores settings for connection to buckets of cloud services, such as Yandex Object Storage. Upon connection to a bucket, you will see its objects in the Jupyter Notebook interface.
Connectors are created in the project and assigned to it. The connector's static key value is stored in a secret-encrypted format.
Note
Avoid using S3 storage in FUSE mode for buckets with single-layer (non-recursive) folders that include many files, as this will significantly decrease storage performance.
Information on an S3 connector as a resource
The system stores the following information about each S3 connector:
- Unique resource ID
- Description
- Endpoint
- Bucket name
- ID and static key value
- Operating mode
- Backend
- Resource creator
- Creation and last update date in UTC format, such as
July 18, 2022, 14:23
Working with S3 connectors
You can create an S3 connector in the DataSphere interface. To make objects from the bucket visible in Jupyter, activate the S3 connector. If you no longer need the connection, deactivate it.
Once created, your S3 connector becomes available for the project. Like any other resource, you can publish the S3 connector in the community to use it in other projects. To do this, you need at least the Editor role in the project and the Developer role in the community in which you want to publish it. You can open the access on the Access tab on the S3 connector view page. The resource available to the community will appear on the community page under Community resources.
Note
You need to set up a NAT gateway for any subnet linked to the project.