Connecting to an S3 storage

You can manage your S3 object storage connection by using the S3 Connector resource on your project page in the DataSphere interface.

To connect to an object storage from the notebook code, follow this guide.

Getting startedGetting started

Get an access key from your S3 storage provider. To do this in Yandex Object Storage, follow these steps:

1. Create a service account.
2. Assign the new account a role with either read or read and write permissions.
3. Create an access key for the service account.

Creating an S3 connectorCreating an S3 connector

1. Select the project in your community or on the DataSphere home page in the Recent projects tab.

2. Optionally, in the top-right corner, click Create resource. In the pop-up window, select Secret and create a secret with the secret part of the static access key for your service account. You can also create a secret when creating your S3 connection.
3. In the top-right corner, click Create resource. In the pop-up window, select S3 Connector.
4. Fill in the fields as follows:

   • Name: Name of the new connector. Follow these naming requirements:

     • The name must be from 3 to 63 characters long.
     • It may contain uppercase and lowercase Latin and Cyrillic letters, numbers, hyphens, underscores, and spaces.
     • The first character must be a letter. The last character cannot be a hyphen, underscore, or space.

   • Description: Description of the new connector. This is an optional parameter.

   • Endpoint: Storage host. For Object Storage, set it to https://storage.yandexcloud.net/.

   • Bucket: Name of the storage bucket.

     Warning

     Do not use buckets with periods in their names for connection. Learn more about buckets here.

   • Mount name: Name of the volume for mounting the bucket into the project file system. Make sure to follow these naming requirements:

     • It must be from 2 to 63 characters long.
     • It can only contain lowercase Latin letters, numbers, and hyphens.
     • It must start with a letter and cannot end with a hyphen.

   • Static access key ID: ID of the static access key used to connect to the storage.

   • Static access key: In the list, select a secret that contains the secret part of the static access key or create a new secret.

   • Mode: Object storage access mode, Read only or Read and write.

5. Click Create.

Attaching an S3 storage to a projectAttaching an S3 storage to a project

Go to the S3 connector page and click Activate. Once activated, the bucket will be available in the /s3/ folder of the JupyterLab file browser, and you will be able to view it as a file system.

Using an S3 storage in a projectUsing an S3 storage in a project

You can access files in the connected bucket from your project code. Select the file you need in the mounted S3 storage on the S3 Mounts tab, right-click it, and select Copy path. This will copy the file path to the clipboard. Paste the copied path wherever you need in the project.

Detaching an S3 storageDetaching an S3 storage

1. On the project page, under Project resources, click S3 Connector.
2. Select the connector and go to the resource page.
3. Click Deactivate in the top-right corner of the page.

You can re-attach the S3 storage to your project whenever you need it.

Sharing an S3 connectorSharing an S3 connector

1. Select the project in your community or on the DataSphere home page in the Recent projects tab.

2. Under Project resources, click S3 Connector.
3. Select the S3 connector from the list.
4. Go to the Access tab.
5. Enable the visibility option next to the name of the community you want to share the S3 connector with.

To make an S3 connector available for use in a different project, the project admin needs to add that connector on the Shared tab.

Deleting an S3 connectorDeleting an S3 connector

You can only delete a deactivated connector that is not available to any community.

1. Select the project in your community or on the DataSphere home page in the Recent projects tab.

2. Under Project resources, click S3 Connector.
3. Next to the S3 storage, click and select Delete.
4. Click Confirm.

You will see a message saying that the secret has been deleted.