Getting started with DataSphere

Yandex DataSphere is an end-to-end ML development environment where you can use familiar IDEs, serverless computing technology, and seamlessly combine a broad range of Yandex Cloud computing resource configurations. Yandex DataSphere is part of the data platform and offers powerful features to easily work with Yandex Cloud services. As an IDE, DataSphere provides Jupyter® Notebook.

In this section, you will learn how to:

1. Create projects.
2. Run projects.
3. Configure the environment.
4. Upload data to projects.
5. Start training.
6. Share your results.

Getting startedGetting started

1. Go to the management console and log in to Yandex Cloud or sign up if not signed up yet.
2. Go to Yandex Cloud Billing and make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account yet, create one.
3. Open the DataSphere home page.
4. Accept the user agreement.
5. Select the organization to work with DataSphere in or create a new one.

Create a projectCreate a project

1. Open the DataSphere home page.
2. In the left-hand panel, select Communities.
3. Select the community to create a project in.
4. On the community page, click Create project.
5. In the window that opens, enter a name and description (optional) for the project.
6. Click Create.

Run the projectRun the project

To run a project, click Open project in JupyterLab.

Configure the environmentConfigure the environment

Popular packages for data analysis and machine learning are pre-installed and ready for use, see the list.

You can install missing packages using the pip package manager.

To install a package:

1. Write the following command in the notebook cell:

   %pip install <package_name>
   

   For example, install the seaborn package to visualize statistics:

   %pip install seaborn
   

   You can use various options that the pip install command supports. See usage examples for this command.

2. Run the cell. To do this, click .

   The package installation result is displayed under the cell.

You can also configure the environment to run your code using Docker images.

Upload data to the projectUpload data to the project

You can upload small data volumes (up to 100 MB) to your DataSphere project through the JupyterLab interface. We recommend uploading larger data volumes from network storages or databases. For large data volumes, datasets make another convenient option.

To upload data to your project through the JupyterLab interface:

1. In the File Browser section, select a folder for the data.
2. Click at the top left.
3. Select the files to upload.

Learn more about project storage.

DataSphere allows you to upload data from different sources:

• Connecting to S3 using the boto3 library
• Connecting to Google Drive
• Connecting to a ClickHouse® database
• Connecting to a PostgreSQL database
• Connecting to Yandex Disk

Start trainingStart training

To start computations:

1. Under the File Browser section, select the notebook with the Python or bash code.

2. Select and run one or more cells with the code by choosing Run → Run Selected Cells, or pressing Shift + Enter.

3. Wait for the operation to complete.

   The execution result is displayed under the cell.

What's nextWhat's next

• Learn about service features.
• See other service guides.
• Deploy the trained model as a service.