Getting started with DataSphere

Yandex DataSphere is an end-to-end ML development environment where you can use well-known IDEs, serverless computing technology, and seamlessly combine a broad range of Yandex Cloud computing resource configurations. Yandex DataSphere is part of the data platform and provides powerful features to work with Yandex Cloud services. As an IDE, DataSphere provides Jupyter® Notebook.

In this section, you will learn how to:

1. Create a project.
2. Run projects.
3. Configure the environment.
4. Upload data to projects.
5. Start training.

Getting startedGetting started

1. Go to the management console and log in to Yandex Cloud or sign up if not signed up yet.
2. Go to Yandex Cloud Billing and make sure you have a billing account linked and its status is ACTIVE or TRIAL_ACTIVE. If you do not have a billing account yet, create one.
3. Open the DataSphere home page.
4. Accept the user agreement.
5. Select the organization to work with DataSphere in or create a new one.

Create a projectCreate a project

1. Open the DataSphere home page.
2. In the left-hand panel, select Communities.
3. Select the community where you want to create a project.
4. On the community page, click Create project.
5. In the window that opens, enter a name for the project. You can also add a description as needed.
6. Click Create.

Run the projectRun the project

To run a project, click Open project in JupyterLab.

Set up your environmentSet up your environment

Popular packages for data analysis and machine learning are pre-installed and ready for use. You can find a list of these packages here.

To install missing packages, use pip.

To install a package:

1. Enter the following command in a notebook cell:

   %pip install <package_name>
   

   For example, install the seaborn package to visualize statistics:

   %pip install seaborn
   

   You can use various options the pip install command supports. For examples of using this command, follow this link.

2. Run the cell. To do this, click .

   The package installation result will show up under the cell.

You can also configure the environment to run your code using Docker images.

Upload data to the projectUpload data to the project

You can upload small data amounts (up to 100 MB) to your DataSphere project through the JupyterLab interface. For larger amounts of data, we recommend loading from network storage or databases. To handle large data amounts, you can also use datasets.

To upload data to your project via the JupyterLab interface:

1. Under File Browser, select the folder for uploading data.
2. At the top left, click .
3. Select the files to upload.

Learn more about project storage here.

DataSphere allows you to upload data from different sources:

• Connecting to S3 using boto3
• Connecting to Google Drive
• Connecting to a ClickHouse® database
• Connecting to a PostgreSQL database
• Connecting to Yandex Disk

Start trainingStart training

To start computations:

1. Under File Browser, select the notebook with the Python or Bash code.

2. Select and run one or more code cells by selecting Run → Run Selected Cells, or pressing Shift + Enter.

3. Wait for the operation to complete.

   The result will show up under the cell.

What's nextWhat's next

• Learn about service features.
• See other service guides.
• Deploy the trained model as a service.