Docker images
You can set up a DataSphere project environment on your own by using a Docker image with any software, libraries, environment variables, and configuration files. The Docker image you apply to your project will be used when running code in all its notebooks.
You cannot change Docker images after you create them.
Warning
Starting January 31, 2027, all Docker images created in DataSphere before December 3, 2026, will no longer work. To use these images in projects, recreate them or use the migration tool.
Ways to store Docker images
There are two ways to store Docker images:
- In Yandex Container Registry
- In DataSphere
Storage in Container Registry
By default, Docker images are stored in Container Registry. They are built on a DataSphere VM and, if successful, pushed to the registry specified in the community settings, with its version logged in the relevant DataSphere resource. Whenever a project using this image runs, DataSphere pulls this version from the registry. Deleting the image version from the registry keeps the image in the project but disables its further optimization, activation, or use.
Storing images in Container Registry is cheaper, but it increases the project start time. You also cannot share an image from the registry with the community.
To reduce the project start time, you can create an optimized copy of your Docker image stored in Container Registry. This optimized copy is stored on a separate disk in DataSphere and greatly reduces start time, but storing it adds extra charges. You can publish an optimized Docker image in the community to use it in multiple projects.
After you create an optimized copy, you can delete the original image from the registry to avoid paying for storing both the original and the copy. If you delete an optimized copy of the Docker image, DataSphere will attempt to pull the Docker image from Container Registry again when the project starts. The project will continue using the original version if it is still available in Container Registry.
Tip
To allow project users to build and store Docker images in Container Registry, the community administrator should specify the registry in the community settings and enable the service agent.
Storage in DataSphere
If you select DataSphere for storage when creating a Docker image, it will be stored on a dedicated disk in DataSphere once built and will not be pushed to Container Registry. This image is immediately considered optimized and billed accordingly.
If you delete a Docker image stored in DataSphere, you will not be able to restore it.
You can publish a Docker image stored in DataSphere in the community to use it in multiple projects. To do this, you need at least the Editor role in the project and the Developer role in the community you want to publish it in. You can open the access on the Access tab on the Docker image view page. The resource available to the community will appear on the community page under Community resources. While the image is published, you cannot delete it.
For more information, see Working with Docker images.
Docker image requirements
For a Docker image to run and work correctly in DataSphere, it must include:
- Python 3.8, 3.9, 3.10, or 3.11 installation
- Pip installation
--uid 1000 jupyteruser
Tip
The Docker Hub image library has limits on anonymous usage. If you encounter slow or failed base image pulls when building a Docker image, try changing your project IP address. To do this:
- Create a subnet.
- Create an egress NAT gateway.
- Create a service account with the
vpc.userrole. - In the project settings, add the subnet and the service account you created.
You can also use basic images from other libraries.
Note
The Docker image templates available in DataSphere already meet these requirements. If you are downloading an image from a third-party source or writing your own, include the required commands.