Datasets in DataSphere

A dataset in DataSphere is a way to store information that provides quick access to large amounts of data. Datasets can store up to 4 TB of data, giving faster access to data than to the main project storage.

A dataset is created and populated during initialization. Once initialized, you cannot change a dataset, as it becomes read-only. If you want to add files to a dataset, create it once again.

Datasets are not included in the main project storage and are priced separately.

To use your data in multiple projects, you can share your datasets within your community, just like any other resources.

When activated in the project, a disk with a dataset is mounted to the project storage. You can read the files of your activated dataset as local files in your project storage at /home/jupyter/datasets/<dataset_name>.

You can have up to three active datasets within a project at the same time. You can activate and deactivate datasets in your projects on the go, without having to restart them. For the full list of DataSphere restrictions, see Quotas and limits in DataSphere.

Information about a dataset as a resourceInformation about a dataset as a resource

For each dataset, the following information is stored:

• Name.
• Status of its connection to the project.
• Name of the user who created the dataset.
• Dataset creation date in UTC format, e.g., July 18, 2022, 14:23.

To view dataset details, click its name in the project's dataset list. On the dataset's Overview tab, you can see:

• Availability zone the dataset is stored in.
• Size.
• Initialization code.

Useful linksUseful links

• Working with datasets