Image generation using the Stable Diffusion model

In DataSphere, you can deploy a neural network based on the Stable Diffusion model and generate images based on text descriptions.

Stable Diffusion is an open-source text-to-image model developed by stability.ai.

In this tutorial, you will generate an image based on a text description by implementing the Stable Diffusion model in the Diffusers library. This library prioritizes ease of use and customization over performance.

To generate an image using the Stable Diffusion model:

1. Set up your infrastructure.
2. Create a model and generate an image.

If you no longer need the resources you created, delete them.

Getting startedGetting started

Before getting started, register in Yandex Cloud, set up a community, and link your billing account to it.

1. On the DataSphere home page, click Try for free and select an account to log in with: Yandex ID or your working account with the identity federation (SSO).
2. Select the Yandex Identity Hub organization you are going to use in Yandex Cloud.
3. Create a community.
4. Link your billing account to the DataSphere community you are going to work in. Make sure you have a linked billing account and its status is ACTIVE or TRIAL_ACTIVE. If you do not have a billing account yet, create one in the DataSphere interface.

Required paid resourcesRequired paid resources

The cost of using the model includes a fee for running code cells (see DataSphere pricing).

Set up your infrastructureSet up your infrastructure

Create a projectCreate a project

1. Open the DataSphere home page.
2. In the left-hand panel, select Communities.
3. Select the community where you want to create a project.
4. On the community page, click Create project.
5. In the window that opens, enter Stable Diffusion as your project name and add a description (optional).
6. Click Create.

Create a notebook and install the librariesCreate a notebook and install the libraries

1. In the DataSphere interface, open the project you created.

2. Create a new notebook:

   1. In the top panel of the project window, click File → New → Notebook.
   2. In the window that opens, select DataSphere Kernel.

3. Install the Diffusers library. Paste the below code into the cell and click :

   %pip install diffusers
   

4. Install the Transformers library:

   %pip install transformers
   

5. Once the installation is complete, select Kernel ⟶ Restart kernel... on the top panel.

Create a model and generate an imageCreate a model and generate an image

1. Import the libraries to the project:

   from diffusers import StableDiffusionPipeline
   import torch
   

2. Create a model:

   model_id = "runwayml/stable-diffusion-v1-5"
   pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
   pipe = pipe.to("cuda")
   

3. Generate an image by its description:

   prompt = "a photo of an astronaut riding a horse on mars"
   image = pipe(prompt).images[0]
   

4. Save the output image:

   image.save("astronaut_rides_horse.png")
   

   The image file will appear next to the notebook. Result:

   

How to delete the resources you createdHow to delete the resources you created

If you no longer plan to use the Stable Diffusion project, delete it.

Useful linksUseful links

• Working with models
• Running computations in DataSphere using the API