Deploying a service in Yandex DataSphere from an ONNX model

DataSphere allows you to deploy and run services based on a model trained in the project.

In this tutorial, you will deploy a service based on an ONNX model. The fast-neural-style-mosaic-onnx model transforms an image as per the specified style. The model is taken from the ONNX model repository.

1. Set up your infrastructure.
2. Create a model.
3. Create a node.
4. Run a health check for the service you deployed.

If you no longer need the resources you created, delete them.

Getting startedGetting started

Before getting started, register in Yandex Cloud, set up a community, and link your billing account to it.

1. On the DataSphere home page, click Try for free and select an account to log in with: Yandex ID or your working account with the identity federation (SSO).
2. Select the Yandex Identity Hub organization you are going to use in Yandex Cloud.
3. Create a community.
4. Link your billing account to the DataSphere community you are going to work in. Make sure you have a linked billing account and its status is ACTIVE or TRIAL_ACTIVE. If you do not have a billing account yet, create one in the DataSphere interface.

Required paid resourcesRequired paid resources

The cost of deploying a service based on a model includes:

• Fee for continuously running node instances (see DataSphere pricing).
• Fee for running code cells for health checks of the deployed service.

Set up your infrastructureSet up your infrastructure

Log in to the Yandex Cloud management console and select the organization you use to access DataSphere. On the Yandex Cloud Billing page, make sure you have a billing account linked.

If you have an active billing account, you can go to the cloud page to create or select a folder to run your infrastructure.

Create a folderCreate a folder

Create a folder for you to deploy your infrastructure and for your service to store the logs.

Create a service account for the DataSphere projectCreate a service account for the DataSphere project

Create a modelCreate a model

The fast-neural-style-mosaic-onnx model is one of the style transfer models designed to convert an image to the style of another image.

1. Open the DataSphere project:

   1. Select the project in your community or on the DataSphere home page in the Recent projects tab.

   2. Click Open project in JupyterLab and wait for the loading to complete.
   3. Open the notebook tab.

2. Install the onnx module:

   %pip install onnx
   

3. Download the mosaic.onnx file:

   !wget -O "mosaic.onnx" "https://github.com/onnx/models/raw/main/validated/vision/style_transfer/fast_neural_style/model/mosaic-8.onnx"
   

4. Load the model into a variable and check it for consistency:

   import onnx
   
   onnx_model = onnx.load("mosaic.onnx")
   onnx.checker.check_model(onnx_model)
   

5. In the right-hand panel, select . In the window that opens, click Create model.

6. Select the name of the variable to build your model from.

7. Name the model, e.g., onnx-model.

8. Click Create.

Create a nodeCreate a node

1. Select the project in your community or on the DataSphere home page in the Recent projects tab.

2. In the top-right corner, click Create resource. In the pop-up window, select Node.
3. Enter a name for the node in the Name field.
4. Under Type, specify the resource type: Model.
5. In the Models field, select onnx-model.
6. Under Folder, select data-folder.
7. Under Provisioning, select the configuration of instance computing resources, availability zone, and the ID of the subnet to host the instance.
8. Under Access control list (ACL), click Add ACL and specify the IDs of the folders to allow connections to the node from. The folder ID of the user creating the node is specified by default.
9. Click Create.

Run a health check for the service you deployedRun a health check for the service you deployed

1. Create secrets to test your node:

   • IAM_SECRET: IAM token value.
   • NODE_ID: Node ID.
   • FOLDER_ID: Folder ID.

2. Open the DataSphere project:

   1. Select the project in your community or on the DataSphere home page in the Recent projects tab.

   2. Click Open project in JupyterLab and wait for the loading to complete.
   3. Open the notebook tab.

3. Install the tritonclient module:

   %pip install tritonclient[http]
   

4. Import the required libraries:

   import numpy as np
   import matplotlib.pyplot as plt
   from PIL import Image
   import requests
   import urllib
   import os
   %matplotlib inline
   

5. Prepare your image:

   url, filename = ("https://github.com/pytorch/hub/raw/master/images/dog.jpg", "dog.jpg")
   try: urllib.URLopener().retrieve(url, filename)
   except: urllib.request.urlretrieve(url, filename)
   
   # loading input and resize if needed
   onnx_image = Image.open(filename)
   
   size_reduction_factor = 5
   onnx_image = onnx_image.resize((224, 224))
   
   # Preprocess image
   x = np.array(onnx_image).astype('float32')
   x = np.transpose(x, [2, 0, 1])
   onnx_input = np.expand_dims(x, axis=0)
   
   plt.figure(figsize=(15, 15))
   plt.imshow(onnx_image)
   plt.show()
   

6. Get authenticated using the secrets created earlier:

   IAM_SECRET = os.environ['IAM_SECRET']
   NODE_ID = os.environ['NODE_ID']
   FOLDER_ID = os.environ['FOLDER_ID']
   
   headers = {
       "Authorization": f"Bearer {IAM_SECRET}", # get IAM token from secrets
       "x-node-id": f"{NODE_ID}", # sample node
       "x-folder-id": f"{FOLDER_ID}" # node folder ID
   }
   

7. Send test requests with the model ID substituted as follows:

   import tritonclient.http as httpclient
   
   model="<model_ID_in_DataSphere>"
   
   # request model config with model ready status
   print(f"""model_name: {model},\n
         model_ready: {triton_client.is_model_ready(model_name=model, headers=headers)},\n
         model_config: {triton_client.get_model_config(model_name=model, headers=headers)}\n""")
   

8. Prepare your image and send a request to the node:

   payload = httpclient.InferInput("input1", list(onnx_input.shape), "FP32")
   payload.set_data_from_numpy(onnx_input, binary_data=False)
   results = triton_client.infer(model, inputs=[payload], headers=headers)
   

9. Get the result of image transformation:

   output = results.as_numpy("output1")[0]
   output = np.clip(output, 0, 255)
   output = output.transpose(1,2,0).astype("uint8")
   img = Image.fromarray(output)
   img
   

   You will get the transformed image in the response.

How to delete the resources you createdHow to delete the resources you created

When deploying and using models, you pay for the uptime of each node instance: from its start to deletion.

If you no longer need the service you deployed, delete the node.

1. Delete the node.
2. Delete the secrets.