Deploying a service from an ONNX model

DataSphere allows you to deploy and run services based on a model trained in the project.

In this tutorial, you will deploy a service from an ONNX model. The fast-neural-style-mosaic-onnx model transforms an image as per the style specified. The model is taken from the ONNX model repository.

1. Prepare your infrastructure.
2. Create a model.
3. Create a node.
4. Run a health check for the service you deployed.

If you no longer need the resources you created, delete them.

Getting startedGetting started

Before getting started, register in Yandex Cloud, set up a community, and link your billing account to it.

1. On the DataSphere home page, click Try for free and select an account to log in with: Yandex ID or your working account in the identity federation (SSO).
2. Select the Yandex Cloud Organization organization you are going to use in Yandex Cloud.
3. Create a community.
4. Link your billing account to the DataSphere community you are going to work in. Make sure that you have a billing account linked and its status is ACTIVE or TRIAL_ACTIVE. If you do not have a billing account yet, create one in the DataSphere interface.

Required paid resourcesRequired paid resources

The cost of deploying a service from a model includes:

• Fee for continuously running node instances (see DataSphere pricing).
• Fee for running code cells for health checks of the deployed service.

Prepare the infrastructurePrepare the infrastructure

Log in to the Yandex Cloud management console and select the organization you use to access DataSphere. On the Yandex Cloud Billing page, make sure you have a billing account linked.

If you have an active billing account, you can create or select a folder to deploy your infrastructure in, on the cloud page.

Create a folderCreate a folder

Create a folder for you to deploy your infrastructure and for your service to store the logs.

Create a service account for the DataSphere projectCreate a service account for the DataSphere project

Create a modelCreate a model

The fast-neural-style-mosaic-onnx model is one of the style transfer models designed to convert an image to the style of another image.

1. Open the DataSphere project:

   1. Select the relevant project in your community or on the DataSphere homepage in the Recent projects tab.

   2. Click Open project in JupyterLab and wait for the loading to complete.
   3. Open the notebook tab.

2. Install the onnx module:

   %pip install onnx
   

3. Download the mosaic.onnx file:

   !wget -O "mosaic.onnx" "https://github.com/onnx/models/raw/main/validated/vision/style_transfer/fast_neural_style/model/mosaic-8.onnx"
   

4. Load the model into a variable and check it for consistency:

   import onnx
   
   onnx_model = onnx.load("mosaic.onnx")
   onnx.checker.check_model(onnx_model)
   

5. In the right-hand panel, select . In the window that opens, click Create model.

6. Select the name of the variable to create your model from.

7. Enter a name for the model, e.g., onnx-model.

8. Click Create.

Create a nodeCreate a node

1. Select the relevant project in your community or on the DataSphere homepage in the Recent projects tab.

2. In the top-right corner, click Create resource. In the pop-up window, select Node.
3. Enter a name for the node in the Name field.
4. Under Type, specify the resource type: Model.
5. In the Models field, select onnx-model.
6. Under Folder, select data-folder.
7. Under Provisioning, select the configuration of instance computing resources, the availability zone, and the ID of the subnet to host the instance in.
8. Under Access control list (ACL), click Add ACL and specify the IDs of the folders to allow connections to the node from. By default, the ID of the folder owned by the user creating the node is specified.
9. Click Create.

Run a health check for the service you deployedRun a health check for the service you deployed

1. Create secrets to test your node:

   • IAM_SECRET with IAM token value.
   • NODE_ID with node ID.
   • FOLDER_ID with folder ID.

2. Open the DataSphere project:

   1. Select the relevant project in your community or on the DataSphere homepage in the Recent projects tab.

   2. Click Open project in JupyterLab and wait for the loading to complete.
   3. Open the notebook tab.

3. Install the tritonclient module:

   %pip install tritonclient[http]
   

4. Import the required libraries:

   import numpy as np
   import matplotlib.pyplot as plt
   from PIL import Image
   import requests
   import urllib
   import os
   %matplotlib inline
   

5. Prepare your image:

   url, filename = ("https://github.com/pytorch/hub/raw/master/images/dog.jpg", "dog.jpg")
   try: urllib.URLopener().retrieve(url, filename)
   except: urllib.request.urlretrieve(url, filename)
   
   # loading input and resize if needed
   onnx_image = Image.open(filename)
   
   size_reduction_factor = 5
   onnx_image = onnx_image.resize((224, 224))
   
   # Preprocess image
   x = np.array(onnx_image).astype('float32')
   x = np.transpose(x, [2, 0, 1])
   onnx_input = np.expand_dims(x, axis=0)
   
   plt.figure(figsize=(15, 15))
   plt.imshow(onnx_image)
   plt.show()
   

6. Authenticate using the secrets created earlier:

   IAM_SECRET = os.environ['IAM_SECRET']
   NODE_ID = os.environ['NODE_ID']
   FOLDER_ID = os.environ['FOLDER_ID']
   
   headers = {
       "Authorization": f"Bearer {IAM_SECRET}", # get IAM token from secrets
       "x-node-id": f"{NODE_ID}", # sample node
       "x-folder-id": f"{FOLDER_ID}" # node folder ID
   }
   

7. Send test requests with the model ID substituted as follows:

   import tritonclient.http as httpclient
   
   model="<DataSphere_model_ID>"
   
   # request model config with model ready status
   print(f"""model_name: {model},\n
         model_ready: {triton_client.is_model_ready(model_name=model, headers=headers)},\n
         model_config: {triton_client.get_model_config(model_name=model, headers=headers)}\n""")
   

8. Prepare your image and send a request to the node:

   payload = httpclient.InferInput("input1", list(onnx_input.shape), "FP32")
   payload.set_data_from_numpy(onnx_input, binary_data=False)
   results = triton_client.infer(model, inputs=[payload], headers=headers)
   

9. Receive the result of image transformation:

   output = results.as_numpy("output1")[0]
   output = np.clip(output, 0, 255)
   output = output.transpose(1,2,0).astype("uint8")
   img = Image.fromarray(output)
   img
   

   You will get the transformed image in the response.

How to delete the resources you createdHow to delete the resources you created

When deploying and using models, you pay for the uptime of each node instance: from its start to deletion.

If you no longer need the service you deployed, delete the node.

1. Delete the node.
2. Delete the secrets.