Contact UsGet started
© 2024 Iron Hive LLC Belgrade

Deploying a service based on a Docker image

Written by
Updated at November 14, 2024

DataSphere enables you to deploy and run services based on any Docker image.

In this tutorial, you will use a node built from a Docker image to deploy an object detection service based on NVIDIA Triton Inference Server.

  1. Prepare your infrastructure.
  2. Prepare a Docker image for the service deployment.
  3. Deploy the service in DataSphere.
  4. Run a health check for the service you deployed.

If you no longer need the resources you created, delete them.

Getting started

Before getting started, register in Yandex Cloud, set up a community, and link your billing account to it.

  1. On the DataSphere home page, click Try for free and select an account to log in with: Yandex ID or your working account in the identity federation (SSO).
  2. Select the Yandex Cloud Organization organization you are going to use in Yandex Cloud.
  3. Create a community.
  4. Link your billing account to the DataSphere community you are going to work in. Make sure that you have a billing account linked and its status is ACTIVE or TRIAL_ACTIVE. If you do not have a billing account yet, create one in the DataSphere interface.

The cost of deploying a service based on a Docker image includes:

  • Fee for continuously running node instances (see DataSphere pricing).
  • Fee for running code cells for health checks of the deployed service.
  • Amount of storage used by Yandex Container Registry data (see Container Registry pricing).

Prepare the infrastructure

Log in to the Yandex Cloud management console and select the organization you use to access DataSphere. On the Yandex Cloud Billing page, make sure you have a billing account linked.

If you have an active billing account, you can create or select a folder to deploy your infrastructure in, on the cloud page.

Note

If you use an identity federation to access Yandex Cloud, billing details might be unavailable to you. In this case, contact your Yandex Cloud organization administrator.

Create a folder

Create a folder for you to deploy your infrastructure and for your service to store the logs.

Note

In our example, both the Yandex Cloud infrastructure and the deployed service operate from the same Yandex Cloud folder; however, this is not a requirement.

  1. In the management console, select a cloud and click Create folder.
  2. Name your folder, e.g., data-folder.
  3. Click Create.

Create a registry in Container Registry

DataSphere can create a node from a Docker image hosted in Yandex Container Registry. To push an image, create a registry.

  1. Go to data-folder.
  2. In the list of services, select Container Registry.
  3. Click Create registry.
  4. Specify a name for the registry, e.g., datasphere-registry, and click Create registry.

Create a service account for the DataSphere project

  1. Go to data-folder.

  2. In the list of services, select Identity and Access Management.

  3. Click Create service account.

  4. Enter a name for the service account, e.g., sa-for-datasphere.

  5. Click Add role and assign the following roles to the service account:

    • container-registry.images.puller to allow DataSphere to pull your Docker image to create a node.
    • vpc.user to use the DataSphere network.
    • (Optional) datasphere.user to send requests to the node.

  6. Click Create.

Create an authorized key for a service account

To allow your service account to get authenticated in Yandex Container Registry, create an authorized key.

Note

Authorized keys do not expire, but you can always get new authorized keys and get authenticated again if something goes wrong.

  1. Go to data-folder.

  2. In the list of services, select Identity and Access Management.

  3. In the left-hand panel, select Service accounts.

  4. In the list that opens, select the sa-for-datasphere service account.

  5. Click Create new key in the top panel.

  6. Select Create authorized key.

  7. Select the encryption algorithm.

  8. Enter a description of the key so that you can easily find it in the management console.

  9. Save both the public and private keys. The private key is not saved in Yandex Cloud, and you will not be able to view the public key in the management console.

    Tip

    You can save the file with the key on your computer. You will need its contents later when creating a secret to access DataSphere in Container Registry.

Prepare your Docker image for the service deployment

If you do not have Docker yet, install it.

Create a Docker image for your service

  1. Create a folder to store the configuration of your Docker image, e.g., /home/docker-images.

  2. Create and save a text file named Dockerfile (without any extension).

    Triton Dockerfile
    # syntax=docker/dockerfile:1
FROM nvcr.io/nvidia/tritonserver:22.01-py3

RUN mkdir -p /models/resnet152_640x640/1/model.savedmodel/ &&\
  curl --location "https://tfhub.dev/tensorflow/faster_rcnn/resnet152_v1_640x640/1?tf-hub-format=compressed" |\
  tar -zxvC /models/resnet152_640x640/1/model.savedmodel/ &&\
  mkdir -p /models/inception_resnet_v2_640x640/1/model.savedmodel/ &&\
  curl --location "https://tfhub.dev/tensorflow/faster_rcnn/inception_resnet_v2_640x640/1?tf-hub-format=compressed" |\
  tar -zxvC /models/inception_resnet_v2_640x640/1/model.savedmodel/

ENTRYPOINT ["/opt/tritonserver/nvidia_entrypoint.sh",\
            "tritonserver",\
            "--model-repository=/models",\
            "--strict-model-config=false"]

  3. Run Docker Desktop.

  4. In the command shell, navigate to the folder with Dockerfile you created:

    cd docker-images

  5. Build the Docker image:

    docker build -t triton-docker --platform linux/amd64 .

Push the Docker image to Container Registry

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

  1. Set data-folder as your default folder:

    yc config set folder-name data-folder

  2. Get authenticated in Container Registry.

    1. Issue an IAM token for your service account:
    yc iam create-token

    The response will contain the IAM token. If you are authenticating using a federated account, the CLI will redirect you to the management console to authenticate and then send you an IAM token.

    Note

    The IAM token has a short lifetime: no more than 12 hours. This makes it a good method for applications that automatically request an IAM token.

    1. Run the command with the token value you got in the previous step in place of <IAM_token>:
    docker login \
  --username iam \
  --password <IAM_token> \
  cr.yandex

  3. Get a list of registries in data-folder:

    yc container registry list

    You will need the registry ID at the next step. Command output example:

    +----------------------+---------------------+----------------------+
|          ID          |        NAME         |      FOLDER ID       |
+----------------------+---------------------+----------------------+
| crp86bmgl1da******** | datasphere-registry | b1g4bh24c406******** |
+----------------------+---------------------+----------------------+

  4. Push the Docker image to Container Registry. Instead of <registry_ID>, use the ID of your registry (datasphere-registry):

    docker tag triton-docker cr.yandex/<registry_ID>/triton:v1
docker push cr.yandex/<registry_ID>/triton:v1

Deploy the service in DataSphere

  1. Open the DataSphere home page. In the left-hand panel, select Communities.

  2. Select a community with a billing account linked.
  3. Create a project named Node from Docker.
  4. In the project settings, specify:
    • Default folder: data-folder.
    • Service account: sa-for-datasphere.
  5. Create a secret named iam-secret containing the IAM token of your user account.
  6. Create a secret named key-for-sa to store the full contents of the authorized key file for the sa-for-datasphere service account.
  7. Create a node. To do this, click Create resource in the top-right corner of the project page. In the pop-up window, select Node. Specify the node settings:
    1. Enter triton as the node name in the Name field.
    2. Under Type:
      • Type: Select Docker.
      • Docker image storage: Select Yandex Container Registry.
      • Image path: Specify the path to the Container Registry image in the cr.yandex/<registry_ID>/<image_name>:<tag> format. You can get it in the management console by copying the full value on the repository page. You can also fill out this field manually. You can get the registry ID in the CLI by running the yc container registry list command.
      • Password secret: Select key-for-sa.
    3. Under Endpoint:
      • Type: Select HTTP.
      • Port: 8000.
    4. Enable Telemetry and specify:
      • Type: Select Prometheus.
      • HTTP path: /metrics.
      • Port: 8000.
    5. Enable Healthcheck and specify:
      • Type: Select HTTP.
      • Path: /v2/health/ready.
      • Port: 8000.
      • Timeout: 1.
      • Interval: 20.
      • Fails threshold: 3.
      • Passes threshold: 3.
    6. Under Folder, select data-folder.
    7. Under Provisioning select the g1.1 configuration.
    8. Click Create.

Run a health check for the service you deployed

  1. Download a notebook with the health check code and upload it to the Node from Docker project's JupyterLab.
  2. Run the cells in the Preparing environment section: select the cells and press Shift + Enter.
  3. Under Authentication, fill out the details to get authenticated in the node. Replace <node_ID> and <folder_ID> with triton and data-folder, respectively.
  4. Run the cells under Authentication.
  5. Run the cells under Test requests. As you will access different models, the service will return objects it has detected in the image in response to each request.

Delete the resources you created

When deploying and using models, you pay for the uptime of each node instance: from its start to deletion.

If you no longer need the service you deployed, delete the node.

  1. Delete the node.
  2. Delete the secrets.
  3. Delete the authorized key of the service account.
  4. Delete the Docker image and Container Registry registry.
Previous
Creating an MLFlow server for logging experiments and artifacts
Next
Deploying a service based on a Docker image with FastAPI