Deploying a service based on a Docker image
DataSphere enables you to deploy and run services based on any Docker image.
In this tutorial, you will use a node built from a Docker image to deploy an object detection service based on NVIDIA Triton Inference Server
- Prepare your infrastructure.
- Prepare a Docker image for the service deployment.
- Deploy the service in DataSphere.
- Run a health check for the service you deployed.
If you no longer need the resources you created, delete them.
Getting started
Before getting started, register in Yandex Cloud, set up a community, and link your billing account to it.
- On the DataSphere home page
, click Try for free and select an account to log in with: Yandex ID or your working account in the identity federation (SSO). - Select the Yandex Cloud Organization organization you are going to use in Yandex Cloud.
- Create a community.
- Link your billing account to the DataSphere community you are going to work in. Make sure that you have a billing account linked and its status is
ACTIVE
orTRIAL_ACTIVE
. If you do not have a billing account yet, create one in the DataSphere interface.
Required paid resources
The cost of deploying a service based on a Docker image includes:
- Fee for continuously running node instances (see DataSphere pricing).
- Fee for running code cells for health checks of the deployed service.
- Amount of storage used by Yandex Container Registry data (see Container Registry pricing).
Prepare the infrastructure
Log in to the Yandex Cloud management console
If you have an active billing account, you can create or select a folder to deploy your infrastructure in, on the cloud page
Note
If you use an identity federation to access Yandex Cloud, billing details might be unavailable to you. In this case, contact your Yandex Cloud organization administrator.
Create a folder
Create a folder for you to deploy your infrastructure and for your service to store the logs.
Note
In our example, both the Yandex Cloud infrastructure and the deployed service operate from the same Yandex Cloud folder; however, this is not a requirement.
- In the management console
, select a cloud and click Create folder. - Name your folder, e.g.,
data-folder
. - Click Create.
Create a registry in Container Registry
DataSphere can create a node from a Docker image hosted in Yandex Container Registry. To push an image, create a registry.
- Go to
data-folder
. - In the list of services, select Container Registry.
- Click Create registry.
- Specify a name for the registry, e.g.,
datasphere-registry
, and click Create registry.
Create a service account for the DataSphere project
-
Go to
data-folder
. -
In the Service accounts tab, click Create service account.
-
Enter a name for the service account, e.g.,
sa-for-datasphere
. -
Click Add role and assign the following roles to the service account:
container-registry.images.puller
to allow DataSphere to pull your Docker image to create a node.vpc.user
to use the DataSphere network.- (Optional)
datasphere.user
to send requests to the node.
-
Click Create.
Create an authorized key for a service account
To allow your service account to get authenticated in Yandex Container Registry, create an authorized key.
Note
Authorized keys do not expire, but you can always get new authorized keys and authenticate again if something goes wrong.
-
Go to
data-folder
. -
At the top of the screen, go to the Service accounts tab.
-
Select the
sa-for-datasphere
service account and click the row with its name. -
Click Create new key in the top panel.
-
Select Create authorized key.
-
Select the encryption algorithm.
-
Enter a description of the key so that you can easily find it in the management console.
-
Save both the public and private keys. The private key is not saved in Yandex Cloud, and you will not be able to view the public key in the management console.
Tip
You can save the file with the key on your computer. You will need its contents later when creating a secret to access DataSphere in Container Registry.
Prepare your Docker image for the service deployment
If you do not have Docker yet, install
Create a Docker image for your service
-
Create a folder to store the configuration of your Docker image, e.g.,
/home/docker-images
. -
Create and save a text file named
Dockerfile
(without any extension).Triton Dockerfile
# syntax=docker/dockerfile:1 FROM nvcr.io/nvidia/tritonserver:22.01-py3 RUN mkdir -p /models/resnet152_640x640/1/model.savedmodel/ &&\ curl --location "https://tfhub.dev/tensorflow/faster_rcnn/resnet152_v1_640x640/1?tf-hub-format=compressed" |\ tar -zxvC /models/resnet152_640x640/1/model.savedmodel/ &&\ mkdir -p /models/inception_resnet_v2_640x640/1/model.savedmodel/ &&\ curl --location "https://tfhub.dev/tensorflow/faster_rcnn/inception_resnet_v2_640x640/1?tf-hub-format=compressed" |\ tar -zxvC /models/inception_resnet_v2_640x640/1/model.savedmodel/ ENTRYPOINT ["/opt/tritonserver/nvidia_entrypoint.sh",\ "tritonserver",\ "--model-repository=/models",\ "--strict-model-config=false"]
-
Run Docker Desktop
. -
In the command shell, navigate to the folder with
Dockerfile
you created:cd docker-images
-
Build the Docker image:
docker build -t triton-docker --platform linux/amd64 .
Push the Docker image to Container Registry
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
-
Set
data-folder
as your default folder:yc config set folder-name data-folder
-
Get authenticated in Container Registry.
- Issue an IAM token for your service account:
yc iam create-token
The response will contain the IAM token. If you are authenticating using a federated account, the CLI will redirect you to the management console to authenticate and then send you an IAM token.
Note
The IAM token has a short lifetime: no more than 12 hours. This makes it a good method for applications that automatically request an IAM token.
- Run the command with the token value you got in the previous step in place of
<IAM_token>
:
docker login \ --username iam \ --password <IAM_token> \ cr.yandex
-
Get a list of registries in
data-folder
:yc container registry list
You will need the registry ID at the next step. Command output example:
+----------------------+---------------------+----------------------+ | ID | NAME | FOLDER ID | +----------------------+---------------------+----------------------+ | crp86bmgl1da******** | datasphere-registry | b1g4bh24c406******** | +----------------------+---------------------+----------------------+
-
Push the Docker image to Container Registry. Instead of
<registry_ID>
, use the ID of your registry (datasphere-registry
):docker tag triton-docker cr.yandex/<registry_ID>/triton:v1 docker push cr.yandex/<registry_ID>/triton:v1
Deploy the service in DataSphere
-
Open the DataSphere home page
. In the left-hand panel, select Communities. - Select a community with a billing account linked.
- Create a project named
Node from Docker
. - In the project settings, specify:
- Default folder:
data-folder
. - Service account:
sa-for-datasphere
.
- Default folder:
- Create a secret named
iam-secret
including an IAM token of your user account. - Create a secret named
key-for-sa
to store the full contents of the authorized key file for thesa-for-datasphere
service account. - Create a node. To do this, click Create resource in the top-right corner of the project page. In the pop-up window, select Node. Specify the node settings:
- Enter
triton
as the node name in the Name field. - Under Type:
- Type: Select Docker.
- Docker image storage: Select Yandex Container Registry.
- Image path: Specify the path to the Container Registry image in the
cr.yandex/<registry_ID>/<image_name>:<tag>
format. You can get it in the management console by copying the full value on the repository page. You can also fill out this field manually. You can get the registry ID in the CLI by running theyc container registry list
command. - Password secret: Select
key-for-sa
.
- Under Endpoint:
- Type: Select HTTP.
- Port: 8000.
- Enable Telemetry and specify:
- Type: Select Prometheus.
- HTTP path:
/metrics
. - Port: 8000.
- Enable Healthcheck and specify:
- Type: Select HTTP.
- Path:
/v2/health/ready
. - Port: 8000.
- Timeout: 1.
- Interval: 20.
- Fails threshold: 3.
- Passes threshold: 3.
- Under Folder, select
data-folder
. - Under Provisioning select the
g1.1
configuration. - Click Create.
- Enter
Run a health check for the service you deployed
- Download a notebook
with the health check code and upload it to theNode from Docker
project's JupyterLab. - Run the cells in the Preparing environment section: select the cells and press Shift + Enter.
- Under Authentication, fill out the details to get authenticated in the node. Replace
<node_ID>
and<folder_ID>
withtriton
anddata-folder
, respectively. - Run the cells under Authentication.
- Run the cells under Test requests. As you will access different models, the service will return objects it has detected in the image in response to each request.
Delete the resources you created
When deploying and using models, you pay for the uptime of each node instance: from its start to deletion.
If you no longer need the service you deployed, delete the node.
- Delete the node.
- Delete the secrets.
- Delete the authorized key of the service account.
- Delete the Docker image and Container Registry registry.