Deploying a gRPC service based on a Docker image
In this tutorial, you will deploy a gRPC echo service based on a Docker image. The service provides metrics in Prometheus9875
.
To deploy the service based on a gRPC node:
- Prepare your infrastructure.
- Prepare a Docker image for the service deployment.
- Deploy the service in DataSphere.
- Run a health check for the service you deployed.
If you no longer need the resources you created, delete them.
Getting started
Before getting started, register in Yandex Cloud, set up a community, and link your billing account to it.
- On the DataSphere home page
, click Try for free and select an account to log in with: Yandex ID or your working account in the identity federation (SSO). - Select the Yandex Cloud Organization organization you are going to use in Yandex Cloud.
- Create a community.
- Link your billing account to the DataSphere community you are going to work in. Make sure that you have a billing account linked and its status is
ACTIVE
orTRIAL_ACTIVE
. If you do not have a billing account yet, create one in the DataSphere interface.
Required paid resources
The cost of deploying a service based on a Docker image includes:
- Fee for continuously running node instances (see DataSphere pricing).
- Fee for running code cells for health checks of the deployed service.
- Amount of storage used by Yandex Container Registry data (see Container Registry pricing).
Prepare the infrastructure
Log in to the Yandex Cloud management console
If you have an active billing account, you can create or select a folder to deploy your infrastructure in, on the cloud page
Note
If you use an identity federation to access Yandex Cloud, billing details might be unavailable to you. In this case, contact your Yandex Cloud organization administrator.
Create a folder
Note
In our example, both the Yandex Cloud infrastructure and the deployed service operate from the same Yandex Cloud folder; however, this is not a requirement.
- In the management console
, select a cloud and click Create folder. - Give your folder a name, e.g.,
data-folder
. - Click Create.
Create a registry in Container Registry
DataSphere can create a node from a Docker image hosted in Yandex Container Registry. To push an image, create a registry.
- Go to the
data-folder
folder. - In the list of services, select Container Registry.
- Click Create registry.
- Specify a name for the registry, e.g.,
datasphere-registry
, and click Create registry.
Create a service account for the DataSphere project
-
Go to the
data-folder
folder. -
In the Service accounts tab, click Create service account.
-
Enter a name for the service account, e.g.,
sa-for-datasphere
. -
Click Add role and assign the following roles to the service account:
container-registry.images.puller
to allow DataSphere to pull your Docker image for creating a node.vpc.user
to use the DataSphere network.datasphere.user
to send requests to the node.
-
Click Create.
Create an authorized key for a service account
To allow your service account to get authenticated in Yandex Container Registry, create an authorized key.
Note
Authorized keys do not expire, but you can always get new authorized keys and authenticate again if something goes wrong.
-
Go to the
data-folder
folder. -
At the top of the screen, go to the Service accounts tab.
-
Choose the
sa-for-datasphere
service account and click the line with its name. -
Click Create new key in the top panel.
-
Select Create authorized key.
-
Select the encryption algorithm.
-
Save both the public and private keys. The private key is not saved in Yandex Cloud, and you will not be able to view the public key in the management console.
Tip
You can save the file with the key on your computer. You will need its contents later when creating a secret to access DataSphere in Container Registry.
Prepare your Docker image for the service deployment
If you do not have Docker yet, install
Create a Docker image
-
Create a folder to store the configuration of the Docker image and all the required files, e.g.,
/home/docker-images
. -
Clone the repository on GitHub
and place the files in the folder you created. -
Run Docker Desktop
: -
In the command shell, navigate to the folder with
Dockerfile
you created.cd docker-images
-
Build the Docker image:
docker build --platform linux/amd64 -t grpc-docker .
Push the Docker image to Container Registry
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
-
Set
data-folder
as your default folder:yc config set folder-id <folder_ID>
-
Get authenticated in Container Registry.
- Issue an IAM token for your service account.
yc iam create-token
The response will contain the IAM token. If you are authenticating using a federated account, the CLI will redirect you to the management console to authenticate and then send you an IAM token.
Note
The IAM token has a short lifetime: no more than 12 hours. This makes it a good method for applications that automatically request an IAM token.
- Run the command with the token value you got in the previous step in place of
<IAM token>
:
docker login \ --username iam \ --password <IAM token> \ cr.yandex
-
Get a list of registries in
data-folder
:yc container registry list
You will need the registry ID at the next step. Command output example:
+----------------------+---------------------+----------------------+ | ID | NAME | FOLDER ID | +----------------------+---------------------+----------------------+ | crp86bmgl1da******** | datasphere-registry | b1g4bh24c406******** | +----------------------+---------------------+----------------------+
-
Push the Docker image to Container Registry by substituting your
datasphere-registry
ID:docker tag grpc-docker cr.yandex/<registry_ID>/grpc:v1 docker push cr.yandex/<registry_ID>/grpc:v1
Deploy the service in DataSphere
-
Open the DataSphere home page
. In the left-hand panel, select Communities. - Select a community with a billing account linked.
- Create a project named
Node from Docker
. - In the project settings, specify:
- Default folder:
data-folder
. - Service account:
sa-for-datasphere
- Default folder:
- Create a secret named
key-for-sa
to store the full contents of the authorized key file for thesa-for-datasphere
service account. - Create a node. To do this, click Create resource in the top-right corner of the project page. In the pop-up window, select Node. Specify the node settings:
- In the Name field, enter the name of the node:
grpc
. - Under Docker image:
- Type: Select Docker.
- Docker image storage: Select Yandex Container Registry.
- Image path: Specify the path to the Container Registry image in the
cr.yandex/<registry_ID>/<image_name>:<tag>
format. You can get it in the management console by copying the full value on the repository page. You can also fill out this field manually. You can get the registry ID in the CLI by running theyc container registry list
command. - Password secret: Select
key-for-sa
.
- Under Endpoint:
- Type: Select gRPC.
- Port: 9875.
- Enable Telemetry and specify:
- Type: Select Prometheus.
- HTTP path:
/
. - Port: 9875.
- Enable Healthcheck and specify:
- Type: Select gRPC.
- Path:
echo.v1.Echo
. - Port: 9875.
- Timeout: 1
- Interval: 15.
- Fails threshold: 3.
- Passes threshold: 3.
- Under Folder, select
data-folder
. - Under Provisioning, select the
g1.1
configuration. - Click Create.
- In the Name field, enter the name of the node:
Run a health check for the service you deployed
-
Select the relevant project in your community or on the DataSphere homepage
in the Recent projects tab. -
In the list of services, select Node.
-
Click the
grpc
node you created. -
Go to the Request tab.
-
In the Path field, enter
echo.v1.Echo/Reflect
. -
In the input field, enter
{"message": "Hello, world!"}
. -
Run the command you got in the terminal by supplementing the IAM token value.
Result:
{ "message": "Hello, world!" }
The client.py
script makes a request to the node. To run it and check the service's operation, open the terminal in the folder containing the downloaded repository and run the following commands:
-
Create and activate a virtual Python environment:
python -m venv venv && source venv/bin/activate
-
Set the required dependencies:
pip install -r requirements.txt
-
Generate the code of your gRPC services from the proto files:
python -m grpc_tools.protoc -I ./proto --python_out=. --grpc_python_out=. ./proto/echo/v1/echo.proto
-
Specify environment variables by inserting the required values:
export NODE_IAM=<IAM_token> export FOLDER_ID=<folder_ID> export NODE_ID=<node_ID>
-
Run the script:
python client.py
Result:
Echo message: Hello, world!
How to delete the resources you created
When deploying and using models, you pay for the uptime of each node instance: from its start to deletion.
If you no longer need the service you deployed, delete the node.