Status monitoring of geographically distributed devices
- Getting started
- Prepare your cloud
- Create a cluster in Managed Service for PostgreSQL
- Create the required Yandex IoT Core resources
- Create a device emulator in Cloud Functions
- Create a function for processing data in Cloud Functions .
- Configure monitoring in DataLens
- How to delete the resources you created
In this tutorial, you will configure status monitoring for devices (e.g., vending machines) connected to Yandex IoT Core and located at various sites throughout the city. You can monitor the status of vending machines on the Yandex DataLens map and charts. To emulate sensors, we will use Yandex Cloud Functions. If you have any connected sensors, use them. Yandex Managed Service for PostgreSQL will be used to store data.
The source code used in this tutorial is available on GitHub
To configure monitoring of sensor readings:
- Prepare your cloud.
- Create a Managed Service for PostgreSQL cluster.
- Create the required Yandex IoT Core resources.
- Create a device emulator in Cloud Functions.
- Create a function for processing data in Cloud Functions.
- Configure monitoring in DataLens.
If you no longer need the resources you created, delete them.
Getting started
Sign up for Yandex Cloud and create a billing account:
- Go to the management console
and log in to Yandex Cloud or create an account if you do not have one yet. - On the Yandex Cloud Billing
page, make sure you have a billing account linked and it has theACTIVE
orTRIAL_ACTIVE
status. If you do not have a billing account, create one.
If you have an active billing account, you can go to the cloud page
Learn more about clouds and folders.
Required paid resources
The support cost includes:
- Fee for using a public IP address (see Yandex Virtual Private Cloud pricing).
- Fee for the number of messages (see Yandex IoT Core pricing).
- Fee for the number of Cloud Functions function calls (see Yandex Cloud Functions pricing).
- Fee for the Managed Service for PostgreSQL cluster computing resources and storage (see Yandex Managed Service for PostgreSQL pricing).
Prepare your cloud
-
Create a service account named
my-emulator-function-service-account
for data sending. Assign thefunctions.functionInvoker
,iot.devices.writer
, andviewer
roles to it. -
Create a service account named
my-db-function-service-account
for data processing. Assign thefunctions.functionInvoker
andeditor
roles to it. -
Create a cloud network and at least one subnet in the
ru-central1-a
,ru-central1-b
, orru-central1-d
availability zone. -
Create a security group named
my-pg-sg
and add a rule for incoming traffic:- Port range:
6432
. - Protocol:
TCP
. - Source:
CIDR
. - CIDR blocks:
0.0.0.0/0
.
- Port range:
Create a cluster in Managed Service for PostgreSQL
The example is based on the minimum values of host parameters. For real tasks, it is better to choose hosts with a guaranteed 100% vCPU share.
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select Managed Service for PostgreSQL.
-
Click Create cluster.
-
Specify a name for the cluster, e.g.,
my-pg-database
. -
In the Environment list, select
PRODUCTION
. -
In the Version list, select
15
. -
Under Host class, specify:
- Platform:
Intel Ice Lake
. - Type:
standard
. - Class:
s3-c2-m8 (2 vCPU, 8 GB)
.
- Platform:
-
Under Size of storage:
- Select
network-hdd
. - Specify the size of
10 GB
.
- Select
-
Under Database, specify:
-
DB name:
db1
. -
Username:
user1
. -
Password: Set a password to access the database.
Note
Save the password as you will need it for authentication.
-
Locale for sorting (LC_COLLATE):
C
. -
Charset locale (LC_CTYPE):
C
.
-
-
Under Network settings:
- Select the cloud network where you created the
my-pg-sg
security group. - Select the
my-pg-sg
security group.
- Select the cloud network where you created the
-
Under Hosts:
- Leave a single host.
- Click
. - In the window that opens, enable Public access and click Save.
-
Under Advanced settings, enable DataLens access and Access from the management console.
-
Click Create cluster.
It may take a few minutes to create a cluster.
Create the required Yandex IoT Core resources
Registry and device are the main components of the Yandex IoT Core service used for exchanging data and commands. Devices can only exchange data if created in the same registry.
Create a registry and configure authorization with your login and password
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select IoT Core.
-
Click Create registry.
-
In the Name field, enter a name for the registry, e.g.,
my-registry
. -
In the Password field, set a password to access the registry. The password must be at least 14 characters long and contain lowercase letters, capital letters, and numbers. To create a password, you can use the password generator
.Note
Save the password as you will need it for authentication.
-
Click Create.
You can also complete authorization using certificates. Learn more about authorization in Yandex IoT Core.
Create devices and configure authorization with your login and password
Create three devices: my-device-1
, my-device-2
, and my-device-3
.
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select IoT Core.
-
Select the registry you created in the previous step.
-
In the left-hand menu, select Devices.
-
Click Add device.
-
In the Name field, enter the device name:
my-device-1
. -
In the Password field, set a password to access the device. The password must be at least 14 characters long and contain lowercase letters, capital letters, and numbers. To create a password, you can use the password generator
.Note
Save the password as you will need it for authentication.
-
Optionally, add an alias:
-
Click Add alias.
-
Fill in the fields: enter an alias, e.g.,
events
, and the topic type after$devices/{id}
, e.g.,events
.As the result, you will be able to use the
events
alias instead of the$devices/<device_ID>/events
topic. -
Repeat the steps for each alias you add.
-
-
Click Create.
-
Similarly, create
my-device-2
andmy-device-3
.
You can also complete authorization using certificates. Learn more about authorization in Yandex IoT Core.
Create a device emulator in Cloud Functions
The emulator will send data from the devices to the Managed Service for PostgreSQL cluster.
To run the emulator, create a function to emulate sending data from device sensors and a trigger to call this function once per minute.
Create a function that emulates transmitting data from the device
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select Cloud Functions.
-
Create a function:
- Click Create function.
- In the Name field, enter a name for the function, e.g.,
my-device-emulator-function
. - Click Create.
-
Create a function version:
-
In the Editor window that opens, select
Node.js 18
. -
Disable the Add files with code examples option.
-
Click Continue.
-
In the Method field, select
Code editor
. -
Click Create file:
- File name:
device-emulator.js
. - File contents: Function code from GitHub
.
- File name:
-
Similarly, create a file named
package.json
with the following content:{ "name": "my-app", "version": "1.0.0", "dependencies": { "yandex-cloud": "*" } }
-
In the Entry point field, specify
device-emulator.handler
. -
Under Parameters, specify:
-
Timeout, sec:
10
-
Memory:
128 MB
-
Service account:
my-emulator-function-service-account
-
Environment variables:
Key Description Value CASH_DRAWER_SENSOR_VALUE
Percentage of used cash box space 67.89
TEMPERATURE_SENSOR_VALUE
Baseline temperature in the pickup box 10.34
POWER_SENSOR_VALUE
Baseline mains voltage 24.12
SERVICE_DOOR_SENSOR_VALUE
State of the open service door sensor False
ITEM1_SENSOR_VALUE
Stock of product 1 50.65
ITEM2_SENSOR_VALUE
Stock of product 2 80.97
ITEM3_SENSOR_VALUE
Stock of product 3 30.33
ITEM4_SENSOR_VALUE
Stock of product 4 15.15
REGISTRY_ID
ID of the registry you created See the management console
in Yandex IoT Core
-
-
Click Save changes.
-
Test the emulation function
-
(Optional) To get detailed information from the sensors, subscribe your registry to the Yandex IoT Core device topic.
CLIIf you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the
--folder-name
or--folder-id
parameter.Run this command:
yc iot mqtt subscribe \ --username <registry_ID> \ --password <registry_password> \ --topic '$devices/<device_ID>/events/#' \ --qos 1
Where:
--username
and--password
: Credentials for authentication with a username and password.--topic
: Device topic for sending data or topic alias.--qos
: Quality of service (QoS).
The command should run without interruption until you finish testing the function.
For more information about the
yc iot mqtt subscribe
command, see the CLI reference. -
Test the emulation function:
Management console-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select Cloud Functions.
-
Select the
my-device-emulator-function
function. -
In the left-hand menu, select Testing.
-
In the Version tag list, select
$latest
, the most recent version of the function. -
Click Run test.
If the function is successful, the Done status will be displayed in the Function status field and the result will appear in the Function output field:
{ "statusCode" : 200 }
If you subscribed to the Yandex IoT Core device topic before running the test, you will get the following JSON object in the terminal:
{ "DeviceId":"areulpu7dnou********", "TimeStamp":"2024-06-19T08:29:14Z", "Values":[ {"Type":"Bool","Name":"Service door sensor","Value":"False"}, {"Type":"Float","Name":"Power Voltage","Value":"24.91"}, {"Type":"Float","Name":"Temperature","Value":"10.51"}, {"Type":"Float","Name":"Cash drawer fullness","Value":"67.89"}, {"Items":[ {"Type":"Float", "Id":"1","Name":"Item 1","Fullness":"50.65"}, {"Type":"Float", "Id":"2","Name":"Item 2","Fullness":"80.97"}, {"Type":"Float", "Id":"3","Name":"Item 3","Fullness":"30.33"}, {"Type":"Float", "Id":"4","Name":"Item 4","Fullness":"15.15"} ]} ] }
-
Create a trigger to call the emulation function once per minute
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select Cloud Functions.
-
In the left-hand panel, select Triggers.
-
Click Create trigger.
-
In the Name field, enter a name for the trigger. For example,
my-emulator-function-trigger
. -
In the Type field, select
Timer
. -
In the Cron expression field, select
Every minute
. -
Under Function settings, specify the function parameters:
- Function:
my-device-emulator-function
. - Function version tag:
$latest
. - Service account:
my-emulator-function-service-account
.
- Function:
-
(Optional) In the Repeat request settings and Dead Letter Queue settings sections, set parameters to prevent data loss in case of failure:
-
Repeat request settings allow you to repeatedly call the function if the current function call fails.
-
Dead Letter Queue settings allow you to redirect messages that the trigger sends to the function.
You can use a standard message queue as a dead-letter queue. If you do not have a message queue, create one in Yandex Message Queue.
-
-
Click Create trigger.
Create a function for processing data in Cloud Functions .
Create a data processing function, test it, and review the result of data processing. Then create a trigger to call the function and see the result it returns.
Create a function for processing received data
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select Cloud Functions.
-
In the left-hand panel, select Functions.
-
Create a function:
- Click Create function.
- In the Name field, enter a name for the function, e.g.,
my-db-function
. - Click Create.
-
Create a function version:
-
In the Editor window that opens, select
Python 3.12
. -
Disable the Add files with code examples option.
-
Click Continue.
-
In the Method field, select
Code editor
. -
Click Create file:
- File name:
myfunction.py
. - File contents: Function code from GitHub
.
- File name:
-
In the Entry point field, specify
myfunction.msgHandler
. -
Under Parameters, specify:
-
Timeout, sec:
10
-
Memory:
128 MB
-
Service account:
my-db-function-service-account
-
Environment variables:
Key Description Value VERBOSE_LOG
Enables and disables writing data True
DB_HOSTNAME
Host FQDN in Managed Service for PostgreSQL See the management console
in Managed Service for PostgreSQLDB_PORT
Port to connect to the cluster in Managed Service for PostgreSQL 6432
DB_NAME
Cluster name in Managed Service for PostgreSQL db1
DB_USER
Username to connect to the cluster in Managed Service for PostgreSQL user1
DB_PASSWORD
Password to connect to the DB in Managed Service for PostgreSQL Password you set in Managed Service for PostgreSQL
-
-
Click Save changes.
-
Test the data processing function
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select Cloud Functions.
-
Select the
my-db-function
function. -
In the left-hand menu, select Testing.
-
In the Version tag list, select
$latest
, the most recent version of the function. -
In the Payload field, paste the following data:
{ "messages": [ { "event_metadata": { "event_id": "160d239876d9714800", "event_type": "yandex.cloud.events.iot.IoTMessage", "created_at": "2020-05-08T19:16:21.267616072Z", "folder_id": "b112345678910" }, "details": { "registry_id": "are1234567890", "device_id": "are0987654321", "mqtt_topic": "$devices/are0987654321/events", "payload": "ewogICAgICAgICAgICAiRGV2aWNlSWQiOiJhcmU1NzBrZTA1N29pcjg1bDlmciIsCiAgICAgICAgICAgICJUaW1lU3RhbXAiOiIyMDIwLTA2LTExVDExOjA3OjIwWiIsCiAgICAgICAgICAgICJWYWx1ZXMiOlsKICAgICAgICAgICAgICAgIHsiVHlwZSI6IkJvb2wiLCJOYW1lIjoiU2VydmljZSBkb29yIHNlbnNvciIsIlZhbHVlIjoiRmFsc2UifSwKICAgICAgICAgICAgICAgIHsiVHlwZSI6IkZsb2F0IiwiTmFtZSI6IlBvd2VyIFZvbHRhZ2UiLCJWYWx1ZSI6IjI1LjA2In0sCiAgICAgICAgICAgICAgICB7IlR5cGUiOiJGbG9hdCIsIk5hbWUiOiJUZW1wZXJhdHVyZSIsIlZhbHVlIjoiMTEuMjEifSwKICAgICAgICAgICAgICAgIHsiVHlwZSI6IkZsb2F0IiwiTmFtZSI6IkNhc2ggZHJhd2VyIGZ1bGxuZXNzIiwiVmFsdWUiOiI2Ny44OSJ9LAogICAgICAgICAgICAgICAgeyJJdGVtcyI6WwogICAgICAgICAgICAgICAgICAgIHsiVHlwZSI6IkZsb2F0IiwgIklkIjoiMSIsIk5hbWUiOiJJdGVtIDEiLCJGdWxsbmVzcyI6IjUwLjY1In0sCiAgICAgICAgICAgICAgICAgICAgeyJUeXBlIjoiRmxvYXQiLCAiSWQiOiIyIiwiTmFtZSI6Ikl0ZW0gMiIsIkZ1bGxuZXNzIjoiODAuOTcifSwKICAgICAgICAgICAgICAgICAgICB7IlR5cGUiOiJGbG9hdCIsICJJZCI6IjMiLCJOYW1lIjoiSXRlbSAzIiwiRnVsbG5lc3MiOiIzMC4zMyJ9LAogICAgICAgICAgICAgICAgICAgIHsiVHlwZSI6IkZsb2F0IiwgIklkIjoiNCIsIk5hbWUiOiJJdGVtIDQiLCJGdWxsbmVzcyI6IjE1LjE1In0KICAgICAgICAgICAgICAgIF19CiAgICAgICAgICAgICAgICBdCiAgICAgICAgICAgIH0=" } }] }
-
Click Run test.
If the function is successful, you will see the Done status in the Function status field and the Function output field will display this result:
{ "statusCode" : 200 , "headers" : { "Content-Type" : "text/plain" }, "isBase64Encoded" : false }
View the data processing result in Managed Service for PostgreSQL
- In the management console
, select the folder you are using to complete this tutorial. - In the list of services, select Managed Service for PostgreSQL.
- Select the
my-pg-database
cluster. - In the left-hand menu, select SQL.
- In the Password field, enter the password you set to access the database and click Connect.
- Select the
public
folder. - View the data processing result in the
iot_events
andiot_position
tables.
Create a trigger to call the data processing function
The trigger will invoke the function when messages appear in the device topic.
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select Cloud Functions.
-
In the left-hand panel, select Triggers.
-
Click Create trigger.
-
In the Name field, enter a name for the trigger. For example,
my-db-func-trigger
. -
In the Type field, select
IoT Core (device)
. -
Under IoT Core message settings, specify the registry and device parameters:
- Registry:
my-registry
. - Device:
Any device
- MQTT topic:
$devices/#
.
- Registry:
-
Under Function settings, specify the function parameters:
- Function:
my-db-function
. - Function version tag:
$latest
. - Service account:
my-db-function-service-account
.
- Function:
-
(Optional) In the Repeat request settings and Dead Letter Queue settings sections, set parameters to prevent data loss in case of failure:
-
Repeat request settings allow you to repeatedly call the function if the current function call fails.
-
Dead Letter Queue settings allow you to redirect messages that the trigger sends to the function.
You can use a standard message queue as a dead-letter queue. If you do not have a message queue, create one in Yandex Message Queue.
-
-
Click Create trigger.
View the trigger result in Managed Service for PostgreSQL
Shortly after creating the trigger, you can check how it works.
- In the management console
, select the folder you are using to complete this tutorial. - In the list of services, select Managed Service for PostgreSQL.
- Select the
my-pg-database
cluster. - In the left-hand menu, select SQL.
- In the Password field, specify the password you set to access the database and click Connect.
- Select the
public
folder. - View the changes in the
iot_events
table.
Configure monitoring in DataLens
To monitor the state of your devices, configure a connection, create a dataset, charts, and a dashboard.
Set up a connection to Managed Service for PostgreSQL
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select DataLens.
-
Click Create connection.
-
Select the PostgreSQL connector.
When connecting to an external data source (which is not a Yandex Cloud resource), grant access to the source for the DataLens IP address ranges.
-
Select a cloud and folder.
-
Select the
my-pg-database
cluster and a host. -
In the Port field, specify
6432
. -
Select
db1
as the database anduser1
as the user. -
In the Password field, specify the password you set to access the database in Managed Service for PostgreSQL.
-
Click Create connection.
-
In the window that opens, specify the
MyPGConnection
name for the connection and click Create.
Create a dataset
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select DataLens.
-
Click Create dataset.
-
In the left-hand panel, click
Add. -
Select a
MyPGConnection
connection. -
In the left-hand menu, select the
public.iot_events
andpublic.iot_position
tables and drag them to the right. -
Click Save.
-
In the window that opens, specify
My-pg-dataset
and click Create.The dataset will appear in the list.
Create a chart for temperature and mains voltage
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select DataLens.
-
Click Create chart.
-
In the left-hand panel, click
Select dataset and select theMy-pg-dataset
you created previously. -
Select Line chart as your chart type.
-
From the Dimensions section in the left-hand menu, drag the dimensions to the Line chart section as follows:
-
event_datetime
, to the X section.A timeline will appear in the lower part of the chart on the X-axis.
-
temperature
andpower_voltage
, to the Y section.
The temperature and mains voltage values will appear on the Y-axis of the chart.
-
-
Click Save.
-
In the window that opens, specify the chart name or use the generated
My-pg-dataset — Line chart
name and click Save.
Create a chart with a map
- In the management console
, select the folder you are using to complete this tutorial. - In the list of services, select DataLens.
- Create a field to define device coordinates:
- In the left-hand menu, select Datasets.
- Select
My-pg-dataset
from the list of datasets. - Go to the Fields tab.
- In the right part of the window, click
Add field. - In the window that opens, in the Field name field, enter
Position
. - In the Formula field, insert
GEOPOINT([latitude],[longitude])
. - Click Create.
- Click Save.
- Create a chart:
- At the top right, click Create chart.
- Select
Map
as the chart type. - From the Dimensions section in the left-hand menu, drag the dimensions to the Map section as follows:
-
Position
, to the Geopoints section. -
item1_fullness
,item2_fullness
,item3_fullness
,item4_fullness
, andcash_drawer
, to the Tooltips section.On the right side of the window, you will see a scalable map with vending machines displayed as dots on the map and tooltips displayed as legend lines when you hover over the dots.
-
- Click Save.
- In the window that opens, specify the chart name or use the generated
My-pg-dataset — Map
name and click Save.
Create a dashboard
-
In the management console
, select the folder you are using to complete this tutorial. -
In the list of services, select DataLens.
-
Click Create dashboard.
-
Add the
My-pg-dataset — Map
andMy-pg-dataset — Line chart
charts, which you created at the previous steps, to the dashboard:- In the bottom panel, click Chart.
- In the Chart field of the window that opens, click Select and select
My-pg-dataset — Map
. - Click Add.
- Repeat the steps to add
My-pg-dataset — Line chart
.
-
Configure the selector:
- In the bottom panel, click Selector.
- In the Dataset field, click Select and select
My-pg-dataset
. - In the Field list, select
device_id
. - In the Default value list, select the ID of any one device you created in Yandex IoT Core.
- In the Header field, enter
Device
. - Click Add.
-
Configure links:
-
At the top right, click Links.
-
In the window that opens, select the
My-pg-dataset — Map
chart from the list. -
In the In.link drop-down list, select
Ignore
.Maps do not support the Links parameter.
-
In the list, select
My-pg-dataset — Line chart
. -
In the In.link drop-down list, select
In.link
. -
Click Save.
-
-
Click Save.
-
In the window that opens, enter a name for the dashboard:
MyDash
. -
Click Create.
After completing the tutorial, you will be able to monitor sensor readings on a map and chart.
How to delete the resources you created
To stop paying for the resources you created: