Developing CRUD APIs for movie services
Using serverless technology, you can create CRUD
The implementation of CRUD APIs employs the Yandex Serverless Containers container, which is designed for a movie database deployed in Yandex Managed Service for YDB.
The container is configured in the Yandex API Gateway API gateway specifications supporting OpenAPI 3.0
The container interacts with Managed Service for YDB and processes external HTTP requests via the API gateway using the Amazon DynamoDB
To deploy a project:
- Configure the environment.
- Initiate Terraform.
- Create a Managed Service for YDB database.
- Run CRUD operations.
- Develop the REST API.
- Check the performance of the CRUD API.
If you no longer need the resources you created, delete them.
Getting started
Sign up for Yandex Cloud and create a billing account:
- Go to the management console
and log in to Yandex Cloud or create an account if you do not have one yet. - On the Yandex Cloud Billing
page, make sure you have a billing account linked and it has theACTIVE
orTRIAL_ACTIVE
status. If you do not have a billing account, create one.
If you have an active billing account, you can go to the cloud page
Learn more about clouds and folders.
Required paid resources
The cost of CRUD API resources includes:
- Fee for YDB operations and data storage (see Managed Service for YDB pricing for serverless mode).
- Fee for the number of container calls, computing resources allocated to execute the application, and outgoing traffic (see Serverless Containers pricing).
- Fee for the number of requests to the API gateway and outgoing traffic (see API Gateway pricing).
Configure the environment
- Install the WSL
utility to run a Linux environment. - Run the Linux subsystem (by default, Ubuntu).
- Configure the environment as described in the Linux manual.
Note
If you use a distribution other than Ubuntu, install the specified utilities using your package manager commands.
-
Install the following utilities in the specified order using commands in the terminal:
-
sudo apt-get install curl git -y
-
WebStorm
or any other development environment that supports TypeScript :sudo snap install webstorm --classic
-
Node.js
16.9.1
or higher:curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash sudo apt-get install nodejs node -v npm -v
-
sudo npm install -g typescript
-
curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bash exec -l $SHELL yc version
-
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip sudo ./aws/install
-
sudo apt-get update sudo apt-get install apt-transport-https ca-certificates curl gnupg lsb-release -y curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io -y sudo docker run hello-world
-
-
Install Terraform
1.0.8
or higher. -
Create a Yandex Cloud CLI profile with basic parameters.
-
Set up the AWS CLI.
-
Set up
Docker management on behalf of a user without privileges.sudo groupadd docker sudo usermod -aG docker $USER newgrp docker docker run hello-world
-
Install the following utilities in the specified order using commands in the terminal:
-
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
-
brew install curl git
-
WebStorm
or any other development environment that supports TypeScript :brew install --cask webstorm
-
Node.js
16.9.1
or higher:brew install node node -v npm -v
-
npm install -g typescript
-
curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bash exec -l $SHELL yc version
-
curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg" sudo installer -pkg AWSCLIV2.pkg -target /
-
brew install --cask docker
-
-
Install Terraform
1.0.8
or higher. -
Create a profile with basic parameters.
-
Set up the AWS CLI.
Initiate Terraform
-
Clone a repository with source files for the CRUD API project:
git clone https://github.com/yandex-cloud-examples/yc-practicum-serverless-web-application-movie-website.git
Open the folder project in WebStorm and review the source files.
-
Go to the
deploy
folder:cd <path_to_deploy_folder>
-
Find out the name of the
ACTIVE
profile of the Yandex Cloud CLI command line interface. In the terminal, run this command:yc config profile list
-
Get the active profile parameters:
yc config profile get <profile_name>
-
Copy the parameters to provider.tf
:token
: OAuth token.cloud-id
: Cloud ID.folder-id
: Folder ID.
-
Export the folder ID to the environment variable:
export FOLDER_ID=<folder_ID> echo $FOLDER_ID
-
Run the Terraform initialization command:
terraform init
Note
Run all the Terraform commands in the
deploy
folder.
Create a Managed Service for YDB database
The project uses a YDB database in serverless mode. The database consists of two tables: movies
to keep movie data and votes
to keep user rates. Each table entry contains the ID and the final set of attributes.
-
The Terraform configuration for database creation is described in ydb.tf
. Create a database:terraform apply -target=yandex_ydb_database_serverless.movies_database
Confirm resource creation: type
yes
in the terminal and press Enter.The command result contains the variables:
movies_database_document_api_endpoint
: The Document API endpoint of a database.movies_database_path
: The relative path to the database.
You can make sure the
movies-database
DB is created in the management console or using the CLI commandyc ydb database list
. -
Export the values
movies_database_document_api_endpoint
andmovies_database_path
from the previous command result to environment variables:export DOCUMENT_API_ENDPOINT=<movies_database_document_api_endpoint> echo $DOCUMENT_API_ENDPOINT export MOVIES_DATABASE_PATH=<movies_database_path> echo $MOVIES_DATABASE_PATH
-
Create the
movies
andvotes
tables in themovies-database
DB:aws dynamodb create-table \ --table-name movies \ --attribute-definitions \ AttributeName=id,AttributeType=N \ AttributeName=title,AttributeType=S \ AttributeName=type,AttributeType=S \ AttributeName=original_title,AttributeType=S \ AttributeName=original_language,AttributeType=S \ AttributeName=release_date,AttributeType=S \ AttributeName=poster_path,AttributeType=S \ AttributeName=popularity,AttributeType=N \ AttributeName=video,AttributeType=S \ AttributeName=vote_count,AttributeType=N \ AttributeName=vote_average,AttributeType=N \ AttributeName=genres,AttributeType=S \ AttributeName=backdrop_path,AttributeType=S \ AttributeName=adult,AttributeType=S \ AttributeName=overview,AttributeType=S \ --key-schema \ AttributeName=id,KeyType=HASH \ --global-secondary-indexes \ "[ { \"IndexName\": \"PopularityIndex\", \"KeySchema\": [{\"AttributeName\":\"type\",\"KeyType\":\"HASH\"}, {\"AttributeName\":\"popularity\",\"KeyType\":\"RANGE\"}], \"Projection\":{ \"ProjectionType\":\"ALL\" } } ]" \ --endpoint ${DOCUMENT_API_ENDPOINT} aws dynamodb create-table \ --table-name votes \ --attribute-definitions \ AttributeName=id,AttributeType=S \ AttributeName=user_id,AttributeType=S \ AttributeName=movie_id,AttributeType=N \ AttributeName=value,AttributeType=N \ --key-schema \ AttributeName=id,KeyType=HASH \ --global-secondary-indexes \ "[ { \"IndexName\": \"MovieIndex\", \"KeySchema\": [{\"AttributeName\":\"movie_id\",\"KeyType\":\"HASH\"}], \"Projection\":{ \"ProjectionType\":\"ALL\" } } ]" \ --endpoint ${DOCUMENT_API_ENDPOINT}
-
Make sure the tables are created:
aws dynamodb describe-table \ --table-name movies \ --endpoint ${DOCUMENT_API_ENDPOINT} aws dynamodb describe-table \ --table-name votes \ --endpoint ${DOCUMENT_API_ENDPOINT}
Each table supports two indexes:
Movies
: The index for quickly searching for a movie by ID, and index for sorting movies by popularity.Votes
: The index for searching for a user's votes by movie, and index for getting all movie rates.
Run CRUD operations
A database layer is used every time data is retrieved, updated, or deleted. These actions are called CRUD operations.
Interaction with the database via the Document API is performed using the AWS SDK for JavaScript v3
- model.ts
defines the models of aMovie
movie andVote
rates via the TypeScript interface. - repository.ws
implements CRUD operations for using these entities.
IAM tokens are used for authorization when data operations are executed. To get an IAM token before the operation, the metadata service is called.
Create a service account
-
The Terraform configuration to create a service account is described in sa.tf
. Create a service account:terraform apply -target=yandex_iam_service_account.movies_api_sa
Confirm resource creation: type
yes
in the terminal and press Enter. -
In the command result, the
movies_api_sa_id
shows the ID of the created service account. Export it to the environment variable:export MOVIES_API_SA_ID=<service_account_ID> echo $MOVIES_API_SA_ID
-
Assign roles to the service account:
yc resource-manager folder add-access-binding ${FOLDER_ID} \ --role ydb.admin \ --subject serviceAccount:${MOVIES_API_SA_ID} yc resource-manager folder add-access-binding ${FOLDER_ID} \ --role container-registry.images.puller \ --subject serviceAccount:${MOVIES_API_SA_ID} yc resource-manager folder add-access-binding ${FOLDER_ID} \ --role serverless.containers.invoker \ --subject serviceAccount:${MOVIES_API_SA_ID}
Where:
--role
: Role being assigned.--subject serviceAccount
: Service account ID.
The service account is assigned roles for the following actions:
- Calling the container in Serverless Containers.
- Executing operations in YDB.
The roles are assigned to the whole folder rather than an individual resource.
Compile the application source code in TypeScript
-
Go to the repository root folder and install all the necessary dependencies:
cd <path_to_folder_sls-web-application> npm ci
After the command is executed, the
node_modules
with all the necessary dependencies appears in the folder. -
Run the project build:
npm run build
After the command is executed, the
dist
folder with the compiled JS files appears in the folder.
Develop the REST API
The openapi/api.yaml
To implement the service according to the specifications, the OpenAPI Backend
Deploy the application in Serverless Containers
Build the application as a Docker image and run it in Serverless Containers:
-
In the OpenAPI specifications
api.yaml
, in thex-yc-apigateway.service_account_id
field, type the ID of the created service account. -
The container-registry.tf
file describes a configuration of the registry and repository to which an application Docker image will be pushed. Go to thedeploy
folder and create resources in Yandex Container Registry:cd <path_to_folder_deploy> terraform apply -target=yandex_container_registry.default terraform apply -target=yandex_container_repository.movies_api_repository
Confirm resource creation: type
yes
in the terminal and press Enter. -
In the command result, the
movies_api_repository_name
variable shows the name of the repository to which a Docker image will be uploaded. Export it to the environment variable:export MOVIES_API_REPOSITORY_NAME=<repository_name> echo $MOVIES_API_REPOSITORY_NAME
-
Set up Docker for the created repository:
yc container registry configure-docker
-
The Dockerfile
describes a configuration to build a Docker image. Build the image and push it to the repository created in the previous step:docker build -t ${MOVIES_API_REPOSITORY_NAME}:0.0.1 . docker push ${MOVIES_API_REPOSITORY_NAME}:0.0.1
-
Create a Serverless Containers container:
yc sls container create \ --name movies-api-container \ --folder-id ${FOLDER_ID}
Where:
--name
: Container name.--folder-id
: Folder ID.
-
The command result shows the ID of the container. Export it to the environment variable:
export MOVIES_API_CONTAINER_ID=<container_ID> echo $MOVIES_API_CONTAINER_ID
-
Create a container revision from a Docker image version
0.0.1
:yc sls container revisions deploy \ --folder-id ${FOLDER_ID} \ --container-id ${MOVIES_API_CONTAINER_ID} \ --memory 512M \ --cores 1 \ --execution-timeout 5s \ --concurrency 4 \ --environment AWS_ACCESS_KEY_ID=FAKE_AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY=FAKE_AWS_SECRET_ACCESS_KEY,DOCUMENT_API_ENDPOINT=${DOCUMENT_API_ENDPOINT} \ --service-account-id ${MOVIES_API_SA_ID} \ --image ${MOVIES_API_REPOSITORY_NAME}:0.0.1
Where:
--folder-id
: Folder ID.--container-id
: Container ID.--memory
: Amount of memory available for the container.--cores
: Number of vCPU cores available for the container.--execution-timeout
: Execution timeout.--concurrency
: Maximum number of concurrent container calls. If the number of requests to a container exceeds the value of the--concurrency
parameter, the service scales the container up by launching additional copies.--environment
: Environment variables. The Document API endpoint of a database is passed to the application via theDOCUMENT_API_ENDPOINT
environment variable.--service-account-id
: Service account ID.--image
: Repository name.
Deploy the API in API Gateway
-
In the OpenAPI specifications
api.yaml
, replace the${MOVIES_API_CONTAINER_ID}
variable with the ID of the created container. -
The api-gateway.tf
file describes a Terraform configuration for creating the API gateway. Deploy the API gateway:terraform apply -target=yandex_api_gateway.movies_api_gateway
Confirm resource creation: type
yes
in the terminal and press Enter. -
In the command result, the
movies_api_gateway_domain
variable shows the domain name of the API gateway. Export it to the environment variable:export MOVIES_API_GATEWAY_DOMAIN=<API_gateway_domain_name> echo $MOVIES_API_GATEWAY_DOMAIN
Check the performance of the created CRUD API
To check the performance of the created CRUD API, run the following HTTP requests using the curl
command:
-
Retrieve a movie list:
curl "${MOVIES_API_GATEWAY_DOMAIN}/movies?limit=10"
The response must return an empty list
[]
, because at the moment, there's no data in the database. -
Add movie details:
curl \ --location \ --request POST 'https://${MOVIES_API_GATEWAY_DOMAIN}/movies' \ --header 'Content-Type: application/json' \ --data-raw '{ "id": "301", "title": "The Matrix", "release_date": 1999 }'
-
Retrieve movie details:
curl \ --location \ --request GET 'https://${MOVIES_API_GATEWAY_DOMAIN}/movies/301'
-
Add details of another movie:
curl \ --location \ --request POST 'https://${MOVIES_API_GATEWAY_DOMAIN}/movies' \ --header 'Content-Type: application/json' \ --data-raw '{ "id": "299", "title": "The Matrix Reloaded", "release_date": 2003 }'
-
Retrieve a movie list:
curl \ --location \ --request GET 'https://${MOVIES_API_GATEWAY_DOMAIN}/movies?from=1&limit=5'
You can also upload the specifications to Postman${MOVIES_API_GATEWAY_DOMAIN}
variable to the servers
section. This enables you to easily run requests to the REST API.
View diagnostic information about the container. In the management console
You can also view monitoring logs and charts on the API gateway page.
How to delete the resources you created
To stop paying for resources created using Terraform, delete them. In the terminal, run this command:
terraform destroy
Confirm the resource deletion: type yes
in the terminal and press Enter.