Developing a custom integration in API Gateway

Using serverless technology, you can create your own integration with Yandex Cloud services.

User integration is a Yandex Cloud Functions function or Yandex Serverless Containers container designed to perform common tasks.

The function or container can be configured in the API Gateway API gateway specifications supporting OpenAPI 3.0 to execute specific HTTP requests.

Develop Yandex Managed Service for YDB integration function for the YDB DBMS. The function interacts with Managed Service for YDB and processes external HTTP requests via the API gateway using the Amazon DynamoDB-compatible HTTP API. The function source code language is TypeScript, the runtime environment is Node.js 16.

The integration will be applied to implement the CRUD API for a movie database deployed in Managed Service for YDB.

To deploy a project:

1. Configure the environment.
2. Download a project with integration.
3. Compile a function.
4. Upload the file of the function to the bucket.
5. Prepare a resource configuration for the integration.
6. Deploy resources for the integration.
7. Test the new CRUD API.

If you no longer need the resources you created, delete them.

Getting startedGetting started

Sign up for Yandex Cloud and create a billing account:

1. Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Required paid resourcesRequired paid resources

The cost of resources for the integration includes:

• Fee for the occupied data storage volume, number of operations with data, and outgoing traffic (see Yandex Object Storage pricing).
• Fee for YDB operations and data storage (see Managed Service for YDB pricing for serverless mode).
• Fee for the number of function calls, computing resources allocated to a function, and outgoing traffic (see Cloud Functions pricing).
• Fee for the number of requests to the API gateway and outgoing traffic (see API Gateway pricing).

Configure the environmentConfigure the environment

Download a project with integrationDownload a project with integration

Clone the integration project repository:

git clone https://github.com/yandex-cloud-examples/yc-serverless-apigw-dynamodb-connector.git

The src folder contains source files for creating the function:

• event.ts: The Event code describing the request structure and RequestContext code describing the request context.
• dynamodb.ts: The code to process calls of a function and main commands.
• iam.ts: The code to retrieve IAM tokens for authorization when executing requests to the YDB.

When a function is called, the operation context is provided in the dynamodb.ts file in the requestContext.apiGateway.operationContext field of the event object.

The operation context is defined in the context parameter in the specification of the API gateway invoking the integration function.

The file event.ts defines interfaces for main Amazon DynamoDB commands and command parameters that are implemented in this integration. This is necessary for specifying the operation context format and working with the format within a function.

Compile a functionCompile a function

1. Open the terminal and go to the project root folder:

   cd <path_to_project_root_directory>
   

2. Set the dependencies required for the project:

   npm ci
   

3. Compile and build the function code:

   npm run build
   

4. Add the built function code to a ZIP archive:

   npm run package
   

Upload the file of the function to the bucketUpload the file of the function to the bucket

1. Create a bucket with public access. Save the bucket name. You will need it later.
2. Upload the apigw-dynamodb-connector-0.0.1.zip ZIP archive with the function code from the build folder to the bucket.

Prepare a resource configuration for the integrationPrepare a resource configuration for the integration

To deploy the CRUD API using the integration function, you will need the Terraform tool.

A special Terraform module developed for this integration example makes it easier to configure Yandex Cloud resources. Created Terraform resources:

• Serverless YDB database.
• Integration function.
• Service account for the function to access the database.
• API gateway.

To prepare configuration files for Terraform:

1. Find out the name of the ACTIVE profile of the Yandex Cloud CLI command line interface. In the terminal, run this command:

   yc config profile list
   

2. Get the active profile parameters:

   yc config profile get <profile_name>
   

   Save the parameters:

   • token: OAuth token.
   • cloud-id: Cloud ID.
   • folder-id: Folder ID.

3. Create a crud-api directory and navigate to it:

   mkdir crud-api
   cd crud-api
   

   Run all subsequent Terraform commands in the crud-api folder.

4. Create a file named main.tf and copy into it the Terraform module configuration. Set the parameters of the resources to be created:

   • cloud_id: Cloud ID.
   • folder_id: Folder ID.
   • oauth_token: OAuth token.
   • database_connector_bucket: Name of the bucket with the integration function.

   locals {
     cloud_id    = "<cloud_ID>"
     folder_id   = "<folder_ID>"
     oauth_token = "<OAuth_token>"
     zone        = "ru-central1-d"
   }
   
   module "crud-api" {
     source = "https://github.com/yandex-cloud-examples/yc-serverless-ydb-api"
   
     folder_id                 = local.folder_id
     api_name                  = "movies-api"
     database_name             = "movies-db"
     service_account_name      = "movies-api-service-account"
     region                    = "region-id"
     openapi_spec              = "api.yaml"
     table_specs               = ["file://table.json"]
     database_connector_bucket = "<name_of_bucket_with_the_integration_function>"
     database_connector_object = "apigw-dynamodb-connector-0.0.1.zip"
   }
   
   terraform {
     required_providers {
       yandex = {
         source = "yandex-cloud/yandex"
       }
       null   = {
         source = "registry.terraform.io/hashicorp/null"
       }
     }
     required_version = ">= 0.13"
   }
   
   provider "yandex" {
     token     = local.oauth_token
     cloud_id  = local.cloud_id
     folder_id = local.folder_id
     zone      = local.zone
   }
   
   output "crud_api_domain" {
     value = module.crud-api.api_gateway_domain
   }
   

5. Create a file named table.json and copy into it the specification of the table schema created by YDB:

   {
     "TableName": "movie",
     "KeySchema": [
       {
         "AttributeName": "id",
         "KeyType": "HASH"
       }
     ],
     "AttributeDefinitions": [
       {
         "AttributeName": "id",
         "AttributeType": "S"
       },
       {
         "AttributeName": "title",
         "AttributeType": "S"
       },
       {
         "AttributeName": "year",
         "AttributeType": "N"
       }
     ]
   }
   

6. Create a file named api.yaml and copy into it the OpenAPI specification of the new API gateway:

   openapi: "3.0.0"
   info:
     version: 1.0.0
     title: Movies API
   x-yc-apigateway:
     service_account_id: ${SERVICE_ACCOUNT_ID}
   
   paths:
     /movies:
       post:
         description: Create movie
         operationId: createMovie
         requestBody:
           description: Movie to create
           required: true
           content:
             application/json:
               schema:
                 $ref: '#/components/schemas/Movie'
         responses:
           '200':
             description: Created or updated movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: PutItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
       get:
         description: Get movies
         operationId: getMovies
         parameters:
           - name: from
             in: query
             description: Identifier from which will be queried movies in ascending order
             required: true
             schema:
               type: string
           - name: limit
             in: query
             description: Maximum number of movies in response
             required: false
             schema:
               type: number
               default: 10
         responses:
           '200':
            description: Movies
             content:
               application/json:
                 schema:
                   type: array
                   items:
                     $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: Scan
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             limit: '{limit}'
             exclusiveStartKey: '{"id": "{from}"}'
     /movies/{movieId}:
       parameters:
         - name: movieId
           in: path
           description: Identifier of movie
           required: true
           schema:
             type: string
       get:
         description: Get movie by id
         operationId: getMovieById
         responses:
           '200':
             description: Movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: GetItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             key: '{"id": "{movieId}"}'
       put:
         description: Update movie by id
         operationId: updateMovieById
         requestBody:
           description: Movie or attributes to update
           required: true
           content:
             application/json:
               schema:
                 $ref: '#/components/schemas/Movie'
         responses:
           '200':
             description: Updated movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: UpdateItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             key: '{"id": "{movieId}"}'
       delete:
         description: Delete movie by id
         operationId: deleteMovieById
         responses:
           '200':
             description: Deleted movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: DeleteItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             key: '{"id": "{movieId}"}'
   components:
     schemas:
       Movie:
         type: object
         required:
           - id
           - title
           - year
         properties:
           id:
             type: string
           title:
             type: string
           year:
             type: integer
   
       Error:
         type: object
         required:
           - message
         properties:
           message:
             type: string
   

Deploy resources for the integration.Deploy resources for the integration.

1. Initiate Terraform. In the terminal, run this command:

   terraform init
   

2. Deploy cloud resources:

   terraform apply
   

3. Confirm creating the resources: type yes in the terminal and press Enter.

   In the command output, the crud_api_domain variable will show the domain address of the new CRUD API. Save the address. You will need it later.

   You can use the management console to check the created resources.

Test the new CRUD APITest the new CRUD API

To test the new CRUD API, send the following HTTP requests:

1. Add movie details. In the terminal, run this command:

   curl \
     --location \
     --request POST 'https://<domain_address_of_CRUD_API>/movies' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "id": "301",
       "title": "The Matrix",
       "year": 1999
     }'
   

2. Retrieve movie details:

   curl \
     --location \
     --request GET 'https://<domain_address_of_CRUD_API>/movies/301'
   

3. Change movie details:

   curl \
     --location \
     --request PUT 'https://<domain_address_of_CRUD_API>/movies/301' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "title": "The Matrix"
     }'
   

4. Add details of another movie:

   curl \
     --location \
     --request POST 'https://<domain_address_of_CRUD_API>/movies' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "id": "299",
       "title": "The Matrix Reloaded",
       "year": 2003
     }'
   

5. Retrieve a movie list:

   curl \
     --location \
     --request GET 'https://<domain_address_of_CRUD_API>/movies?from=1&limit=5'
   

6. Delete details of a movie:

   curl \
     --location \
     --request DELETE 'https://<domain_address_of_CRUD_API>/movies/301' \
     --data-raw ''
   

How to delete the resources you createdHow to delete the resources you created

To stop paying for the resources you created:

• Delete the resources created with Terraform. In the terminal, run this command:

  terraform destroy
  

  Confirm the resource deletion: type yes in the terminal and press Enter.

• Delete the bucket with the function file.