Developing a custom integration in API Gateway

With serverless technology, you can create your own integration with Yandex Cloud services.

Custom integration is a function in Yandex Cloud Functions or a container in Yandex Serverless Containers designed for a specific use case.

You can configure a function or container in the OpenAPI 3.0-based API Gateway specification to handle specific HTTP requests.

Develop a Yandex Managed Service for YDB integration function for YDB operations. The function will communicate with Managed Service for YDB and handle external HTTP requests via an API gateway using the Amazon DynamoDB-compatible HTTP API. The function is written in TypeScript and runs on Node.js 16.

We will use this integration to implement a CRUD API for accessing the movie database deployed in Managed Service for YDB.

To deploy a project:

1. Set up your environment.
2. Download the integration project.
3. Compile a function.
4. Upload the function file to the bucket.
5. Set up a resource configuration for the integration.
6. Deploy resources for the integration.
7. Test the new CRUD API.

If you no longer need the resources you created, delete them.

Getting startedGetting started

Sign up in Yandex Cloud and create a billing account:

1. Navigate to the management console and log in to Yandex Cloud or register a new account.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can navigate to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Required paid resourcesRequired paid resources

The cost of integration resources includes:

• Fee for the amount of stored data, number of data operations, and outbound traffic (see Yandex Object Storage pricing).
• Fee for YDB operations and data storage (see Managed Service for YDB pricing for serverless mode).
• Fee for function invocation count, computing resources allocated to run the function, and outbound traffic (see Cloud Functions pricing).
• Fee for the number of requests to the API gateway and outbound traffic (see API Gateway pricing).

Set up your environmentSet up your environment

Download the integration projectDownload the integration project

Clone the integration project repository:

git clone https://github.com/yandex-cloud-examples/yc-serverless-apigw-dynamodb-connector.git

The src directory contains these source files for creating a function:

• event.ts: Event code describing the request structure and RequestContext code.
• dynamodb.ts: Code to process function invocation and basic commands.
• iam.ts: Code to retrieve IAM tokens for YDB request authorization.

When invoking a function defined in dynamodb.ts, provide the operation context in the requestContext.apiGateway.operationContext field of the event object.

The context parameter in the API gateway specification sets the operation context for the integration function.

event.ts defines interfaces for basic Amazon DynamoDB commands and command parameters to implement in this integration. This helps specify the format of the operation context and how the function handles it.

Compile a functionCompile a function

1. Open the terminal and navigate to the project root directory:

   cd <path_to_project_root_directory>
   

2. Install the project dependencies:

   npm ci
   

3. Compile and build the function code:

   npm run build
   

4. Package it into a ZIP archive:

   npm run package
   

Upload the function file to the bucketUpload the function file to the bucket

1. Create a publicly accessible bucket. Save the bucket name, as you will need it later.
2. Upload the apigw-dynamodb-connector-0.0.1.zip archive with the function code from the build directory to the bucket.

Set up a resource configuration for the integrationSet up a resource configuration for the integration

To deploy the CRUD API via the integration function, use Terraform.

With a special Terraform module developed for this integration scenario, you will easily configure your Yandex Cloud resources. You will use Terraform to create the following resources:

• Serverless YDB-enabled database.
• Integration function.
• Service account the function will use to access the database.
• API gateway.

To set up configuration files for Terraform:

1. Find out the name of the ACTIVE profile of the Yandex Cloud CLI. In the terminal, run this command:

   yc config profile list
   

2. Get the active profile properties:

   yc config profile get <profile_name>
   

   Save these properties:

   • token: OAuth token.
   • cloud-id: Cloud ID.
   • folder-id: Folder ID.

3. Create a directory named crud-api and open it:

   mkdir crud-api
   cd crud-api
   

   Run all subsequent Terraform commands in the crud-api directory.

4. Create a file named main.tf and paste the Terraform module configuration into it. Set the following properties for the new resources:

   • cloud_id: Cloud ID.
   • folder_id: Folder ID.
   • oauth_token: OAuth token.
   • database_connector_bucket: Name of the bucket with the integration function.

   locals {
     cloud_id    = "<cloud_ID>"
     folder_id   = "<folder_ID>"
     oauth_token = "<OAuth_token>"
     zone        = "ru-central1-d"
   }
   
   module "crud-api" {
     source = "github.com/yandex-cloud-examples/yc-serverless-ydb-api"
   
     folder_id                 = local.folder_id
     api_name                  = "movies-api"
     database_name             = "movies-db"
     service_account_name      = "movies-api-service-account"
     region                    = "region-id"
     openapi_spec              = "api.yaml"
     table_specs               = ["file://table.json"]
     database_connector_bucket = "<name_of_bucket_with_integration_function>"
     database_connector_object = "apigw-dynamodb-connector-0.0.1.zip"
   }
   
   terraform {
     required_providers {
       yandex = {
         source = "yandex-cloud/yandex"
       }
       null   = {
         source = "registry.terraform.io/hashicorp/null"
       }
     }
     required_version = ">= 0.13"
   }
   
   provider "yandex" {
     token     = local.oauth_token
     cloud_id  = local.cloud_id
     folder_id = local.folder_id
     zone      = local.zone
   }
   
   output "crud_api_domain" {
     value = module.crud-api.api_gateway_domain
   }
   

5. Create a file named table.json and paste into it the schema specification for the table to create in YDB:

   {
     "TableName": "movie",
     "KeySchema": [
       {
         "AttributeName": "id",
         "KeyType": "HASH"
       }
     ],
     "AttributeDefinitions": [
       {
         "AttributeName": "id",
         "AttributeType": "S"
       },
       {
         "AttributeName": "title",
         "AttributeType": "S"
       },
       {
         "AttributeName": "year",
         "AttributeType": "N"
       }
     ]
   }
   

6. Create a file named api.yaml and paste into it the OpenAPI specification for the new API gateway:

   openapi: "3.0.0"
   info:
     version: 1.0.0
     title: Movies API
   x-yc-apigateway:
     service_account_id: ${SERVICE_ACCOUNT_ID}
   
   paths:
     /movies:
       post:
         description: Create movie
         operationId: createMovie
         requestBody:
           description: Movie to create
           required: true
           content:
             application/json:
               schema:
                 $ref: '#/components/schemas/Movie'
         responses:
           '200':
             description: Created or updated movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: PutItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
       get:
         description: Get movies
         operationId: getMovies
         parameters:
           - name: from
             in: query
             description: Identifier from which will be queried movies in ascending order
             required: true
             schema:
               type: string
           - name: limit
             in: query
             description: Maximum number of movies in response
             required: false
             schema:
               type: number
               default: 10
         responses:
           '200':
            description: Movies
             content:
               application/json:
                 schema:
                   type: array
                   items:
                     $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: Scan
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             limit: '{limit}'
             exclusiveStartKey: '{"id": "{from}"}'
     /movies/{movieId}:
       parameters:
         - name: movieId
           in: path
           description: Identifier of movie
           required: true
           schema:
             type: string
       get:
         description: Get movie by id
         operationId: getMovieById
         responses:
           '200':
             description: Movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: GetItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             key: '{"id": "{movieId}"}'
       put:
         description: Update movie by id
         operationId: updateMovieById
         requestBody:
           description: Movie or attributes to update
           required: true
           content:
             application/json:
               schema:
                 $ref: '#/components/schemas/Movie'
         responses:
           '200':
             description: Updated movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: UpdateItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             key: '{"id": "{movieId}"}'
       delete:
         description: Delete movie by id
         operationId: deleteMovieById
         responses:
           '200':
             description: Deleted movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: DeleteItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             key: '{"id": "{movieId}"}'
   components:
     schemas:
       Movie:
         type: object
         required:
           - id
           - title
           - year
         properties:
           id:
             type: string
           title:
             type: string
           year:
             type: integer
   
       Error:
         type: object
         required:
           - message
         properties:
           message:
             type: string
   

Deploy resources for the integrationDeploy resources for the integration

1. Initialize Terraform. In the terminal, run this command:

   terraform init
   

2. Deploy the cloud resources:

   terraform apply
   

3. Confirm creating the resources by typing yes in the terminal and pressing Enter.

   In the command output, the crud_api_domain variable will show the domain address of the new CRUD API. Save the address, as you will need it later.

   You can use the management console to check the created resources.

Test the new CRUD APITest the new CRUD API

To test the new CRUD API, send the following HTTP requests:

1. Add movie details. In the terminal, run this command:

   curl \
     --location \
     --request POST 'https://<domain_address_of_CRUD_API>/movies' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "id": "301",
       "title": "The Matrix",
       "year": 1999
     }'
   

2. Retrieve movie details:

   curl \
     --location \
     --request GET 'https://<domain_address_of_CRUD_API>/movies/301'
   

3. Modify movie details:

   curl \
     --location \
     --request PUT 'https://<domain_address_of_CRUD_API>/movies/301' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "title": "The Matrix"
     }'
   

4. Add details for another movie:

   curl \
     --location \
     --request POST 'https://<domain_address_of_CRUD_API>/movies' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "id": "299",
       "title": "The Matrix Reloaded",
       "year": 2003
     }'
   

5. Retrieve a movie list:

   curl \
     --location \
     --request GET 'https://<domain_address_of_CRUD_API>/movies?from=1&limit=5'
   

6. Delete the details of one of the movies:

   curl \
     --location \
     --request DELETE 'https://<domain_address_of_CRUD_API>/movies/301' \
     --data-raw ''
   

How to delete the resources you createdHow to delete the resources you created

To stop paying for the resources you created:

• Delete the resources you created with Terraform. In the terminal, run this command:

  terraform destroy
  

  Confirm the resource deletion by typing yes in the terminal and pressing Enter.

• Delete the bucket with the function file.