Developing a custom integration in API Gateway

Using serverless technologies, you can create your own integration with Yandex Cloud services.

User integration is a Yandex Cloud Functions-enabled function or Yandex Serverless Containers-enabled container designed to perform common tasks.

You can configure a function or container in the API Gateway specification under OpenAPI 3.0 to execute specific HTTP requests.

Develop an integration function with Yandex Managed Service for YDB for YDB. The function will interact with Managed Service for YDB and processes external HTTP requests over the API gateway using the Amazon DynamoDB-compatible HTTP API. The function source code language is TypeScript; the runtime environment is Node.js 16.

The integration will be applied to implement the CRUD API for a movie database deployed in Managed Service for YDB.

To deploy a project:

1. Set up your environment.
2. Download a project with integration.
3. Compile a function.
4. Upload the function file to the bucket.
5. Prepare a resource configuration for the integration.
6. Deploy resources for the integration.
7. Test the new CRUD API.

If you no longer need the resources you created, delete them.

Getting startedGetting started

Sign up in Yandex Cloud and create a billing account:

1. Navigate to the management console and log in to Yandex Cloud or register a new account.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can navigate to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Required paid resourcesRequired paid resources

The cost of integration resources includes:

• Fee for the occupied data storage volume, number of data operations, and outbound traffic (see Yandex Object Storage pricing).
• Fee for YDB operations and data storage (see Managed Service for YDB pricing for serverless mode).
• Fee for the number of function calls, computing resources allocated to a function, and outbound traffic (see Cloud Functions pricing).
• Fee for the number of requests to the API gateway and outbound traffic (see API Gateway pricing).

Set up your environmentSet up your environment

Download a project with integrationDownload a project with integration

Clone the integration project repository:

git clone https://github.com/yandex-cloud-examples/yc-serverless-apigw-dynamodb-connector.git

The src folder contains source files for creating the function:

• event.ts: Event code describing the request structure and RequestContext code.
• dynamodb.ts: Code to process function calls and basic commands.
• iam.ts: Code to retrieve IAM tokens for authorization when executing requests to YDB.

When a function is called, the operation context is provided in the dynamodb.ts file in the requestContext.apiGateway.operationContext field of the event object.

The operation context is defined in the context parameter in the specification of the API gateway invoking the integration function.

The event.ts file defines interfaces for basic Amazon DynamoDB commands and command parameters to implement in this integration. This is required for specifying the operation context format and working with the format within a function.

Compile a functionCompile a function

1. Open the terminal and navigate to the project root folder:

   cd <path_to_project_root_directory>
   

2. Set the dependencies required for the project:

   npm ci
   

3. Compile and build the function code:

   npm run build
   

4. Add the function code you built to a ZIP archive:

   npm run package
   

Upload the function file to the bucketUpload the function file to the bucket

1. Create a bucket with public access. Save the bucket name. You will need it later.
2. Upload the apigw-dynamodb-connector-0.0.1.zip archive with the function code from the build directory to the bucket.

Set up a resource configuration for the integrationSet up a resource configuration for the integration

To deploy the CRUD API using the integration function, you will need Terraform.

A special Terraform module developed for this integration tutorial simplifies configuring Yandex Cloud resources. Terraform resources being created:

• Serverless YDB database.
• Integration function.
• Service account for the function to access the database.
• API gateway.

To get configuration files for Terraform ready:

1. Find out the name of the ACTIVE profile of the Yandex Cloud CLI command line interface. In the terminal, run this command:

   yc config profile list
   

2. Get the active profile properties:

   yc config profile get <profile_name>
   

   Save the properties:

   • token: OAuth token
   • cloud-id: Cloud ID
   • folder-id: Folder ID

3. Create a crud-api directory and navigate to it:

   mkdir crud-api
   cd crud-api
   

   Run all subsequent Terraform commands in the crud-api folder.

4. Create a file named main.tf and copy into it the Terraform module configuration. Configure the resources being created:

   • cloud_id: Cloud ID.
   • folder_id: Folder ID.
   • oauth_token: OAuth token.
   • database_connector_bucket: Name of the bucket with the integration function.

   locals {
     cloud_id    = "<cloud_ID>"
     folder_id   = "<folder_ID>"
     oauth_token = "<OAuth_token>"
     zone        = "ru-central1-d"
   }
   
   module "crud-api" {
     source = "github.com/yandex-cloud-examples/yc-serverless-ydb-api"
   
     folder_id                 = local.folder_id
     api_name                  = "movies-api"
     database_name             = "movies-db"
     service_account_name      = "movies-api-service-account"
     region                    = "region-id"
     openapi_spec              = "api.yaml"
     table_specs               = ["file://table.json"]
     database_connector_bucket = "<name_of_bucket_with_integration_function>"
     database_connector_object = "apigw-dynamodb-connector-0.0.1.zip"
   }
   
   terraform {
     required_providers {
       yandex = {
         source = "yandex-cloud/yandex"
       }
       null   = {
         source = "registry.terraform.io/hashicorp/null"
       }
     }
     required_version = ">= 0.13"
   }
   
   provider "yandex" {
     token     = local.oauth_token
     cloud_id  = local.cloud_id
     folder_id = local.folder_id
     zone      = local.zone
   }
   
   output "crud_api_domain" {
     value = module.crud-api.api_gateway_domain
   }
   

5. Create a file named table.json and copy into it the specification of the table schema created by YDB:

   {
     "TableName": "movie",
     "KeySchema": [
       {
         "AttributeName": "id",
         "KeyType": "HASH"
       }
     ],
     "AttributeDefinitions": [
       {
         "AttributeName": "id",
         "AttributeType": "S"
       },
       {
         "AttributeName": "title",
         "AttributeType": "S"
       },
       {
         "AttributeName": "year",
         "AttributeType": "N"
       }
     ]
   }
   

6. Create a file named api.yaml and copy into it the OpenAPI specification of the new API gateway:

   openapi: "3.0.0"
   info:
     version: 1.0.0
     title: Movies API
   x-yc-apigateway:
     service_account_id: ${SERVICE_ACCOUNT_ID}
   
   paths:
     /movies:
       post:
         description: Create movie
         operationId: createMovie
         requestBody:
           description: Movie to create
           required: true
           content:
             application/json:
               schema:
                 $ref: '#/components/schemas/Movie'
         responses:
           '200':
             description: Created or updated movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: PutItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
       get:
         description: Get movies
         operationId: getMovies
         parameters:
           - name: from
             in: query
             description: Identifier from which will be queried movies in ascending order
             required: true
             schema:
               type: string
           - name: limit
             in: query
             description: Maximum number of movies in response
             required: false
             schema:
               type: number
               default: 10
         responses:
           '200':
            description: Movies
             content:
               application/json:
                 schema:
                   type: array
                   items:
                     $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: Scan
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             limit: '{limit}'
             exclusiveStartKey: '{"id": "{from}"}'
     /movies/{movieId}:
       parameters:
         - name: movieId
           in: path
           description: Identifier of movie
           required: true
           schema:
             type: string
       get:
         description: Get movie by id
         operationId: getMovieById
         responses:
           '200':
             description: Movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: GetItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             key: '{"id": "{movieId}"}'
       put:
         description: Update movie by id
         operationId: updateMovieById
         requestBody:
           description: Movie or attributes to update
           required: true
           content:
             application/json:
               schema:
                 $ref: '#/components/schemas/Movie'
         responses:
           '200':
             description: Updated movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: UpdateItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             key: '{"id": "{movieId}"}'
       delete:
         description: Delete movie by id
         operationId: deleteMovieById
         responses:
           '200':
             description: Deleted movie
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Movie'
           default:
             description: error
             content:
               application/json:
                 schema:
                   $ref: '#/components/schemas/Error'
         x-yc-apigateway-integration:
           type: cloud_functions
           function_id: ${FUNCTION_ID}
           context:
             command: DeleteItem
             endpoint: ${DATABASE_ENDPOINT}
             tableName: movie
             key: '{"id": "{movieId}"}'
   components:
     schemas:
       Movie:
         type: object
         required:
           - id
           - title
           - year
         properties:
           id:
             type: string
           title:
             type: string
           year:
             type: integer
   
       Error:
         type: object
         required:
           - message
         properties:
           message:
             type: string
   

Deploy resources for the integrationDeploy resources for the integration

1. Initiate Terraform. In the terminal, run this command:

   terraform init
   

2. Deploy cloud resources:

   terraform apply
   

3. Confirm creating the resources: type yes in the terminal and press Enter.

   In the command output, the crud_api_domain variable will show the domain address of the new CRUD API. Save the address. You will need it later.

   You can use the management console to check the created resources.

Test the new CRUD APITest the new CRUD API

To test the new CRUD API, send the following HTTP requests:

1. Add movie details. In the terminal, run this command:

   curl \
     --location \
     --request POST 'https://<domain_address_of_CRUD_API>/movies' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "id": "301",
       "title": "The Matrix",
       "year": 1999
     }'
   

2. Retrieve movie details:

   curl \
     --location \
     --request GET 'https://<domain_address_of_CRUD_API>/movies/301'
   

3. Modify movie details:

   curl \
     --location \
     --request PUT 'https://<domain_address_of_CRUD_API>/movies/301' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "title": "The Matrix"
     }'
   

4. Add details for another movie:

   curl \
     --location \
     --request POST 'https://<domain_address_of_CRUD_API>/movies' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "id": "299",
       "title": "The Matrix Reloaded",
       "year": 2003
     }'
   

5. Retrieve a movie list:

   curl \
     --location \
     --request GET 'https://<domain_address_of_CRUD_API>/movies?from=1&limit=5'
   

6. Delete the details of one of the movies:

   curl \
     --location \
     --request DELETE 'https://<domain_address_of_CRUD_API>/movies/301' \
     --data-raw ''
   

How to delete the resources you createdHow to delete the resources you created

To stop paying for the resources you created:

• Delete the resources you created with Terraform. In the terminal, run this command:

  terraform destroy
  

  Confirm the resource deletion: type yes in the terminal and press Enter.

• Delete the bucket with the function file.