Contact UsTry it for free
© 2026 Direct Cursus Technology L.L.C.

Creating an agent based on the OpenAI Agents SDK with response streaming via web sockets on Yandex Cloud Functions and Yandex API Gateway

Written by
Updated at January 27, 2026

In this tutorial, you will create an agent with response streaming via web sockets on Yandex Cloud Functions and Yandex API Gateway. To access YandexGPT API, the function will use the OpenAI Agents SDK.

Agents may take a long time to respond when handling complex requests, e.g., generation of large texts with reasoning, search operations, or indexing. In such cases, it is essential to monitor progress and receive incremental results in real time. Response streaming enables immediate output of tokens, phrases, intermediate messages, step statuses, and logs, followed by the final response, without waiting for the entire scenario to complete. This enhances the perceived speed, provides a more interactive UI/UX, and enables users to cancel, retry, and dynamically update the interface. Streaming is supported by most frameworks, including the OpenAI Agents SDK.

To create an agent:

  1. Get your cloud ready.
  2. Set up your environment.
  3. Prepare the project files.
  4. Create a service account.
  5. Create an API key.
  6. Create a secret for the API key.
  7. Create a function.
  8. Create an API gateway.
  9. Check the result.

If you no longer need the resources you created, delete them.

Tip

If you do not want to tie your AI agent to a specific vendor, deploy the function in Yandex Serverless Containers as described in Developing functions in Functions Framework and deploying them in Yandex Serverless Containers.

What an AI agent is and how to use it

An AI agent is an AI-powered software assistant which can follow instructions, give answers to questions, and interact with users or other systems within a given context. Unlike standard generative models, AI agents can:

  • Support personalized instructions and have a personality.
  • Use external sources and third-party tools to gather information.
  • Maintain conversation context.
  • Perform multi-step actions to solve complex tasks.

Why use serverless functions to work with AI agents

Functions offered by Cloud Functions provide multiple benefits when deploying AI agents:

  • Scalability: Automatic scaling to accommodate the load.
  • Cost-effectiveness: You only pay for the actual execution time.
  • No infrastructure management required: You do not need to configure or maintain servers.
  • Fast deployment: AI agents are easy to create and update.
  • Multiple integration options: Simple connection to APIs and other Yandex Cloud services.

Get your cloud ready

Sign up for Yandex Cloud and create a billing account:

  1. Navigate to the management console and log in to Yandex Cloud or create a new account.
  2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can navigate to the cloud page to create or select a folder for your infrastructure.

Learn more about clouds and folders here.

The infrastructure support cost for this tutorial includes:

Set up your environment

  1. Install jq to work with JSON:

    apt-get install jq

    brew install jq

  2. Install wscat to test the agent:

    npm install -g wscat

Prepare the project files

To complete this tutorial, you need files containing the function code and the API gateway specification.

Clone the repository:

git clone https://github.com/yandex-cloud-examples/yc-serverless-streaming-openai-agent

In the repository, you will see:

  • function.zip containing the function.py source file and the requirements.txt dependency file.
  • gateway-spec.yaml file with the API gateway specification.

  1. Create a directory named yc-serverless-streaming-openai-agent and open it.

  2. Create a file named function.py and paste this code into it:

    import asyncio
import json
import os
import random
from typing import Dict, Any

from agents import Agent, OpenAIProvider, Runner, RunConfig, function_tool, set_tracing_disabled
from openai import AsyncOpenAI
from openai.types.responses import ResponseTextDeltaEvent
from yandex.cloud.serverless.apigateway.websocket.v1.connection_service_pb2 import SendToConnectionRequest
from yandex.cloud.serverless.apigateway.websocket.v1.connection_service_pb2_grpc import ConnectionServiceStub
from yandexcloud import SDK

BASE_URL = os.getenv("BASE_URL")
API_KEY = os.getenv("API_KEY")
MODEL_NAME = os.getenv("MODEL_NAME")
FOLDER_ID = os.environ.get('FOLDER_ID')

client = AsyncOpenAI(base_url=BASE_URL, api_key=API_KEY)
set_tracing_disabled(disabled=True)

@function_tool
def how_many_jokes() -> int:
    return random.randint(1, 10)

# Initializing the SDK Yandex Cloud
sdk = SDK()

def get_websocket_service():
    return sdk.client(ConnectionServiceStub)

def stream_to_websocket(connection_id: str, message: str):
    """Sending message to WebSocket connection"""
    websocket_service = get_websocket_service()
    request = SendToConnectionRequest(
        connection_id=connection_id,
        type=SendToConnectionRequest.TEXT,
        data=message.encode('utf-8')
    )
    websocket_service.Send(request)

async def process_stream(agent: Agent, run_config: RunConfig, connection_id: str, input_text: str):
    """Processing agent stream and sending to WebSocket"""
    result = Runner.run_streamed(
        agent,
        input=input_text,
        run_config=run_config
    )

    print("=== Run starting ===")

    async for event in result.stream_events():
        if event.type == "raw_response_event" and isinstance(event.data, ResponseTextDeltaEvent):
            stream_to_websocket(connection_id, event.data.delta)

    print("=== Run complete ===")

def handler(event: Dict[str, Any], context):
    """Cloud Function main handler"""
    try:
        # Getting event parameters
        input_text = event['body']
        request_context = event.get('requestContext')
        connection_id = request_context.get('connectionId')

        if not connection_id or not input_text:
            return {
                'statusCode': 400,
                'body': json.dumps({'error': 'Missing required parameters'})
            }

        # Creating the agent
        agent = Agent(
            name="Joker",
            instructions="First call the `how_many_jokes` tool, then tell that many jokes about topic from input.",
            tools=[how_many_jokes],
            model=f"gpt://{FOLDER_ID}/yandexgpt/latest",
        )

        run_config = RunConfig(
            model_provider=OpenAIProvider(
                api_key=API_KEY,
                project=FOLDER_ID,
                base_url="https://rest-assistant.api.cloud.yandex.net/v1",
                use_responses=True
            )
        )

        # Running asynchronous handling
        asyncio.run(process_stream(agent, run_config, connection_id, input_text))

        return {
            'statusCode': 200
        }

    except Exception as e:
        return {
            'statusCode': 500,
            'body': json.dumps({'error': str(e)})
        }

  3. Create a file named requirements.txt and paste this code into it:

    openai-agents>=0.0.17
yandexcloud>=0.227.0
grpcio>=1.60.0
protobuf>=4.25.1
openai~=1.86.0

  4. Create the function.zip archive and add the function.py and requirements.txt files to it.

  5. Create a file named gateway-spec.yaml and paste this code into it:

    openapi: 3.0.0
info:
  title: Sample API
  version: 1.0.0
paths:
  /:
    x-yc-apigateway-websocket-message:
      x-yc-apigateway-integration:
        payload_format_version: '0.1'
        function_id: <function_ID>
        tag: $latest
        type: cloud_functions
        service_account_id: <service_account_ID>

    After you create the service account and functions, specify their IDs.

Create a service account

Using the service account, the function will get access to the secret and YandexGPT, and the API gateway will get access to the function.

  1. In the management console, select the folder where you are going to create your infrastructure.

  2. In the list of services, select Identity and Access Management.

  3. Click Create service account.

  4. Enter the service account name: agent-streamer-sa.

  5. Click Add role and select these roles:

    • serverless.functions.invoker
    • lockbox.payloadViewer
    • api-gateway.websocketWriter
    • ai.languageModels.user

  6. Click Create.

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

  1. Create a service account:

    yc iam service-account create --name agent-streamer-sa

    Result:

    id: ajehqs5gee2e********
folder_id: b1g681qpemb4********
created_at: "2025-07-12T17:53:28.180991864Z"
name: agent-streamer-sa

  2. Assign roles to the service account:

    yc resource-manager folder add-access-binding <folder_name_or_ID> \
  --role serverless.functions.invoker \
  --subject serviceAccount:<service_account_ID>

yc resource-manager folder add-access-binding <folder_name_or_ID> \
  --role lockbox.payloadViewer \
  --subject serviceAccount:<service_account_ID>

yc resource-manager folder add-access-binding <folder_name_or_ID> \
  --role api-gateway.websocketWriter \
  --subject serviceAccount:<service_account_ID>

yc resource-manager folder add-access-binding <folder_name_or_ID> \
  --role ai.languageModels.user \
  --subject serviceAccount:<service_account_ID>

    Result:

    effective_deltas:
  - action: ADD
    access_binding:
      role_id: serverless.functions.invoker
      subject:
        id: ajehqs5gee2e********
        type: serviceAccount

effective_deltas:
  - action: ADD
    access_binding:
      role_id: lockbox.payloadViewer
      subject:
        id: ajehqs5gee2e********
        type: serviceAccount

effective_deltas:
  - action: ADD
    access_binding:
      role_id: api-gateway.websocketWriter
      subject:
        id: ajehqs5gee2e********
        type: serviceAccount

effective_deltas:
  - action: ADD
    access_binding:
      role_id: ai.languageModels.user
      subject:
        id: ajehqs5gee2e********
        type: serviceAccount

To create a service account, use the create REST API method for the ServiceAccount resource or the ServiceAccountService/Create gRPC API call.

To assign the service account the serverless.functions.invoker, lockbox.payloadViewer, api-gateway.websocketWriter, and ai.languageModels.user roles for the folder, use the updateAccessBindings REST API method for the Folder resource or the FolderService/UpdateAccessBindings gRPC API call.

Create an API key

The function will use the API key to get access to YandexGPT.

  1. In the management console, select Identity and Access Management.

  2. Select the agent-streamer-sa service account you created earlier.

  3. In the top panel, click Create new key and select Create API key.

  4. In the Scope field, select the yc.ai.languageModels.execute scope.

  5. Click Create.

  6. Save the ID and secret key to later create the function.

    Alert

    After you close this dialog, the key value will no longer be available.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

Run this command:

yc iam api-key create \
  --service-account-id <service_account_ID> \
  --scopes yc.ai.languageModels.execute

Where:

  • --service-account-id: agent-streamer-sa service account ID.
  • --scopes: Key scopes.

Result:

api_key:
  id: aje3dkdmq2qn********
  service_account_id: ajehqs5gee2e********
  created_at: "2025-07-12T18:00:46.418035313Z"
  scope: yc.ai.languageModels.execute
  scopes:
    - yc.ai.languageModels.execute
secret: AQVNw20bbQtXhfpblT04zJs8Z1wUT5rD********

Save the value of the secret field.

To create an API key, use the create REST API method for the ApiKey resource or the ApiKeyService/Create gRPC API call.

Create a Yandex Lockbox secret

The Yandex Lockbox secret will store the secret key.

  1. In the management console, select Lockbox.
  2. Click Create secret.
  3. In the Name field, specify the secret name: api-key-secret.
  4. In the Secret type field, select Custom.
  5. In the Key field, enter api-key.
  6. In the Value field, paste the secret key you obtained in the previous step.
  7. Click Create.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

Run this command:

yc lockbox secret create \
  --name api-key-secret \
  --payload "[{'key': 'api-key', 'text_value': '<secret_key>'}]"

Where text_value is the API gateway secret key you got in the previous step.

Result:

id: e6q0rdjdggjp********
folder_id: b1g681qpemb4********
created_at: "2025-07-12T18:23:49.844Z"
name: api-key-secret
status: ACTIVE
current_version:
  id: e6qbp772i014********
  secret_id: e6q0rdjdggjp********
  created_at: "2025-07-12T18:23:49.844Z"
  status: ACTIVE
  payload_entry_keys:
    - api-key

To create a secret, use the create REST API method for the Secret resource or the SecretService/Create gRPC API call.

Create a function

The function will be created based on the archive with its code and dependencies.

  1. In the management console, select Cloud Functions.

  2. Create a function:

    1. Click Create function.
    2. In the window that opens, enter agent-streamer as the function name.
    3. Click Create.

  3. Create a function version:

    1. Select Python 3.12 as the runtime environment, disable Add files with code examples, and click Continue.

    2. In the Method field, select ZIP archive and attach the function.zip archive you created earlier.

    3. Specify the entry point: function.handler.

    4. Under Parameters, specify:

      • Timeout: 30 seconds.

      • Memory: 512 MB.

      • Service account: Select the agent-streamer-sa service account.

      • Environment variables:

        • BASE_URL: Yandex AI Studio URL, https://llm.api.cloud.yandex.net/v1.

        • MODEL_NAME: URI of the Yandex AI Studio text generation model.

          For example, gpt://<folder_ID>/yandexgpt/latest, where <folder_ID> is the ID of the folder you are creating the infrastructure in.

        • FOLDER_ID: ID of the folder you are creating the infrastructure in.

      • Lockbox secrets:

        • In the Environment variable field, specify API_KEY and select the previously created api-key-secret, its version, and api-key.

      • If you prefer to opt out of logging so as not to pay for Cloud Logging, disable the Write logs option.

    5. Click Save changes.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

  1. Create a function:

    yc serverless function create --name agent-streamer

    Result:

    id: d4eem33cun2k********
folder_id: b1g681qpemb4********
created_at: "2025-07-12T18:15:59.854Z"
name: agent-streamer
http_invoke_url: https://functions.yandexcloud.net/d4eem33cun2k********
status: ACTIVE

  2. Create a function version:

    yc serverless function version create \
  --function-name agent-streamer \
  --runtime python312 \
  --entrypoint function.handler \
  --memory 512m \
  --execution-timeout 30s \
  --source-path <path_to_archive> \
  --service-account-id <service_account_ID> \
  --environment BASE_URL=https://llm.api.cloud.yandex.net/v1 \
  --environment MODEL_NAME=gpt://<folder_ID>/yandexgpt/latest \
  --environment FOLDER_ID=<folder_ID> \
  --secret name=api-key-secret,key=api-key,environment-variable=API_KEY

    Where:

    • --source-path: Path to function.zip.

    • --service-account-id: agent-streamer-sa service account ID.

    • --environment: Environment variables:

      • BASE_URL: Yandex AI Studio URL, https://llm.api.cloud.yandex.net/v1.

      • MODEL_NAME: URI of the Yandex AI Studio text generation model.

        For example, gpt://<folder_ID>/yandexgpt/latest, where <folder_ID> is the ID of the folder you are creating the infrastructure in.

      • FOLDER_ID: ID of the folder you are creating the infrastructure in.

    • --secret: api-key-secret secret.

    Result:

    id: d4e6d89oi342********
function_id: d4eem33cun2k********
created_at: "2025-07-12T18:27:29.887Z"
runtime: python312
entrypoint: function.handler
resources:
  memory: "536870912"
execution_timeout: 30s
service_account_id: ajehqs5gee2e********
image_size: "59891712"
status: ACTIVE
tags:
  - $latest
environment:
  BASE_URL: https://llm.api.cloud.yandex.net/foundationModels/v1
  FOLDER_ID: b1g681qpemb4********
  MODEL_NAME: gpt://b1g681qpemb4********/yandexgpt/latest
secrets:
  - id: e6q0rdjdggjp********
    version_id: e6qbp772i014********
    key: api-key
    environment_variable: API_KEY
log_options:
  folder_id: b1g681qpemb4********
concurrency: "1"

To create a function, use the create REST API method for the Function resource or the FunctionService/Create gRPC API call.

To create a function version, use the createVersion REST API method for the Function resource or the FunctionService/CreateVersion gRPC API call.

Create an API gateway

Create the API gateway for access to the function.

  1. Open the gateway-spec.yaml file and specify the following:

    • function_id: agent-streamer function ID.
    • service_account_id: agent-streamer-sa service account ID.

  2. Create an API gateway:

    1. In the management console, select API Gateway.
    2. Click Create API gateway.
    3. In the Name field, enter agent-streamer-gateway as the API gateway name.
    4. Under Specification, paste the contents of the gateway-spec.yaml file.
    5. If you prefer to opt out of logging so as not to pay for Cloud Logging, disable the Write logs option.
    6. Click Create.
    7. Select the created API gateway. Save the WebSocket endpoint field value as you will need it at the next step.

    By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

    Run this command:

    yc serverless api-gateway create \
--name agent-streamer-gateway \
--spec <specification_file_path>

    Where --spec is the path to the gateway-spec.yaml file.

    Result:

    id: d5dgs4pn5iil********
folder_id: b1g681qpemb4********
created_at: "2025-07-12T18:34:29.535Z"
name: agent-streamer-gateway
status: ACTIVE
domain: d5dgs4pn5iil********.********.apigw.yandexcloud.net
connectivity: {}
log_options:
  folder_id: b1g681qpemb4********
execution_timeout: 300s

    Save the domain field value as you will need it at the next step.

    To create an API gateway, use the create REST API method for the ApiGateway resource or the ApiGatewayService/Create gRPC API call.

Test the agent

  1. Connect to WebSocket:

    wscat -c wss://<API_gateway_domain>

  2. Send a message:

    Tell me a joke about programming

    Result:

    < Here
<  are 5 jokes about
<  programming:
...

    The actual response text may differ.

How to delete the resources you created

Delete the resources you no longer need to avoid paying for them:

  1. Delete the API gateway.
  2. Delete the function.
  3. Delete the secret.
  4. If you had left the function logging feature on, delete the log group.
Previous
Creating an AI agent with Yandex Cloud Functions
Next
Configuring Postfix to send emails via Yandex Cloud Postbox