Contact UsTry it for free
© 2025 Direct Cursus Technology L.L.C.

Working with an AI assistant in SpeechSense

Written by
Updated at December 12, 2025

You can use an AI assistant to retrieve information from a dialog by connecting it to audio conversations and chats. This tutorial explains how to connect an agent to chats and uses a conversation between a travel agency and a customer as an example.

To retrieve information you need from a chat using an AI assistant:

  1. Get your cloud ready.
  2. Set up the infrastructure.
  3. Create an AI assistant.
  4. Upload data to the project.
  5. Review the AI assistant's output in the SpeechSense interface.
  6. Get the dialog information using the API.

If you no longer need the resources you created, delete them.

Get your cloud ready

Sign up for Yandex Cloud and create a billing account:

  1. Navigate to the management console and log in to Yandex Cloud or create a new account.
  2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can navigate to the cloud page to create or select a folder for your infrastructure.

Learn more about clouds and folders here.

The user must have the billing.accounts.editor, billing.accounts.admin, or billing.accounts.owner role for the billing account to manage it.

Note

If you are unable to manage roles, contact your cloud or organization administrator.

  • SpeechSense: Number of characters in each text conversation (see SpeechSense pricing).
  • AI assistant: Text generation (see Yandex AI Studio pricing). Charges for using the assistant will apply as soon as you enable it.

Set up your infrastructure

Create a service account

  1. In the management console, select the relevant folder.
  2. In the list of services, select Identity and Access Management.
  3. Click Create service account.
  4. Name the service account, e.g., speechsense.
  5. Click Create.

Create an API key for the service account

  1. In the management console, select Identity and Access Management.

  2. In the left-hand panel, select Service accounts.

  3. Select the speechsense service account.

  4. In the top panel, click Create new key and select Create API key.

  5. In the Scope field, select yc.speech-sense.use.

  6. Click Create.

  7. Save the ID and secret key for later.

    Alert

    After you close this dialog, the key value will no longer be available.

Create a space and link a billing account

  1. Open the SpeechSense home page.
  2. Click Create space.
  3. Enter a name for the space.
  4. Click Create.
  5. Link a billing account to the space to pay for SpeechSense.

Add the service account to the space

  1. Navigate to the new space in the SpeechSense interface.
  2. Click Add participant → Add from organization.
  3. Copy the ID of the speechsense service account you created earlier and paste it to the search bar.
  4. Select the speechsense service account and specify the Data editor role. This role will allow the service account to upload data to SpeechSense.
  5. Click Add.

Create a connection for a chat

  1. Open the SpeechSense home page.
  2. Go to the space of your choice.
  3. Navigate to the Connections tab.
  4. Click Create connection → Blank form.
  5. Specify a connection name, e.g., chats-default-metadata.
  6. Select the Сhat data type.
  7. Click Create connection.
  8. On the page that opens, click ID in the top-left corner to copy the ID of the connection you created. Save this ID as you will need it later to upload data to the project.

Create a project

  1. Open the SpeechSense home page.
  2. Go to the space of your choice.
  3. Click Create project.
  4. Enter a project name, e.g., chats-travel-agency.
  5. Under Connection, click Add connection and select the chats-default-metadata connection.
  6. Click Create project.
  7. On the page that opens, click ID in the top-left corner to copy the ID of the project you created. Save this ID as you will need it later to work with the API.

Create an AI assistant

  1. Open the SpeechSense home page.

  2. Go to the space you need and select the chats-travel-agency project.

  3. Select the Assistants tab.

  4. Click Create assistant.

  5. Configure the new assistant:

    1. Enter Dialog subject as the name.

    2. Under Instructions, enter this prompt:

      You are analyzing a conversation between a call center agent and customer. Pay close attention when generating a response.
Determine the subject of the customer’s inquiry by selecting one from this list: Visa, Country, Guided Tours.
If nothing of the above is suitable, put "Other"."

    3. Select the YandexGPT 5.1 Pro model.

    4. Under Output, specify:

      • Field name: Subject.
      • Field type: String.
      • Description: Most suitable subject.

  6. Enable the Enable assistant section to activate the assistant.

  7. Click Create.

Upload data to the project

To use the Yandex Cloud API, you will need Git, Python 3.6 or higher, and the grpcio-tools package. Learn how to install Python.

  1. Clone the cloudapi repository:

    cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi

    Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

  2. Install the grpcio-tools package using the pip package manager:

    pip install grpcio-tools

  3. Go to the directory hosting the Yandex Cloud API repository, create a directory named upload_data, and generate the client interface code in it. Then, go to the upload_data directory:

    cd ~/cloudapi/ && \
mkdir upload_data && \
python3 -m grpc_tools.protoc -I . -I third_party/googleapis \
     --python_out=./upload_data/ \
     --grpc_python_out=./upload_data/ \
     google/api/http.proto \
     google/api/annotations.proto \
     yandex/cloud/api/operation.proto \
     google/rpc/status.proto \
     yandex/cloud/operation/operation.proto \
     yandex/cloud/validation.proto \
     yandex/cloud/speechsense/v1/*.proto \
     yandex/cloud/speechsense/v1/*/*.proto
cd upload_data

  4. In the upload_data folder, create the upload_text.py Python script to upload the chat conversation to SpeechSense:

    import argparse
import json
from typing import Dict
import grpc

from yandex.cloud.speechsense.v1 import talk_service_pb2
from yandex.cloud.speechsense.v1 import talk_service_pb2_grpc
from yandex.cloud.speechsense.v1 import text_pb2
from google.protobuf.timestamp_pb2 import Timestamp

def upload_talk(connection_id: str, metadata: Dict[str, str], api_key: str, text_data):
   credentials = grpc.ssl_channel_credentials()
   channel = grpc.secure_channel('api.speechsense.yandexcloud.net:443', credentials)

   talk_service_stub = talk_service_pb2_grpc.TalkServiceStub(channel)

   messageList = []
   for message in text_data['messages']:
      timestamp = Timestamp()
      timestamp.FromJsonString(value=str(message['timestamp']))
      messageProto = text_pb2.Message(
         user_id=str(message['user_id']),
         text=text_pb2.TextPayload(text=str(message['text'])),
         timestamp=timestamp
      )
      messageList.append(messageProto)

   # Forming a request to the API
   request = talk_service_pb2.UploadTextRequest(
      metadata=talk_service_pb2.TalkMetadata(
         connection_id=str(connection_id),
         fields=metadata),
      text_content=text_pb2.TextContent(
         messages=messageList)
   )

   # Authentication type: API key
   response = talk_service_stub.UploadText(request, metadata=(
      ('authorization', f'Api-Key {api_key}'),
   ))

   # Displaying the dialog ID
   print(f'Dialog ID: {response.talk_id}')

if __name__ == '__main__':
   parser = argparse.ArgumentParser()
   parser.add_argument('--key', required=True, help='API key', type=str)
   parser.add_argument('--connection-id', required=True, help='Connection ID', type=str)
   parser.add_argument('--text-path', required=True, help='JSON with text chat data', type=str)
   parser.add_argument('--meta-path', required=False, help='JSON with the dialog metadata', type=str, default=None)
   args = parser.parse_args()

   with open(args.meta_path, 'r') as fp:
      metadata = json.load(fp)

   with open(args.text_path, 'r') as fp:
      text_data = json.load(fp)
   upload_talk(args.connection_id, metadata, args.key, text_data)

  5. In the upload_data folder, create a file named metadata.json with your conversation metadata:

    {
   "operator_name": "Sarah",
   "operator_id": "24",
   "client_name": "Michael",
   "client_id": "145",
   "date": "2025-12-01T12:34:19+03:00",
   "direction_outgoing": "false",
   "language": "ru-ru"
}

  6. In the upload_data folder, create a file named chat.json with your text messages:

    {
   "messages": [
      {
         "user_id": 145,
         "text" : "Hi there! Do you sell guided tours?",
         "timestamp" : "2025-12-01T12:34:19+03:00"
      },
      {
         "user_id": 24,
         "text" : "Hi Michael. Yes, we are offering guided tours to Europe and Asia. What destinations are you interested in?",
         "timestamp" : "2025-12-01T12:35:45+03:00"
      },
      {
         "user_id": 145,
         "text" : "Which Asian countries can I visit without a visa?",
         "timestamp" : "2025-12-01T12:37:14+03:00"
      },
      {
         "user_id": 24,
         "text" : "South Korea.",
         "timestamp" : "2025-12-01T12:38:55+03:00"
      },
      {
         "user_id": 145,
         "text" : "Thank you, I’ll think about it. Bye.",
         "timestamp" : "2025-12-01T12:41:21+03:00"
      },
      {
         "user_id": 24,
         "text" : "Goodbye.",
         "timestamp" : "2025-12-01T12:41:50+03:00"
      }
   ]
}

  7. Specify the service account's API key:

    export API_KEY=<secret_part_of_API_key>

  8. Run the upload_text.py script:

    python3 upload_text.py \
   --text-path chat.json \
   --meta-path metadata.json \
   --connection-id <connection_ID> \
   --key ${API_KEY}

    Result:

    Dialog ID: audh5bbnrstk********

    Save the ID of the dialog you created as you will need it later to work with the API.

Review the AI assistant's output in the SpeechSense interface

To view the dialog subject selected by the AI assistant:

  1. Open the SpeechSense home page.
  2. Go to the space you need and select the chats-travel-agency project.
  3. On the Dialogs tab, enable Assistants.

Below the dialog line, you will see a line indicating the Guided Tours subject. This is the subject the AI assistant has selected as the most suitable one for the dialog.

Get the dialog information using the API

Get the AI assistant and its field IDs

Call the Assistants.List method and run the following request, e.g., via cURL:

curl \
    --request POST \
    --header "Authorization: Api-Key $API_KEY" \
    --url 'https://rest-api.speechsense.yandexcloud.net/speechsense/v1/assistants/list' \
    --data '{
              "projectId": "<project_ID>"
            }'

Result:

{
  "assistants": [
    {
      "id": "eagdf4es87u0********",
      "project_id": "eags34o1skdi********",
      "model_id": "YGPT_LITE",
      "name": "Dialog subject",
      "description": "",
      "prompt": "You are analyzing a conversation between a call center agent and customer. Pay close attention when generating a response. \nDetermine the subject of the customer’s inquiry by selecting one from this list: Visa, Country, Guided Tours. \nIf nothing of the above is suitable, put "Other"."
      "fields": [
        {
          "id": "eagldsqk9fp4********",
          "name": "Subject",
          "description": "Most suitable subject.",
          "type": "ASSISTANT_FIELD_TYPE_STRING"
        }
      ],
      "enabled": true,
      "labels": {},
      "created_at": "2025-11-13T23:38:00.686738Z",
      "created_by": "ajegrmkclceh********",
      "modified_at": "2025-11-14T01:32:48.358727Z",
      "modified_by": "ajegrmkclceh********"
    }
  ]
}

Call the AssistantsService.List method, e.g., via the following gRPCurl request:

grpcurl \
  -format json \
  -import-path ~/cloudapi/ \
  -import-path ~/cloudapi/third_party/googleapis/ \
  -proto ~/cloudapi/yandex/cloud/speechsense/v1/assistants_service.proto \
  -rpc-header "Authorization: Api-Key $API_KEY" \
  -d '{
        "project_id": "<project_ID>"
      }' \
  api.speechsense.yandexcloud.net:443 \
  yandex.cloud.speechsense.v1.AssistantsService.List

Result:

{
  "assistants": [
    {
      "id": "eagdf4es87u0********",
      "projectId": "eags34o1skdi********",
      "modelId": "YGPT_LITE",
      "name": "Dialog subject",
      "prompt": "You are analyzing a conversation between a call center agent and customer. Pay close attention when generating a response. \nDetermine the subject of the customer’s inquiry by selecting one from this list: Visa, Country, Guided Tours. \nIf nothing of the above is suitable, put "Other"."
      "fields": [
        {
          "id": "eagldsqk9fp4********",
          "name": "Subject",
          "description": "Most suitable subject.",
          "type": "ASSISTANT_FIELD_TYPE_STRING"
        }
      ],
      "enabled": true,
      "createdAt": "2025-11-13T23:38:00.686738Z",
      "createdBy": "ajegrmkclceh********",
      "modifiedAt": "2025-11-14T01:32:48.358727Z",
      "modifiedBy": "ajegrmkclceh********"
    }
  ]
}

Save the values of these fields:

  • assistants.id: ID of the Dialog subject AI assistant.
  • assistants.fields.id: ID of the Subject field.

Get the dialog information

Call the Talk.Get method, e.g., via the following cURL request:

curl \
    --request POST \
    --header "Authorization: Api-Key $API_KEY" \
    --url 'https://rest-api.speechsense.yandexcloud.net/speechsense/v1/talks/get' \
    --data '{
              "projectId": "<project_ID>",
              "talkIds": [
                "string"
              ],
              "resultsMask": 
                {
                  "paths": [
                    "assistants"
                  ]
                }
            }'

Result:

{
  "talk": [
    {
      "id": "audh5bbnrstk********",
      "organization_id": "********",
      "space_id": "f3fuclf1kufs********",
      "connection_id": "eagojm8e3bfv********",
      "project_ids": [
        "eags34o1skdi********"
      ],
      ...
      "assistants": {
        "assistant_results": [
          {
            "assistant_id": "eagdf4es87u0********",
            "results": [
              {
                "field_id": "eagldsqk9fp4********",
                "string_result": "Guided Tours"
              }
            ]
          }
        ]
      },
      ...
    }
  ]
}

To find the dialog subject value, search the result by the previously saved AI assistant and field IDs. This value is indicated in the talk.assistants.assistant_results.results.string_result field: Guided Tours.

Call the TalkService.Get method, e.g., via the following gRPCurl request:

grpcurl \
    -format json \
    -import-path ~/cloudapi/ \
    -import-path ~/cloudapi/third_party/googleapis/ \
    -proto ~/cloudapi/yandex/cloud/speechsense/v1/talk_service.proto \
    -rpc-header "Authorization: Api-Key $API_KEY" \
    -d '{
          "project_id": "<project_ID>",
          "talk_ids": [
            <dialog_ID>
          ],
          "results_mask": 
            {
              "paths": [
                "assistants"
              ]
            }
        }' \
    api.speechsense.yandexcloud.net:443 \
    yandex.cloud.api.speechsense.v1.TalkService.Get

Result:

{
  "talk": [
    {
      "id": "audh5bbnrstk********",
      "organizationId": "********",
      "spaceId": "f3fuclf1kufs********",
      "connectionId": "eagojm8e3bfv********",
      "projectIds": [
        "eags34o1skdi********"
      ],
      ...
      "assistants": {
        "assistantResults": [
          {
            "assistantId": "eagdf4es87u0********",
            "results": [
              {
                "fieldId": "eagldsqk9fp4********",
                "stringResult": "Guided Tours"
              }
            ]
          }
        ]
      },
      ...
    }
  ]
}

To find the dialog subject value, search the result by the previously saved AI assistant and field IDs. This value is indicated in the talk.assistants.assistantResults.results.stringResult field: Guided Tours.

Delete resources

Some resources are not free of charge. To stop paying for them, delete the SpeechSense project if you no longer need it. The AI assistant will be deleted when you delete the project.

Previous
SpeechSense integration with Bitrix24
Next
Integrating generative models into Visual Studio Code as a coding assistant