SpeechSense

Yandex SpeechSense is a speech analytics platform within Stackland. It transcribes and analyzes audio recordings of conversations, extracts text, classifies customer interactions, and generates analytical reports.

SpeechSense does not come with the basic Stackland package and requires a separate license.

ArchitectureArchitecture

SpeechSense comprises three components:

• YaGPT: Handles natural language processing and text generation. It is used to create conversation summaries and classify interactions. Requires GPU resources.
• SpeechKit: Performs speech recognition and synthesis, converts audio files to text. Requires GPU resources.
• SpeechSense: Provides speech analytics, data processing, and the web UI. SpeechSense orchestrates the YaGPT and SpeechKit workflows and features a UI for managing analysis results.

Infrastructure requirementsInfrastructure requirements

GPUGPU

SpeechSense requires the following NVIDIA® GPU nodes:

• YaGPT Pro: 2 × NVIDIA® H100
• SpeechKit STT Backend: 1 × NVIDIA® A100
• SpeechKit Embeddings: 1 × NVIDIA® H100

Before installing SpeechSense, enable the NVIDIA® GPU support component.

ResourcesResources

TA services operate without a GPU but require significant CPU and RAM resources. We recommend allocating at least 32 vCPUs and 64 GB RAM for TA services.

DependenciesDependencies

SpeechSense relies on the following Stackland components:

• Managed Service for PostgreSQL: Metadata and state storage.
• Managed Service for ClickHouse®: Analytical queries and storage of large-scale data.
• Managed Service for Apache Kafka®: Streaming data processing.
• Object Storage: Storage of audio files and models.
• NVIDIA® GPU support: GPU resource management.
• Identity and Access Management: User authentication and authorization.

When SpeechSense is enabled, the controller automatically checks for these dependencies and provisions the required database clusters, Apache Kafka® topics, and certificates.

ConfigurationConfiguration

To manage SpeechSense, use the SpeechsenseConfig custom resource.

Here is an example:

apiVersion: stackland.yandex.cloud/v1alpha1
kind: SpeechsenseConfig
metadata:
  name: default
spec:
  enabled: true
  settings:
    s3: # Optional. Add data if you need an external storage
      endpoint: "<object_storage_address>"
      accessKeyID: "<key_ID>"
      secretAccessKey: "<secret_key>"

Where:

• enabled: Enables and disables the component.
• settings.s3.endpoint: Object Storage address.
• settings.s3.accessKeyID: Storage access key ID.
• settings.s3.secretAccessKey: Storage secret access key.

See alsoSee also

• Uploading SpeechSense images: Pulling SpeechSense images.
• Configuring and activating SpeechSense: Configuring and activating SpeechSense.
• Disabling SpeechSense: Disabling SpeechSense.
• NVIDIA® GPU support: GPU management in a cluster.