Yandex Cloud
Search
Contact UsGet started
  • Blog
  • Pricing
  • Documentation
  • All Services
  • System Status
    • Featured
    • Infrastructure & Network
    • Data Platform
    • Containers
    • Developer tools
    • Serverless
    • Security
    • Monitoring & Resources
    • ML & AI
    • Business tools
  • All Solutions
    • By industry
    • By use case
    • Economics and Pricing
    • Security
    • Technical Support
    • Customer Stories
    • Gateway to Russia
    • Cloud for Startups
    • Education and Science
  • Blog
  • Pricing
  • Documentation
Yandex project
© 2025 Yandex.Cloud LLC
Yandex Foundation Models
    • Overview
    • API authentication
        • Overview
          • Overview
          • Tokenize
          • TokenizeCompletion
  • Yandex Cloud ML SDK
  • Compatibility with OpenAI
  • Access management
  • Pricing policy
  • Public materials
  • Release notes
  1. API references
  2. Text Generation API
  3. REST
  4. Tokenizer
  5. Overview

Foundation Models Text Generation API, REST: Tokenizer

Written by
Yandex Cloud
Updated at October 17, 2024

Service for tokenizing input content.

Methods

Method

Description

Tokenize

RPC method for tokenizing text.

TokenizeCompletion

RPC method for tokenizing content of CompletionRequest

Was the article helpful?

Previous
Completion
Next
Tokenize
Yandex project
© 2025 Yandex.Cloud LLC