Yandex Cloud
Search
Contact UsGet started
  • Pricing
  • Customer Stories
  • Documentation
  • Blog
  • All Services
  • System Status
    • Featured
    • Infrastructure & Network
    • Data Platform
    • Containers
    • Developer tools
    • Serverless
    • Security
    • Monitoring & Resources
    • AI Studio
    • Business tools
  • All Solutions
    • By industry
    • By use case
    • Economics and Pricing
    • Security
    • Technical Support
    • Start testing with double trial credits
    • Cloud credits to scale your IT product
    • Gateway to Russia
    • Cloud for Startups
    • Education and Science
    • Yandex Cloud Partner program
  • Pricing
  • Customer Stories
  • Documentation
  • Blog
© 2025 Direct Cursus Technology L.L.C.
Yandex Foundation Models
    • Overview
    • API authentication
        • Overview
          • Overview
          • Tokenize
          • TokenizeCompletion
  • Yandex Cloud ML SDK
  • Compatibility with OpenAI
  • Access management
  • Pricing policy
  • Audit Trails events
  • Public materials
  • Release notes
  1. API references
  2. Text Generation API
  3. REST
  4. Tokenizer
  5. Overview

Foundation Models Text Generation API, REST: Tokenizer

Written by
Yandex Cloud
Updated at October 17, 2024

Service for tokenizing input content.

MethodsMethods

Method

Description

Tokenize

RPC method for tokenizing text.

TokenizeCompletion

RPC method for tokenizing content of CompletionRequest

Was the article helpful?

Previous
Completion
Next
Tokenize
© 2025 Direct Cursus Technology L.L.C.