Contact UsGet started

Foundation Models Text Generation API, gRPC: TokenizerService.TokenizeCompletion

Written by
Updated at November 26, 2024

RPC method for tokenizing content of CompletionRequest

gRPC request

rpc TokenizeCompletion (CompletionRequest) returns (TokenizeResponse)

CompletionRequest

{
  "model_uri": "string",
  "completion_options": {
    "stream": "bool",
    "temperature": "google.protobuf.DoubleValue",
    "max_tokens": "google.protobuf.Int64Value"
  },
  "messages": [
    {
      "role": "string",
      // Includes only one of the fields `text`, `tool_call_list`, `tool_result_list`
      "text": "string",
      "tool_call_list": {
        "tool_calls": [
          {
            // Includes only one of the fields `function_call`
            "function_call": {
              "name": "string",
              "arguments": "google.protobuf.Struct"
            }
            // end of the list of possible fields
          }
        ]
      },
      "tool_result_list": {
        "tool_results": [
          {
            // Includes only one of the fields `function_result`
            "function_result": {
              "name": "string",
              // Includes only one of the fields `content`
              "content": "string"
              // end of the list of possible fields
            }
            // end of the list of possible fields
          }
        ]
      }
      // end of the list of possible fields
    }
  ],
  "tools": [
    {
      // Includes only one of the fields `function`
      "function": {
        "name": "string",
        "description": "string",
        "parameters": "google.protobuf.Struct"
      }
      // end of the list of possible fields
    }
  ]
}

Request for the service to generate text completion.

Field

Description

model_uri

string

The ID of the model to be used for completion generation.

completion_options

CompletionOptions

Configuration options for completion generation.

messages[]

Message

A list of messages representing the context for the completion model.

tools[]

Tool

List of tools that are available for the model to invoke during the completion generation.
Note: This parameter is not yet supported and will be ignored if provided.

CompletionOptions

Defines the options for completion generation.

Field

Description

stream

bool

Enables streaming of partially generated text.

temperature

google.protobuf.DoubleValue

Affects creativity and randomness of responses. Should be a double number between 0 (inclusive) and 1 (inclusive).
Lower values produce more straightforward responses while higher values lead to increased creativity and randomness.
Default temperature: 0.3

max_tokens

google.protobuf.Int64Value

The limit on the number of tokens used for single completion generation.
Must be greater than zero. This maximum allowed parameter value may depend on the model being used.

Message

A message object representing a wrapper over the inputs and outputs of the completion model.

Field

Description

role

string

The ID of the message sender. Supported roles:

  • system: Special role used to define the behaviour of the completion model.
  • assistant: A role used by the model to generate responses.
  • user: A role used by the user to describe requests to the model.

text

string

Textual content of the message.

Includes only one of the fields text, tool_call_list, tool_result_list.

Message content.

tool_call_list

ToolCallList

List of tool calls made by the model as part of the response generation.

Includes only one of the fields text, tool_call_list, tool_result_list.

Message content.

tool_result_list

ToolResultList

List of tool results returned from external tools that were invoked by the model.

Includes only one of the fields text, tool_call_list, tool_result_list.

Message content.

ToolCallList

Represents a list of tool calls.

Field

Description

tool_calls[]

ToolCall

A list of tool calls to be executed.

ToolCall

Represents a call to a tool.

Field

Description

function_call

FunctionCall

Represents a call to a function.

Includes only one of the fields function_call.

FunctionCall

Represents the invocation of a function with specific arguments.

Field

Description

name

string

The name of the function being called.

arguments

google.protobuf.Struct

The structured arguments passed to the function.
These arguments must adhere to the JSON Schema defined in the corresponding function's parameters.

ToolResultList

Represents a list of tool results.

Field

Description

tool_results[]

ToolResult

A list of tool results.

ToolResult

Represents the result of a tool call.

Field

Description

function_result

FunctionResult

Represents the result of a function call.

Includes only one of the fields function_result.

FunctionResult

Represents the result of a function call.

Field

Description

name

string

The name of the function that was executed.

content

string

The result of the function call, represented as a string.
This field can be used to store the output of the function.

Includes only one of the fields content.

Tool

Represents a tool that can be invoked during completion generation.

Field

Description

function

FunctionTool

Represents a function that can be called.

Includes only one of the fields function.

FunctionTool

Represents a function tool that can be invoked during completion generation.

Field

Description

name

string

The name of the function.

description

string

A description of the function's purpose or behavior.

parameters

google.protobuf.Struct

A JSON Schema that defines the expected parameters for the function.
The schema should describe the required fields, their types, and any constraints or default values.

TokenizeResponse

{
  "tokens": [
    {
      "id": "int64",
      "text": "string",
      "special": "bool"
    }
  ],
  "model_version": "string"
}

Response containing tokenized content from request.

Field

Description

tokens[]

Token

A list of tokens obtained from tokenization.

model_version

string

Model version (changes with model releases).

Token

Represents a token, the basic unit of content, used by the foundation model.

Field

Description

id

int64

An internal token identifier.

text

string

The textual representation of the token.

special

bool

Indicates whether the token is special or not. Special tokens may define the model's behavior and are not visible to users.
Previous
Tokenize
Next
Overview