Sending requests to fine-tuned models
Written by
Updated at June 25, 2024
You can send requests to fine-tuned models in YandexGPT. Requests to the following models are supported:
- YandexGPT model fine-tuned for summarizing texts.
- Fine-tuned foundation models in Yandex DataSphere.
Getting started
Get API authentication credentials as described in Authentication with the Yandex Foundation Models API.
Send a request for text summarization
-
Create a file with the request body, e.g.,
body.json
:{ "modelUri": "gpt://<folder_ID>/summarization", "completionOptions": { "stream": false, "temperature": 0.1, "maxTokens": "2000" }, "messages": [ { "role": "user", "text": "<article_text>" } ] }
-
modelUri
: ID of the model to generate the response. The parameter contains the ID of a Yandex Cloud folder or the ID of a model fine-tuned in DataSphere. -
completionOptions
: Request configuration options:stream
: Enables streaming of partially generated text. It may take either thetrue
orfalse
value.temperature
: With a higher temperature, you get more creative and randomized response from the model. This parameter accepts values between0
and1
, inclusive. The default value is0.3
.maxTokens
: Sets a limit on the model's output in tokens. The maximum number of tokens per generation depends on the model. For more information, see Quotas and limits in Yandex Foundation Models.
-
messages
: List of messages that set the context for the model:-
role
: Message sender's role:user
: Used to send user messages to the model.system
: Used to set request context and define the model's behavior.assistant
: Used for responses generated by the model. In chat mode, the model's responses tagged with theassistant
role are included in the message to save the conversation context. Do not send user messages with this role.
-
text
: Text content of the message.
-
-
-
To send the request to the model, run this command:
curl --request POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer <IAM_token>" \ -d "@<path_to_json_file>" \ "https://llm.api.cloud.yandex.net/foundationModels/v1/completion"
Where:
<path_to_json_file>
: Relative or absolute path to the file containing the body of request to the model.<IAM_token>
: Value of the IAM token you get before you start.
Here is an example result for the SSH protocol article:
{ "result": { "alternatives": [ { "message": { "role": "assistant", "text": "- SSH is a network protocol for secure interaction between computers.\n - SSH is used for connecting to remote servers, executing commands, copying and editing files.\n - SSH is widely used in corporate infrastructure for installing updates and managing business-critical systems.\n - SSH support is built into Unix and Linux operating systems, as well as in Windows with free SSH clients.\n - SSH operation involves an SSH server and an SSH client, which establish a secure connection for transmitting encrypted data.\n - The reliability of SSH is ensured by user authentication and processing of transmitted data.\n - The SSH server supports three authentication methods: by IP address, by login/password, and by key.\n - SSH keys provide a high degree of protection and convenience for use in scripts to automate routine operations." }, "status": "ALTERNATIVE_STATUS_FINAL" } ], "usage": { "inputTextTokens": "1609", "completionTokens": "168", "totalTokens": "1777" }, "modelVersion": "06.12.2023" } }