Text generation overview
Yandex Foundation Models provides access to large text models which can quickly generate text content, e.g., product descriptions, articles, news stories, newsletters, blog posts, and many other things. The quality of the neural network's response depends directly on the accuracy of the instructions you provide. With a more specific prompt, you are more likely to get the result you expect.
YandexGPT models understand prompts in more than 20 languages, including English and Japanese; however, Russian texts are its first priority. In addition to a text description, prompts must contain a special parameter called temperature that determines the variability of the model's response: the higher the temperature value, the less predictable the model's output is going to be.
Foundation Models also provides access to the Llama 3.1 8b and Llama 3.1 70b models.
To interact with text generation models in Yandex Cloud, there are two interfaces available. You can submit requests to AI Playground or integrate the model into your applications using the API. You can use the REST and gRPC interfaces for integration. You can also use the API to work with models in asynchronous mode. You can view the examples of working with YandexGPT via the API in Step-by-step guides for YandexGPT API.
AI Playground is a good option for introduction and testing: use it to submit synchronous requests to different models, set up parameters, and choose prompts. When communicating, the model saves the dialog context, but you can also create a new experiment if you need to change the context. Additionally, YandexGPT Playground interfaces in chat format or prompt mode are available in the left-hand navigation menu. Use them when you need to fully repeat the model's behavior via the API and you do not want to save the results of the dialog.
To use YandexGPT models, you need the ai.languageModels.user role or higher for the folder.
AI Playground is a good option for introduction and testing: use it to submit synchronous requests to different models, set up parameters, and choose prompts. When communicating, the model saves the dialog context, but you can also create a new experiment if you need to change the context. Additionally, YandexGPT Playground interfaces in chat format or prompt mode are available in the left-hand navigation menu. Use them when you need to fully repeat the model's behavior via the API and you do not want to save the results of the dialog.
For more information, see Text generation models.
Formatting of model responses
By default, the model returns a response formatted using Markdown. Use the prompt text to get a response with additional formatting, e.g., with emoji, or in a different format, e.g., JSON, XML, etc.
Here is an example:
{
"modelUri": "gpt://<folder_ID>/yandexgpt/latest",
"completionOptions": {
"stream": false,
"temperature": 0.6,
"maxTokens": "2000"
},
"messages": [
{
"role": "system",
"text": "You are a smart assistant."
},
{
"role": "user",
"text": "Name any three groups of products one can find in a grocery store. For each group, provide three subgroups. Present the result as a JSON object, where each group of products is represented by a key in the JSON object, and arrays from the relevant subgroups are the values. No introductory phrases or explanations needed, just data. Do not use Markdown."
}
]
}
Result:
{
"result": {
"alternatives": [
{
"message": {
"role": "assistant",
"text": "{\n \"meat\": [\"beef\", \"pork\", \"mutton\"],\n \"dairy products\": [\"milk\", \"curd\", \"sour cream\"],\n \"fruit\": [\"apples\", \"bananas\", \"oranges\"]\n}"
},
"status": "ALTERNATIVE_STATUS_FINAL"
}
],
"usage": {
"inputTextTokens": "87",
"completionTokens": "58",
"totalTokens": "145"
},
"modelVersion": "07.03.2024"
}
}
The model returned a response in JSON format with line breaks replaced with \n and quotation marks escaped.
If you do not get the result you expect using the prompt, try fine-tuning the model.
1 Llama was created by Meta. Meta is designated as an extremist organization and its activities are prohibited in Russia.