Compatibility with OpenAI
The Foundation Models text generation API is partially compatible with the OpenAI API. You can quickly adapt your applications designed to work with OpenAI by changing a few parameters in the query.
Use the Yandex Cloud ML SDK API and library to access all Foundation Models features.
Configuring OpenAI to work with Foundation Models
To use the Foundation Models text generation models in OpenAI libraries, change the basic endpoint and specify the API key:
import openai
client = openai.OpenAI(
api_key="<API_key_value>",
base_url="https://llm.api.cloud.yandex.net/v1"
)
import OpenAI from "openai";
const openai = new OpenAI(
api_key="<API_key_value>",
base_url="https://llm.api.cloud.yandex.net/v1");
How to get an API key for Foundation Models.
Example of a query to a model
Before sending the query, in the model URI, specify the ID of the folder you got the API key in.
# Install OpenAI SDK using pip
# pip install openai
import openai
client = openai.OpenAI(
api_key="<API_key_value>",
base_url="https://llm.api.cloud.yandex.net/v1"
)
response = client.chat.completions.create(
model="gpt://<folder_ID>/yandexgpt/latest",
messages=[
{"role": "assistant", "content": "You are a very smart assistant."},
{"role": "user", "content": "How much does a query to YandexGPT Pro cost?"}
],
max_tokens=10000,
temperature=0.7,
stream=True
)
for chunk in response:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="")
import OpenAI from "openai";
const openai = new OpenAI(
api_key="<API_key_value>",
base_url="https://llm.api.cloud.yandex.net/v1");
async function main() {
const completion = await openai.chat.completions.create({
messages: [{"role": "assistant", "content": "You are a very smart assistant."},
{"role": "user", "content": "How much does a query to YandexGPT Pro cost?"}],
model: "gpt://<folder_ID>/yandexgpt/latest",
});
console.log(completion.choices[0]);
}
main();
curl https://llm.api.cloud.yandex.net/v1/chat/completions
-H "Content-Type: application/json"
-H "Authorization: Bearer <API_key>"
-d '{
"model": "gpt://<folder_ID>/yandexgpt/latest",
"messages": [
{
"role": "system",
"content": "You are a very smart assistant."
},
{
"role": "user",
"content": "How much does a query to YandexGPT Pro cost?"
}
]
}'
Current limitations
Foundation Models is partially compatible with the OpenAI API. If not using the OpenAI SDK yet, we recommend you to build your apps with Yandex Cloud ML SDK or the Foundation Models API right from the start.