Getting started with YandexGPT Lite and YandexGPT Pro

To use the examples of requests using SDK:

1. Create a service account and assign the ai.languageModels.user role to it.

2. Get the service account API key and save it.

   The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.

3. Use the pip package manager to install the ML SDK library:

   pip install yandex-cloud-ml-sdk
   

To run sample requests using the API, install cURL.

To work with the YandexGPT API, you need to get authenticated using your account:

1. Get an IAM token: see the guide for a Yandex account or federated account.

2. Get the ID of the folder for which your account has the ai.languageModels.user role or higher.

3. When accessing YandexGPT Lite or YandexGPT Pro via the API, provide the received parameters:

   • In the request file, specify the folder ID in the modelUri parameter.
   • In the request, specify the IAM token in the Authorization header.

   Authorization: Bearer <IAM_token>
   

For information about other API authentication methods, see Authentication with the Yandex Foundation Models API.

1. Create a file named generate-text.py and paste the following code into it:

   #!/usr/bin/env python3
   
   from __future__ import annotations
   from yandex_cloud_ml_sdk import YCloudML
   
   messages = [
       {
           "role": "system",
           "text": "Find errors in the text and correct them",
       },
       {
           "role": "user",
           "text": """Laminate flooring is sutiable for instalation in the kitchen or in a child's 
   room. It withsatnds moisturre and mechanical dammage thanks to 
   a 0.2 mm thick proctive layer of melamine films and 
   a wax-treated interlocking system.""",
       },
   ]
   
   
   def main():
       sdk = YCloudML(
           folder_id="<folder_ID>",
           auth="<API_key>",
       )
   
       result = (
           sdk.models.completions("yandexgpt").configure(temperature=0.5).run(messages)
       )
   
       for alternative in result:
           print(alternative)
   
   
   if __name__ == "__main__":
       main()
   

   Where:

   Note

   As input data for a request, Yandex Cloud ML SDK can accept a string, a dictionary, an object of the TextMessage class, or an array containing any combination of these data types. For more information, see Yandex Cloud ML SDK usage.

   • messages: List of messages that set the context for the model:

     • role: Message sender's role:

       • user: Used for sending user messages to the model.
       • system: Used to set the query context and define the model's behavior.
       • assistant: Used for responses generated by the model. In chat mode, the model's responses tagged with the assistant role are included in the message to save the conversation context. Do not send user messages with this role.

     • text: Message text.

   • <folder_ID>: ID of the folder in which the service account was created.

   • <API_key>: Service account API key you got earlier required for authentication in the API.

     The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.

   For more information about accessing a specific model version, see Accessing models.

2. Run the created file:

   python3 generate-text.py
   

   Result:

   Alternative(role='assistant', text='Laminate flooring is suitable for installation in the kitchen or in a child's room. It withstands moisture and mechanical damage thanks to a 0.2 mm thick protective layer of melamine films and a wax-treated interlocking system.', status=<AlternativeStatus.FINAL: 3>)
   

1. Create a file with the request body, e.g., prompt.json:

   {
     "modelUri": "gpt://<folder_ID>/yandexgpt",
     "completionOptions": {
       "stream": false,
       "temperature": 0.6,
       "maxTokens": "2000",
       "reasoningOptions": {
         "mode": "DISABLED"
       }
     },
     "messages": [
       {
         "role": "system",
         "text": "Find errors in the text and correct them."
       },
       {
         "role": "user",
         "text": "Laminate flooring is sutiable for instalation in the kitchen or in a child's room. It withsatnds moisturre and mechanical dammage thanks to a 0.2 mm thick proctive layer of melamine films and a wax-treated interlocking systme."
       }
     ]
   }
   

   Where:

   • modelUri: ID of the model that will be used to generate the response. The parameter contains the Yandex Cloud folder ID or the tuned model's ID.

   • completionOptions: Request configuration options:

     • stream: Enables streaming of partially generated text. It can either be true or false.

     • temperature: With a higher temperature, you get more creative and randomized responses from the model. Its values range from 0 to 1, inclusive. The default value is 0.3.

     • maxTokens: Sets a limit on the model's output in tokens. The maximum number of tokens per generation depends on the model. For more information, see Quotas and limits in Yandex Foundation Models.

     • reasoningOptions.mode: Reasoning mode parameters. This is an optional parameter. The default value is DISABLED. The possible values are:

       • DISABLED: Reasoning mode is disabled.
       • ENABLED_HIDDEN: Reasoning mode is enabled. The model will decide by itself whether or not to use this mode for each particular request.

   • messages: List of messages that set the context for the model:

     • role: Message sender's role:

       • user: Used for sending user messages to the model.
       • system: Used to set the query context and define the model's behavior.
       • assistant: Used for responses generated by the model. In chat mode, the model's responses tagged with the assistant role are included in the message to save the conversation context. Do not send user messages with this role.

     • text: Message text.

2. Use the completion method to send a request to the neural network in the following command:

   export FOLDER_ID=<folder_ID>
   export IAM_TOKEN=<IAM_token>
   curl \
     --request POST \
     --header "Content-Type: application/json" \
     --header "Authorization: Bearer ${IAM_TOKEN}" \
     --data "@prompt.json" \
     "https://llm.api.cloud.yandex.net/foundationModels/v1/completion"
   

   Where:

   • FOLDER_ID: ID of the folder for which your account has the ai.languageModels.user role or higher.
   • IAM_TOKEN: IAM token you got before you started.

   The service will respond with the generated text:

   {
     "result": {
       "alternatives": [
         {
           "message": {
             "role": "assistant",
             "text": "Laminate flooring is suitable for installation in the kitchen or in a child's room. It withstands moisture and mechanical damage thanks to a 0.2 mm thick protective layer of melamine films and a wax-treated interlocking system."
           },
           "status": "ALTERNATIVE_STATUS_TRUNCATED_FINAL"
         }
       ],
       "usage": {
         "inputTextTokens": "67",
         "completionTokens": "50",
         "totalTokens": "117"
       },
       "modelVersion": "06.12.2023"
     }
   }