How to create a chat with YandexGPT

Get API authentication credentials as described in Authentication with the Yandex Foundation Models API.

To use the examples, install cURL.

Get API authentication credentials as described in Authentication with the Yandex Foundation Models API.

1. Prepare a model request file, e.g., body.json:

   {
     "modelUri": "gpt://<folder_ID>/yandexgpt-lite",
     "completionOptions": {
       "stream": false,
       "temperature": 0.6,
       "maxTokens": "2000"
     },
     "messages": [
       {
         "role": "system",
         "text": "You are a smart assistant"
       },
       {
         "role": "user", 
         "text": "Hi! What fields of science did Albert Einstein study?"
       }
     ]
   }
   

   Where:

   • modelUri: ID of the model that will be used to generate the response. The parameter contains the Yandex Cloud folder ID or the ID of the tuned model.

   • completionOptions: Request configuration options:

     • stream: Enables streaming of partially generated text. It can either be true or false.
     • temperature: With a higher temperature, you get more creative and randomized responses from the model. Its values range from 0 to 1, inclusive. The default value is 0.3.
     • maxTokens: Sets a limit on the model's output in tokens. The maximum number of tokens per generation depends on the model. For more information, see Quotas and limits in Yandex Foundation Models.

   • messages: List of messages that set the context for the model:

     • role: Message sender's role:

       • user: Used for sending user messages to the model.
       • system: Used to set the query context and define the model's behavior.
       • assistant: Used for responses generated by the model. In chat mode, the model's responses tagged with the assistant role are included in the message to save the conversation context. Do not send user messages with this role.

     • text: Message text.

2. Send a request to the model:

   export FOLDER_ID=<folder_ID>
   export IAM_TOKEN=<IAM_token>
   curl \
     --request POST \
     --header "Content-Type: application/json" \
     --header "Authorization: Bearer ${IAM_TOKEN}" \
     --header "x-folder-id: ${FOLDER_ID}" \
     --data "@<path_to_JSON_file>" \
     "https://llm.api.cloud.yandex.net/foundationModels/v1/completion"
   

   Where:

   • FOLDER_ID: ID of the folder for which your account has the ai.languageModels.user role or higher.
   • IAM_TOKEN: Your account's IAM token.

3. In response to your request, the model will return the generated text:

   Result:

   {
     "result": {
       "alternatives": [
         {
           "message": {
             "role": "assistant",
             "text": "Albert Einstein was an outstanding physicist, whose works in theoretical physics, theoretical mechanics, and philosophy of science became fundamental. He dedicated his career to studying the fundamentals of the universe, including the theory of relativity, both special and general. Additionally, Albert Einstein studied:\n\n* thermodynamics,\n* statistical mechanics,\n* electromagnetism,\n* quantum theory,\n* special relativity, and more.\n\nHis general relativity works found wide recognition and had a profound influence on the development of modern physics."
           },
           "status": "ALTERNATIVE_STATUS_FINAL"
         }
       ],
       "usage": {
         "inputTextTokens": "28",
         "completionTokens": "110",
         "totalTokens": "138"
       },
       "modelVersion": "18.01.2024"
     }
   }
   

   Save the message record value for use in subsequent requests.

4. Add the message record value from the model's response to the previous request and the user's new question to the end of the messages array in the request file:

   {
     "modelUri": "gpt://<folder_ID>/yandexgpt-lite",
     "completionOptions": {
       "stream": false,
       "temperature": 0.6,
       "maxTokens": "2000"
     },
     "messages": [
       {
         "role": "system",
         "text": "You are a smart assistant"
       },
       {
         "role": "user", 
         "text": "Hi! What fields of science did Albert Einstein study?"
       },
       {
         "role": "assistant", 
         "text": "Albert Einstein was an outstanding physicist, whose works in theoretical physics, theoretical mechanics, and philosophy of science became fundamental. He dedicated his career to studying the fundamentals of the universe, including the theory of relativity, both special and general. Additionally, Albert Einstein studied:\n\n* thermodynamics,\n* statistical mechanics,\n* electromagnetism,\n* quantum theory,\n* special relativity, and more.\n\nHis general relativity works found wide recognition and had a profound influence on the development of modern physics."
       },
       {
         "role": "user", 
         "text": "What important discoveries did he make?"
       }
     ]
   }
   

5. Send a new request to the model repeating Step 2 of this guide. In response to your request, the model will again return the generated text:

   Result:

   {
     "result": {
       "alternatives": [
         {
           "message": {
             "role": "assistant",
             "text": Here are some discoveries commonly associated with the name of Albert Einstein:\n1. **Special theory of relativity (STR)**: Theory describing motion, laws of mechanics, and spatiotemporal relationships at random velocities less than the speed of light in a vacuum.\n1. **General theory of relativity (GTR)**: Geometric theory of gravitation based on the prediction of the interrelation between the spacetime, matter, and energy coordinates.\n1. **Quantum theory of the photoelectric effect**: Explanation of the photoelectric effect, a phenomenon of electron emission from matter under the influence of electromagnetic radiation.\n1. **Brownian motion**: Phenomenon of chaotic, random movement of microscopic particles suspended in a liquid or gas, caused by the thermal motion of molecules in the surrounding medium.\n1. **Equivalence principle**: Idea that gravitational and inertial mass are different aspects of the same physical quantity.\n1. **Cosmological solutions of the GTR equations**: Contribution to the theory of the structure and dynamics of the universe.\n1. **Nuclear model of the atom**: Description of the atom structure based on protons, neutrons, and electrons.\nThese are just some of the discoveries made by Albert Einstein. His contributions to science and philosophy are immense, and his research continues to shape our ideas of nature and the universe."
           },
           "status": "ALTERNATIVE_STATUS_FINAL"
         }
       ],
       "usage": {
         "inputTextTokens": "154",
         "completionTokens": "278",
         "totalTokens": "432"
       },
       "modelVersion": "18.01.2024"
     }
   }
   

   Once again, save the message record value for use in subsequent requests.

6. Continue expanding the request context with the received responses and user questions:

   {
     "modelUri": "gpt://<folder_ID>/yandexgpt-lite",
     "completionOptions": {
       "stream": false,
       "temperature": 0.6,
       "maxTokens": "2000"
     },
     "messages": [
       {
         "role": "system",
         "text": "You are a smart assistant"
       },
       {
         "role": "user", 
         "text": "Hi! What fields of science did Albert Einstein study?"
       },
       {
         "role": "assistant", 
         "text": "Albert Einstein was an outstanding physicist, whose works in theoretical physics, theoretical mechanics, and philosophy of science became fundamental. He dedicated his career to studying the fundamentals of the universe, including the theory of relativity, both special and general. Additionally, Albert Einstein studied:\n\n* thermodynamics,\n* statistical mechanics,\n* electromagnetism,\n* quantum theory,\n* special relativity, and more.\n\nHis general relativity works found wide recognition and had a profound influence on the development of modern physics."
       },
       {
         "role": "user", 
         "text": "What important discoveries did he make?"
       },
       {
         "role": "assistant",
         "text": Here are some discoveries commonly associated with the name of Albert Einstein:\n1. **Special theory of relativity (STR)**: Theory describing motion, laws of mechanics, and spatiotemporal relationships at random velocities less than the speed of light in a vacuum.\n1. **General theory of relativity (GTR)**: Geometric theory of gravitation based on the prediction of the interrelation between the spacetime, matter, and energy coordinates.\n1. **Quantum theory of the photoelectric effect**: Explanation of the photoelectric effect, a phenomenon of electron emission from matter under the influence of electromagnetic radiation.\n1. **Brownian motion**: Phenomenon of chaotic, random movement of microscopic particles suspended in a liquid or gas, caused by the thermal motion of molecules in the surrounding medium.\n1. **Equivalence principle**: Idea that gravitational and inertial mass are different aspects of the same physical quantity.\n1. **Cosmological solutions of the GTR equations**: Contribution to the theory of the structure and dynamics of the universe.\n1. **Nuclear model of the atom**: Description of the atom structure based on protons, neutrons, and electrons.\nThese are just some of the discoveries made by Albert Einstein. His contributions to science and philosophy are immense, and his research continues to shape our ideas of nature and the universe."
       },
       {
         "role": "user",
         "text": "Give me a shorter response"
       }
     ]
   }
   

   Result:

   {
     "result": {
       "alternatives": [
         {
           "message": {
             "role": "assistant",
             "text": "Albert Einstein was an outstanding physicist who studied the fundamentals of the universe, including the theory of relativity, thermodynamics, and electromagnetism.\n\nSome of his well-known discoveries include the following: special and general theories of relativity, quantum theory of the photoelectric effect, and others."
           },
           "status": "ALTERNATIVE_STATUS_FINAL"
         }
       ],
       "usage": {
         "inputTextTokens": "452",
         "completionTokens": "54",
         "totalTokens": "506"
       },
       "modelVersion": "18.01.2024"
     }
   }
   

1. Prepare a model request file, e.g., body.json:

   {
     "modelUri": "gpt://<folder_ID>/yandexgpt-lite",
     "completionOptions": {
       "stream": false,
       "temperature": 0.6,
       "maxTokens": "2000"
     },
     "messages": [
       {
         "role": "system",
         "text": "You are a smart assistant"
       },
       {
         "role": "user", 
         "text": "Hi! What fields of science did Albert Einstein study?"
       }
     ]
   }
   

   Where:

   • modelUri: ID of the model that will be used to generate the response. The parameter contains the Yandex Cloud folder ID or the ID of the tuned model.

   • completionOptions: Request configuration options:

     • stream: Enables streaming of partially generated text. It can either be true or false.
     • temperature: With a higher temperature, you get more creative and randomized responses from the model. Its values range from 0 to 1, inclusive. The default value is 0.3.
     • maxTokens: Sets a limit on the model's output in tokens. The maximum number of tokens per generation depends on the model. For more information, see Quotas and limits in Yandex Foundation Models.

   • messages: List of messages that set the context for the model:

     • role: Message sender's role:

       • user: Used for sending user messages to the model.
       • system: Used to set the query context and define the model's behavior.
       • assistant: Used for responses generated by the model. In chat mode, the model's responses tagged with the assistant role are included in the message to save the conversation context. Do not send user messages with this role.

     • text: Message text.

2. Send a request to the model:

   1. Create a file named index.py and add the following code to it:

      import requests
      import json
      import os
      
      def gpt(auth_headers):
      
          url = 'https://llm.api.cloud.yandex.net/foundationModels/v1/completion'
      
          with open('body.json', 'r', encoding='utf-8') as f:
              data = json.dumps(json.load(f))
          resp = requests.post(url, headers=auth_headers, data=data)
      
          if resp.status_code != 200:
              raise RuntimeError(
                  'Invalid response received: code: {}, message: {}'.format(
                      {resp.status_code}, {resp.text}
                  )
              )
      
          return resp.text
      
      if __name__ == "__main__":
          if os.getenv('IAM_TOKEN') is not None:
              iam_token = os.environ['IAM_TOKEN']
              headers = {
                  'Authorization': f'Bearer {iam_token}',
              }
          elif os.getenv('API_KEY') is not None:
              api_key = os.environ['API_KEY']
              headers = {
                  'Authorization': f'Api-Key {api_key}',
              }
          else:
              print ('Please save either an IAM token or an API key into a corresponding `IAM_TOKEN` or `API_KEY` environment variable.')
              exit()
      
          print(gpt(headers))
      

   2. Save the authentication data to the environment variable:

      Authentication with an IAM token:

      export IAM_TOKEN=<IAM_token>
      

      Authentication with an API key (for service accounts only):

      export API_KEY=<API_key>
      

   3. Run the created file:

      python index.py
      

3. In response to your request, the model will return the generated text:

   Result:

   {
     "result": {
       "alternatives": [
         {
           "message": {
             "role": "assistant",
             "text": "Albert Einstein was an outstanding physicist, whose works in theoretical physics, theoretical mechanics, and philosophy of science became fundamental. He dedicated his career to studying the fundamentals of the universe, including the theory of relativity, both special and general. Additionally, Albert Einstein studied:\n\n* thermodynamics,\n* statistical mechanics,\n* electromagnetism,\n* quantum theory,\n* special relativity, and more.\n\nHis general relativity works found wide recognition and had a profound influence on the development of modern physics."
           },
           "status": "ALTERNATIVE_STATUS_FINAL"
         }
       ],
       "usage": {
         "inputTextTokens": "28",
         "completionTokens": "110",
         "totalTokens": "138"
       },
       "modelVersion": "18.01.2024"
     }
   }
   

   Save the message record value for use in subsequent requests.

4. Add the message record value from the model's response to the previous request and the user's new question to the end of the messages array in the request file:

   {
     "modelUri": "gpt://<folder_ID>/yandexgpt-lite",
     "completionOptions": {
       "stream": false,
       "temperature": 0.6,
       "maxTokens": "2000"
     },
     "messages": [
       {
         "role": "system",
         "text": "You are a smart assistant"
       },
       {
         "role": "user", 
         "text": "Hi! What fields of science did Albert Einstein study?"
       },
       {
         "role": "assistant", 
         "text": "Albert Einstein was an outstanding physicist, whose works in theoretical physics, theoretical mechanics, and philosophy of science became fundamental. He dedicated his career to studying the fundamentals of the universe, including the theory of relativity, both special and general. Additionally, Albert Einstein studied:\n\n* thermodynamics,\n* statistical mechanics,\n* electromagnetism,\n* quantum theory,\n* special relativity, and more.\n\nHis general relativity works found wide recognition and had a profound influence on the development of modern physics."
       },
       {
         "role": "user", 
         "text": "What important discoveries did he make?"
       }
     ]
   }
   

5. Send a new request to the model repeating Step 2.3 of this guide. In response to your request, the model will again return the generated text:

   Result:

   {
     "result": {
       "alternatives": [
         {
           "message": {
             "role": "assistant",
             "text": Here are some discoveries commonly associated with the name of Albert Einstein:\n1. **Special theory of relativity (STR)**: Theory describing motion, laws of mechanics, and spatiotemporal relationships at random velocities less than the speed of light in a vacuum.\n1. **General theory of relativity (GTR)**: Geometric theory of gravitation based on the prediction of the interrelation between the spacetime, matter, and energy coordinates.\n1. **Quantum theory of the photoelectric effect**: Explanation of the photoelectric effect, a phenomenon of electron emission from matter under the influence of electromagnetic radiation.\n1. **Brownian motion**: Phenomenon of chaotic, random movement of microscopic particles suspended in a liquid or gas, caused by the thermal motion of molecules in the surrounding medium.\n1. **Equivalence principle**: Idea that gravitational and inertial mass are different aspects of the same physical quantity.\n1. **Cosmological solutions of the GTR equations**: Contribution to the theory of the structure and dynamics of the universe.\n1. **Nuclear model of the atom**: Description of the atom structure based on protons, neutrons, and electrons.\nThese are just some of the discoveries made by Albert Einstein. His contributions to science and philosophy are immense, and his research continues to shape our ideas of nature and the universe."
           },
           "status": "ALTERNATIVE_STATUS_FINAL"
         }
       ],
       "usage": {
         "inputTextTokens": "154",
         "completionTokens": "278",
         "totalTokens": "432"
       },
       "modelVersion": "18.01.2024"
     }
   }
   

   Once again, save the message record value for use in subsequent requests.

6. Continue expanding the request context with the received responses and user questions:

   {
     "modelUri": "gpt://<folder_ID>/yandexgpt-lite",
     "completionOptions": {
       "stream": false,
       "temperature": 0.6,
       "maxTokens": "2000"
     },
     "messages": [
       {
         "role": "system",
         "text": "You are a smart assistant"
       },
       {
         "role": "user", 
         "text": "Hi! What fields of science did Albert Einstein study?"
       },
       {
         "role": "assistant", 
         "text": "Albert Einstein was an outstanding physicist, whose works in theoretical physics, theoretical mechanics, and philosophy of science became fundamental. He dedicated his career to studying the fundamentals of the universe, including the theory of relativity, both special and general. Additionally, Albert Einstein studied:\n\n* thermodynamics,\n* statistical mechanics,\n* electromagnetism,\n* quantum theory,\n* special relativity, and more.\n\nHis general relativity works found wide recognition and had a profound influence on the development of modern physics."
       },
       {
         "role": "user", 
         "text": "What important discoveries did he make?"
       },
       {
         "role": "assistant",
         "text": Here are some discoveries commonly associated with the name of Albert Einstein:\n1. **Special theory of relativity (STR)**: Theory describing motion, laws of mechanics, and spatiotemporal relationships at random velocities less than the speed of light in a vacuum.\n1. **General theory of relativity (GTR)**: Geometric theory of gravitation based on the prediction of the interrelation between the spacetime, matter, and energy coordinates.\n1. **Quantum theory of the photoelectric effect**: Explanation of the photoelectric effect, a phenomenon of electron emission from matter under the influence of electromagnetic radiation.\n1. **Brownian motion**: Phenomenon of chaotic, random movement of microscopic particles suspended in a liquid or gas, caused by the thermal motion of molecules in the surrounding medium.\n1. **Equivalence principle**: Idea that gravitational and inertial mass are different aspects of the same physical quantity.\n1. **Cosmological solutions of the GTR equations**: Contribution to the theory of the structure and dynamics of the universe.\n1. **Nuclear model of the atom**: Description of the atom structure based on protons, neutrons, and electrons.\nThese are just some of the discoveries made by Albert Einstein. His contributions to science and philosophy are immense, and his research continues to shape our ideas of nature and the universe."
       },
       {
         "role": "user",
         "text": "Give me a shorter response"
       }
     ]
   }
   

   Result:

   {
     "result": {
       "alternatives": [
         {
           "message": {
             "role": "assistant",
             "text": "Albert Einstein was an outstanding physicist who studied the fundamentals of the universe, including the theory of relativity, thermodynamics, and electromagnetism.\n\nSome of his well-known discoveries include the following: special and general theories of relativity, quantum theory of the photoelectric effect, and others."
           },
           "status": "ALTERNATIVE_STATUS_FINAL"
         }
       ],
       "usage": {
         "inputTextTokens": "452",
         "completionTokens": "54",
         "totalTokens": "506"
       },
       "modelVersion": "18.01.2024"
     }
   }