Sending an asynchronous request

To use the examples of requests using SDK:

1. Create a service account and assign the ai.languageModels.user role to it.

2. Get the service account API key and save it.

   The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.

   Note

   If you are using Windows, we recommend installing the WSL shell first and then using it to perform other operations.

3. Install Python 3.10 or higher.

4. Install Python venv to create isolated virtual environments in Python.

5. Create a new Python virtual environment and activate it:

   python3 -m venv new-env
   source new-env/bin/activate
   

6. Use the pip package manager to install the ML SDK library:

   pip install yandex-cloud-ml-sdk
   

When using Yandex Cloud ML SDK, you can configure your code to wait for the operation to complete and return the response. To do this, use either the sleep function of the time module or the wait method. The example utilizes both of these methods one by one.

1. Create a file named generate-deferred.py and paste the following code into it:

   #!/usr/bin/env python3
   
   from __future__ import annotations
   import time
   from yandex_cloud_ml_sdk import YCloudML
   
   messages_1 = [
       {
           "role": "system",
           "text": "Find errors in the text and correct them",
       },
       {
           "role": "user",
           "text": """Laminate flooring is sutiable for instalation in the kitchen or in a child's room 
   room. It withsatnds moisturre and mechanical dammage thanks to 
   a 0.2 mm thick proctive layer of melamine films and 
   a wax-treated interlocking system.""",
       },
   ]
   
   messages_2 = [
       {"role": "system", "text": "Find errors in the text and correct them"},
       {"role": "user", "text": "Erors wyll not corrct themselfs."},
   ]
   
   
   def main():
   
       sdk = YCloudML(
           folder_id="<folder_ID>",
           auth="<API_key>",
       )
   
       model = sdk.models.completions("yandexgpt")
   
       # Variant 1: wait for the operation to complete using 5-second sleep periods
   
       print("Variant 1:")
   
       operation = model.configure(temperature=0.5).run_deferred(messages_1)
   
       status = operation.get_status()
       while status.is_running:
           time.sleep(5)
           status = operation.get_status()
   
       result = operation.get_result()
       print(result)
   
       # Variant 2: wait for the operation to complete using the wait method
   
       print("Variant 2:")
   
       operation = model.run_deferred(messages_2)
   
       result = operation.wait()
       print(result)
   
   
   if __name__ == "__main__":
       main()
   

   Where:

   Note

   As input data for a request, Yandex Cloud ML SDK can accept a string, a dictionary, an object of the TextMessage class, or an array containing any combination of these data types. For more information, see Yandex Cloud ML SDK usage.

   • messages_1 and messages_2: Arrays of messages providing the context for the model, each used for a different method of getting an asynchronous request result:

     • role: Message sender's role:

       • user: To send user messages to the model.
       • system: To set the request context and define the model's behavior.
       • assistant: For responses generated by the model. In chat mode, the model's responses tagged with the assistant role are included in the message to save the conversation context. Do not send user messages with this role.

   • text: Message text.

   • <folder_ID>: ID of the folder in which the service account was created.

   • <API_key>: Service account API key you got earlier required for authentication in the API.

     The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.

   For more information about accessing a specific model version, see Accessing models.

2. Run the file you created:

   python3 generate-deferred.py
   

   Result:

   Variant 1:
   GPTModelResult(alternatives=(Alternative(role='assistant', text='Laminate flooring is suitable for installation in the kitchen or in a child's room. It withstands moisture and mechanical damage thanks to a 0.2 mm thick protective layer of melamine films and a wax-treated interlocking system.', status=<AlternativeStatus.FINAL: 3>),), usage=Usage(input_text_tokens=74, completion_tokens=46, total_tokens=120), model_version='23.10.2024')
   Variant 2:
   GPTModelResult(alternatives=(Alternative(role='assistant', text='Errors will not correct themselves.\n\nErors → errors.', status=<AlternativeStatus.FINAL: 3>),), usage=Usage(input_text_tokens=32, completion_tokens=16, total_tokens=48), model_version='23.10.2024')
   

   The code waits for the result of the first method and then of the second one.