Sending an asynchronous request

To use the examples of requests using SDK:

1. Create a service account and assign the ai.languageModels.user role to it.

2. Get the service account API key and save it.

   The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.

3. Use the pip package manager to install the ML SDK library:

   pip install yandex-cloud-ml-sdk
   

When using Yandex Cloud ML SDK, you can configure your code to wait for the operation to complete and return the response. To do this, use either the sleep function of the time module or the wait method. The example utilizes both of these methods one by one.

1. Create a file named generate-deferred.py and paste the following code into it:

   #!/usr/bin/env python3
   
   from __future__ import annotations
   import time
   from yandex_cloud_ml_sdk import YCloudML
   
   messages_1 = [
       {
           "role": "system",
           "text": "Find errors in the text and correct them",
       },
       {
           "role": "user",
           "text": """Laminate flooring is sutiable for instalation in the kitchen or in a child's 
   room. It withsatnds moisturre and mechanical dammage thanks to 
   a 0.2 mm thick proctive layer of melamine films and 
   a wax-treated interlocking system.""",
       },
   ]
   
   messages_2 = [
       {"role": "system", "text": "Find errors in the text and correct them"},
       {"role": "user", "text": "Erors wyll not corrct themselfs."},
   ]
   
   
   def main():
   
       sdk = YCloudML(
           folder_id="<folder_ID>",
           auth="<API_key>",
       )
   
       model = sdk.models.completions("yandexgpt")
   
       # Variant 1: wait for the operation to complete using 5-second sleep periods
   
       print("Variant 1:")
   
       operation = model.configure(temperature=0.5).run_deferred(messages_1)
   
       status = operation.get_status()
       while status.is_running:
           time.sleep(5)
           status = operation.get_status()
   
       result = operation.get_result()
       print(result)
   
       # Variant 2: wait for the operation to complete using the wait method
   
       print("Variant 2:")
   
       operation = model.run_deferred(messages_2)
   
       result = operation.wait()
       print(result)
   
   
   if __name__ == "__main__":
       main()
   

   Where:

   Note

   As input data for a request, Yandex Cloud ML SDK can accept a string, a dictionary, an object of the TextMessage class, or an array containing any combination of these data types. For more information, see Yandex Cloud ML SDK usage.

   • messages_1 and messages_2: Arrays of messages providing the context for the model, each used for a different method of getting an asynchronous request result:

     • role: Message sender's role:

       • user: Used for sending user messages to the model.
       • system: Used to set the query context and define the model's behavior.
       • assistant: Used for responses generated by the model. In chat mode, the model's responses tagged with the assistant role are included in the message to save the conversation context. Do not send user messages with this role.

   • text: Message text.

   • <folder_ID>: ID of the folder in which the service account was created.

   • <API_key>: Service account API key you got earlier required for authentication in the API.

     The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.

   For more information about accessing a specific model version, see Accessing models.

2. Run the created file:

   python3 generate-deferred.py
   

   Result:

   Variant 1:
   GPTModelResult(alternatives=(Alternative(role='assistant', text='Ламинат подойдёт для укладки на кухне или в детской комнате – он не боится влаги и механических повреждений благодаря защитному слою из облицованных меламиновых плёнок толщиной 0,2 мм и обработанным воском замкам.', status=<AlternativeStatus.FINAL: 3>),), usage=Usage(input_text_tokens=74, completion_tokens=46, total_tokens=120), model_version='23.10.2024')
   Variant 2:
   GPTModelResult(alternatives=(Alternative(role='assistant', text='Errors will not correct themselves.\n\nErors → errors.', status=<AlternativeStatus.FINAL: 3>),), usage=Usage(input_text_tokens=32, completion_tokens=16, total_tokens=48), model_version='23.10.2024')
   

   The code waits for the result of the first method and then of the second one.