Yandex Cloud ML SDK

Yandex Cloud AI Studio provides Yandex Cloud ML SDK, a library of tools and code examples for Python development. ML SDK employs a standardized method of working with foundation models and simplifies integration with other Yandex Cloud services.

The ML SDK library implements the synchronous and asynchronous Python interfaces based on gRPC API calls of AI Studio services. ML SDK offers the following features:

• Text and image generation based on any supported model.
• Working with embeddings.
• Working with YandexGPT-based classifiers.
• Creating AI assistants.
• Fine-tuning of text generation models and classifiers.
• Integration with LangChain.

You can check the full list of supported functions, library source code, and use cases on GitHub.

InstallationInstallation

You can install the ML SDK library using the pip package manager:

pip install yandex-cloud-ml-sdk

Authentication in Yandex Cloud ML SDKAuthentication in Yandex Cloud ML SDK

To authenticate in Yandex Cloud ML SDK, you need to provide the YCloudML object to the model. This object contains the following fields:

• folder_id: ID of the folder you are going to use to work with models.

• auth: Key, token, or other authentication data to identify the user. You can specify the auth field value explicitly or get it automatically from the environment:

  Explicitly set value

  Value obtained from the environment

  If set explicitly, the auth field value can be one of the following:

  • string: As a string, you can provide:

    • IAM token of a user or service account.
    • Secret part of the service account API key.
    • OAuth token of a user account.

    The SDK will automatically determine the type of authentication data.

  • Object of one of the following classes:

    • APIKeyAuth: Allows you to explicitly set authentication by the provided API key.
      For example: auth = APIKeyAuth('<api_key>').

    • EnvIAMTokenAuth: Allows you to explicitly set authentication using the IAM token specified in YC_TOKEN or any other environment variable.
      For example: auth = EnvIAMTokenAuth() or auth = EnvIAMTokenAuth("env_var").

      The SDK obtains the IAM token from this environment variable with each request, so you can occasionally update the IAM token in the environment variable yourself outside the SDK. This authentication option is optimal for use with a service agent in Yandex DataSphere if that service has access to other resources in the user's cloud.

    • IAMTokenAuth: Allows you to explicitly set authentication by the provided IAM token.
      For example: auth = IAMTokenAuth('<iam_token>').

    • MetadataAuth: Allows you to explicitly set authentication under the service account specified in the Yandex Compute Cloud VM metadata.
      For example: auth = MetadataAuth().

    • NoAuth: Allows you to specify that no authentication data will be provided.
      For example: auth = NoAuth().

    • OAuthTokenAuth: Allows you to explicitly set authentication by the provided OAuth token.
      For example: auth = OAuthTokenAuth('<oauth_token>').

    • YandexCloudCLIAuth: Allows you to explicitly set authentication as a user or service account specified in the Yandex Cloud CLI profile on the user's computer.
      For example: auth = YandexCloudCLIAuth().

    You can get these classes by importing them from the ML SDK library. For example:

    from yandex_cloud_ml_sdk.auth import APIKeyAuth
    

  If the auth field is not explicitly set, the SDK will automatically try to select one of the authentication options in the following order:

  1. Authenticate using the API key from the YC_API_KEY environment variable if it is set.

  2. Authenticate using the IAM token from the YC_IAM_TOKEN environment variable if it is set.

  3. Authenticate using the OAuth token from the YC_OAUTH_TOKEN environment variable if it is set.

  4. If none of these environment variables are set, the SDK will attempt to authenticate using the IAM token of the service account specified in the VM metadata.

  5. Authenticate using the IAM token from the YC_TOKEN environment variable if it is set.

     The SDK obtains the IAM token from this environment variable with each request, so you can occasionally update the IAM token in the YC_TOKEN environment variable yourself outside the SDK.

  6. If the previous options fail, the SDK will attempt to authenticate using the IAM token of the user or service account specified in the Yandex Cloud CLI profile on the user's computer.

  Note

  The maximum lifetime of an IAM token is 12 hours. Keep this in mind when sending requests with authentication based on an IAM token specified in a string, object of the IAMTokenAuth class, or the YC_IAM_TOKEN environment variable.

UsageUsage

As input data for the request, ML SDK can accept the following types:

• String, e.g., What is heaven?

• Dictionary, a data structure similar to JSON, e.g., {"role": "role", "text": "text"}.

• ML SDK TextMessage class object, e.g., result[0].

  The result object of the TextMessage class is an array of alternatives from the model's responses. With such an object, you can provide the previous response of the model in your next request.

• Array containing any combination of the above data types, e.g., ["text", {"role": "role", "text": "text"}].

The example below will prompt YandexGPT Pro with the "What is heaven?" string.

from yandex_cloud_ml_sdk import YCloudML

sdk = YCloudML(
    folder_id="<folder_ID>",
    auth="<authentication_data>",
)

model = sdk.models.completions("yandexgpt")
model = model.configure(temperature=0.5)
result = model.run("What is heaven?")

print(f'{result=}')

print(f'{result[0]=}')

print(f'{result.alternatives[0].role=}')

print(f'{result.alternatives[0].text=}')

print(f'{result.alternatives[0].status=}')

Where:

• folder_id: Service account folder ID.
• auth: Key, token, or other authentication data to identify the user.

Result:

1. The result variable contains an array of alternatives from the model's responses:

   GPTModelResult(alternatives=(Alternative(role='assistant', text=’Heaven, in many religions, is the place believed to be the home of God where good people go when they die. It is sometimes imagined to be in the sky, inhabited by angels, souls, saints, and other celestial beings.\n\nThe word _heaven_ can also be used figuratively to mean something sublime, ideal, or divine.', status=<AlternativeStatus.FINAL: 3>),), usage=Usage(input_text_tokens=14, completion_tokens=83, total_tokens=97), model_version='23.10.2024')
   

2. The result[0] array element contains the result.alternatives[0] object of the ML SDK TextMessage class, which in turn contains the role, text, and status fields:

   Alternative(role='assistant', text='Heaven, in many religions, is the place believed to be the home of God where good people go when they die. It is sometimes imagined to be in the sky, inhabited by angels, souls, saints, and other celestial beings.\n\nThe word _heaven_ can also be used figuratively to mean something sublime, ideal, or divine.', status=<AlternativeStatus.FINAL: 3>)
   

3. The result.alternatives[0].role field states one of these message sender roles:

   • user: To send user messages to the model.
   • system: To set the request context and define the model's behavior.
   • assistant: For responses generated by the model.

   assistant
   

4. The result.alternatives[0].text field contains the message text:

   Heaven, in many religions, is the place believed to be the home of God where good people go when they die. It is sometimes imagined to be in the sky, inhabited by angels, souls, saints, and other celestial beings.
   
   The word _heaven_ can also be used figuratively to mean something sublime, ideal, or divine.
   

5. The result.alternatives[0].status field states the message status. The possible status values include:

   • UNSPECIFIED: Status is not defined.
   • PARTIAL: Part of the generated text that may change while generation is still in progress.
   • TRUNCATED_FINAL: Final generated text where the result exceeds the limits.
   • FINAL: Final generated text within the limits.
   • CONTENT_FILTER: Generation was stopped because the prompt or generated text contains sensitive data or ethically inappropriate topics.

   AlternativeStatus.FINAL: 3
   

For more examples of working with ML SDK, see the step-by-step guides for Yandex AI Studio.