Contact UsGet started

Yandex Cloud ML SDK

Written by
Updated at December 27, 2024

Yandex Cloud AI Studio provides Yandex Cloud ML SDK, a library of tools and code examples for Python development. ML SDK provides a standardized method of working with foundation models and simplifies integration with other Yandex Cloud services.

The ML SDK library implements the synchronous and asynchronous Python interfaces based on gRPC API calls of AI Studio services. ML SDK offers the following features:

You can see the full list of supported functions, library source code, and use cases on GitHub.

Installation

You can install the ML SDK library using the pip package manager:

pip install yandex-cloud-ml-sdk

Authentication in Yandex Cloud ML SDK

To authenticate in Yandex Cloud ML SDK, you need to provide a YCloudML object to the model. This object contains the following fields:

  • folder_id: ID of the folder you are going to use to work with models.

  • auth: Key, token, or other authentication data to identify the subject. You can specify the auth field value explicitly or get it automatically from the environment:

    If set explicitly, the auth field value can be one of the following:

    • string: As a string, you can provide:

      The SDK will automatically determine the type of the authentication data.

    • Object of one of the following classes:

      • APIKeyAuth: Allows you to explicitly set authentication by the provided API key. Example: auth = APIKeyAuth('<API_key>').

      • IAMTokenAuth: Allows you to explicitly set authentication by the provided API token. Example: auth = IAMTokenAuth('<IAM_token>').

      • OAuthTokenAuth: Allows you to explicitly set authentication by the provided OAuth token. Example: auth = OAuthTokenAuth('<OAuth_token>').

      • MetadataAuth: Allows you to explicitly set authentication as the service account specified in the Yandex Compute Cloud VM metadata. Example: auth = MetadataAuth().

      • EnvIAMTokenAuth: Allows you to explicitly set authentication using the IAM token specified in the YC_TOKEN or any other environment variable. Example: auth = EnvIAMTokenAuth() or auth = EnvIAMTokenAuth("ENV_VAR").

        The SDK obtains the IAM token from this environment variable with each request, so you can occasionally update the IAM token in the environment variable yourself outside the SDK. This authentication option is optimal for use with a service agent in Yandex DataSphere if that service has access to other resources in the user's cloud.

      • YandexCloudCLIAuth: Allows you to explicitly set authentication as a user or service account specified in the Yandex Cloud CLI profile on the user's computer. Example: auth = YandexCloudCLIAuth().

      • NoAuth: Specifies that no authentication data will be provided. Example: auth = NoAuth().

      You can get these classes by importing them from the ML SDK library. Here is an example:

      from yandex_cloud_ml_sdk.auth import APIKeyAuth

    If the auth field is not explicitly set, the SDK will automatically try to select one of the authentication options in the following order:

    1. Authenticate using the API key from the YC_API_KEY environment variable if it is set.

    2. Authenticate using the IAM token from the YC_IAM_TOKEN environment variable if it is set.

    3. Authenticate using the OAuth token from the YC_OAUTH_TOKEN environment variable if it is set.

    4. If none of these environment variables are set, the SDK will attempt to authenticate using the IAM token of the service account specified in the VM metadata.

    5. Authenticate using the IAM token from the YC_TOKEN environment variable if it is set.

      The SDK obtains the IAM token from this environment variable with each request, so you can occasionally update the IAM token in the YC_TOKEN environment variable yourself outside the SDK.

    6. If the previous options fail, the SDK will attempt to authenticate using the IAM token of the user or service account specified in the Yandex Cloud CLI profile on the user's computer.

    Note

    The maximum lifetime of an IAM token is 12 hours. Keep this in mind when sending requests with authentication based on an IAM token specified in a string, object of the IAMTokenAuth class, or the YC_IAM_TOKEN environment variable.

Usage

As input data for the request, ML SDK can accept the following types:

  • String, e.g., What is heaven?

  • Dictionary, a data structure similar to JSON, e.g., {"role": "role", "text": "text"}.

  • ML SDK TextMessage class object, e.g., result[0].

    The result object of the TextMessage class is an array of alternatives contained in the model's responses. With such an object, you can provide the previous response of the model in your next request.

  • Array containing any combination of the above data types. Example: ["text", {"role": "role", "text": "text"}].

The example below will prompt YandexGPT Pro with the "What is heaven?" string.

from yandex_cloud_ml_sdk import YCloudML

sdk = YCloudML(
    folder_id="<folder_ID>",
    auth="<authentication_data>",
)

model = sdk.models.completions("yandexgpt")
model = model.configure(temperature=0.5)
result = model.run("What is heaven?")

print(f'{result=}')

print(f'{result[0]=}')

print(f'{result.alternatives[0].role=}')

print(f'{result.alternatives[0].text=}')

print(f'{result.alternatives[0].status=}')

Result:

  1. The result variable contains an array of alternatives from the model's responses:

    GPTModelResult(alternatives=(Alternative(role='assistant', text=’Heaven, in many religions, is the place believed to be the home of God where good people go when they die. It is sometimes imagined to be in the sky, inhabited by angels, souls, saints, and other celestial beings.\n\nThe word _heaven_ can also be used figuratively to mean something sublime, ideal, or divine.', status=<AlternativeStatus.FINAL: 3>),), usage=Usage(input_text_tokens=14, completion_tokens=83, total_tokens=97), model_version='23.10.2024')

  2. The result[0] array element contains the result.alternatives[0] object of the ML SDK TextMessage class, which in turn contains the role, text, and status fields:

    Alternative(role='assistant', text='Heaven, in many religions, is the place believed to be the home of God where good people go when they die. It is sometimes imagined to be in the sky, inhabited by angels, souls, saints, and other celestial beings.\n\nThe word _heaven_ can also be used figuratively to mean something sublime, ideal, or divine.', status=<AlternativeStatus.FINAL: 3>)

  3. The result.alternatives[0].role field states one of these message sender roles:

    • user: Used for sending user messages to the model.
    • system: Used to set the query context and define the model's behavior.
    • assistant: Used for responses generated by the model.
    assistant

  4. The result.alternatives[0].text field contains the message text:

    Heaven, in many religions, is the place believed to be the home of God where good people go when they die. It is sometimes imagined to be in the sky, inhabited by angels, souls, saints, and other celestial beings.

The word _heaven_ can also be used figuratively to mean something sublime, ideal, or divine.

  5. The result.alternatives[0].status field states the message status. Possible status values:

    • UNSPECIFIED: Status is not defined.
    • PARTIAL: Part of the generated text that may change while generation is still in progress.
    • TRUNCATED_FINAL: Final generated text where the result exceeds the limits.
    • FINAL: Final generated text within the limits.
    • CONTENT_FILTER: Generation was stopped because the prompt or generated text contain sensitive data or ethically inappropriate topics.
    AlternativeStatus.FINAL: 3

For more examples of working with ML SDK, see the step-by-step guides for Yandex Foundation Models.

Previous
ListErrors
Next
Access management