Contact UsGet started

Fine-tuning in Foundation Models

Written by
Updated at December 27, 2024

Model tuning based on the LoRA method is at the Preview stage and available upon request. You can fill out the form in the management console.

This example shows how to fine-tune a YandexGPT Lite model based on the LoRA method in Foundation Models.

Getting started

To use the examples:

  1. Create a service account and assign the ai.editor role to it.

  2. Get the service account API key and save it.

    The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.

  3. Use the pip package manager to install the beta version of the ML SDK library:

pip install yandex-cloud-ml-sdk --upgrade --pre

  1. Get API authentication credentials as described in Authentication with the Yandex Foundation Models API.

  2. Install gRPCurl.

  3. To use the examples, install cURL.

  4. (Optional) Install the jq JSON stream processor.

  5. Get an IAM token used for authentication in the API.

    Note

    The IAM token has a short lifetime: no more than 12 hours.

Upload the dataset

Prepare UTF-8-encoded model tuning data in JSON Lines format. If you want to split your data into two datasets for tuning and validation, repeat the steps below for each dataset. Use the IDs you got after uploading the datasets to start the fine-tuning process.

In this example, we only use the tuning dataset for fine-tuning.

Create a tuning dataset:

  1. Create a file named dataset-create.py and add the following code to it:

    #!/usr/bin/env python3

from __future__ import annotations
import asyncio
import pathlib
from yandex_cloud_ml_sdk import AsyncYCloudML
from yandex_cloud_ml_sdk.auth import YandexCloudCLIAuth


def local_path(path: str) -> pathlib.Path:
    return pathlib.Path(__file__).parent / path


async def main():

    sdk = AsyncYCloudML(
        folder_id="<folder_ID>",
        auth="<API_key>",
    )

    # Creating a tuning dataset for the YandexGPT Lite base model
    dataset_draft = sdk.datasets.from_path_deferred(
        task_type="TextToTextGeneration",
        path="<path_to_file>",
        upload_format="jsonlines",
        name="YandexGPT tuning",
    )

    # Waiting for the data to be uploaded and the dataset to be created
    operation = await dataset_draft.upload()
    dataset = await operation
    print(f"new {dataset=}")


if __name__ == "__main__":
    asyncio.run(main())

    Where:

  2. Run the created file:

    python3 dataset-create.py

  1. Create the dataset object:

    grpcurl \
  -H "Authorization: Bearer $<IAM_token>" \
  -d @ \
  llm.api.cloud.yandex.net:443 yandex.cloud.ai.dataset.v1.DatasetService/Create <<EOM
  {
    "folder_id": "<folder_ID>", 
    "name": "My awesome dataset", 
    "task_type": "TextToTextGeneration", 
    "upload_format": "jsonlines"
  }
EOM

  2. Get a link to upload data into the dataset. In the size_bytes field, specify the size of the file with fine-tuning data in bytes. The size of the dataset in the example is 10 KB:

    grpcurl \
  -H "Authorization: Bearer $<IAM_token>" \
  -d '{"dataset_id": "<dataset_ID>", "size_bytes": 10240}' \
  llm.api.cloud.yandex.net:443 yandex.cloud.ai.dataset.v1.DatasetService/GetUploadDraftUrl | jq

    In response, you will get a link to the dataset template you created.

    Tip

    If you did not use jq, replace all \u0026 occurrences with & in the link to use it to upload the dataset.

  3. Upload your data:

    curl --request PUT --upload-file <path_to_file> "<link>"

  4. After the data upload is complete, run the dataset validation:

    grpcurl \
  -H "Authorization: Bearer $<IAM_token>" \
  -d '{"dataset_id": "<dataset_ID>"}' \
  llm.api.cloud.yandex.net:443 yandex.cloud.ai.dataset.v1.DatasetService/Validate

    In response, you will get an object containing the validation ID.

  5. Dataset validation may take some time. To find out validation status and get an error report (if any), send a request containing the ID from the previous step:

    grpcurl \
  -H "Authorization: Bearer $<IAM_token>" \
  -d '{"operation_id": "ftnq****************"}' \
  llm.api.cloud.yandex.net:443 yandex.cloud.operation.OperationService/Get

Start tuning

  1. Create a file named start-tuning.py and add the following code to it:

    #!/usr/bin/env python3

from __future__ import annotations
import pathlib
import uuid
from yandex_cloud_ml_sdk import YCloudML


def local_path(path: str) -> pathlib.Path:
    return pathlib.Path(__file__).parent / path


def main():
    sdk = YCloudML(
        folder_id="<folder_ID>",
        auth="<API_key>",
    )

    # Viewing the list of valid datasets
    for dataset in sdk.datasets.list(status="READY", name_pattern="completions"):
        print(f"List of existing datasets {dataset=}")

    # Setting the tuning dataset and the base model
    train_dataset = sdk.datasets.get("<dataset_ID>")
    base_model = sdk.models.completions("yandexgpt-lite")

    # Defining minimum parameters
    # To control more parameters, use `base_model.tune_deferred()`
    tuned_model = base_model.tune(train_dataset, name=str(uuid.uuid4()))
    print(f"Resulting {tuned_model}")

    # Starting the tuning
    completion_result = tuned_model.run("hey!")
    print(f"{completion_result=}")

    # Saving the URI of the tuned model
    tuned_uri = tuned_model.uri
    model = sdk.models.completions(tuned_uri)

    completion_result = model.run("hey!")
    print(f"{completion_result=}")


if __name__ == "__main__":
    main()

  2. Run the created file:

    python3 start-tuning.py

  1. Start tuning:

    grpcurl \
  -H "Authorization: Bearer $<IAM_token>" \
  -d @ \
  llm.api.cloud.yandex.net:443 yandex.cloud.ai.tuning.v1.TuningService/Tune <<EOM
  {
    "base_model_uri": "gpt://<folder_ID>/yandexgpt-lite/latest",
    "train_datasets": [{"dataset_id": "<dataset_ID>", "weight": 1.0}],
    "name": "my first model",
    "text_to_text_completion": {}
  } 
EOM

    In response, you will get the Operation object:

    {"id":"f**********","description":"","createdAt":null,"createdBy":"","modifiedAt":null,"done":false,"metadata":null}

    Save the operation id you get in the response.

  2. Model tuning may take up to 1 day depending on the size of the dataset and the system load. To check if the fine-tuning is complete, request the operation status:

    grpcurl \
  -H "Authorization: Bearer $<IAM_token>" \
  -d '{"operation_id": "ftnq****************"}' \
  llm.api.cloud.yandex.net:443 yandex.cloud.operation.OperationService/Get

    If fine-tuning process is over, the Operation object will contain the tuned model's ID in the targetModelUri field:

    {
  "id": "ftnq****************",
  "createdAt": "2024-12-04T10:56:08Z",
  "modifiedAt": "2024-12-04T11:14:25Z",
  "done": true,
  "metadata": {
    "@type": "type.googleapis.com/yandex.cloud.ai.tuning.v1.TuningMetadata",
    "status": "COMPLETED",
    "tuningTaskId": "ftn7****************"
  },
  "response": {
    "@type": "type.googleapis.com/yandex.cloud.ai.tuning.v1.TuningResponse",
    "status": "COMPLETED",
    "targetModelUri": "<fine-tuned_model_modelUri>",
    "tuningTaskId": "ftn7****************"
  }
}

Use the fine-tuned model's ID as modelURI to send requests to the fine-tuned model.

See also

For more examples, see our GitHub repository.

Previous
Creating a search assistant
Next
About Yandex Foundation Models