Tuning a text generation model
Model tuning based on the LoRA method is at the Preview stage.
This example shows how to fine-tune a YandexGPT Lite model based on the LoRA method in Foundation Models. Links to other examples are available in the See also section.
Getting started
To use the examples:
You can start working from the management console right away.
-
Create a service account and assign the
ai.editor
role to it. -
Get the service account API key and save it.
The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.
-
Use the pip
package manager to install the ML SDK library:pip install yandex-cloud-ml-sdk
-
Get API authentication credentials as described in Authentication with the Yandex Foundation Models API.
-
Install gRPCurl
. -
To use the examples, install cURL
. -
(Optional) Install the jq
JSON stream processor. -
Get an IAM token used for authentication in the API.
Note
The IAM token has a short lifetime: no more than 12 hours.
Upload the dataset
Prepare UTF-8
In this example, we only use the tuning dataset for fine-tuning.
Create a tuning dataset:
-
In the management console
, select the folder for which your account has theai.playground.user
andai.datasets.editor
roles or higher. -
From the list of services, select Foundation Models.
-
In the left-hand panel, click
Datasets. -
Click Create dataset.
-
Enter a name and descriptions for the dataset. Follow these naming requirements:
- It must be from 2 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- It must start with a letter and cannot end with a hyphen.
-
In the Type field, select Text generation.
-
Delete or add dataset labels. You can use them to split or join resources into logical groups.
-
Click Select file or drag the JSON file you created earlier to the loading area.
-
Click Create dataset.
-
Create a file named
dataset-create.py
and add the following code to it:#!/usr/bin/env python3 from __future__ import annotations import pathlib from yandex_cloud_ml_sdk import YCloudML from yandex_cloud_ml_sdk.exceptions import DatasetValidationError def main() -> None: sdk = YCloudML( folder_id="<folder_ID>", auth="<API_key>", ) # Viewing the list of all previously uploaded datasets for dataset in sdk.datasets.list(): print(f"List of existing datasets {dataset=}") # Deleting all previously uploaded datasets for dataset in sdk.datasets.list(): dataset.delete() # Creating a tuning dataset for the YandexGPT Lite base model dataset_draft = sdk.datasets.draft_from_path( task_type="TextToTextGeneration", path="<file_path>", upload_format="jsonlines", name="YandexGPT Lite tuning", ) # Waiting for the data to be uploaded and the dataset to be created operation = dataset_draft.upload_deferred() tuning_dataset = operation.wait() print(f"new {tuning_dataset=}") if __name__ == "__main__": main()
Where:
-
<folder_ID>
: ID of the folder the service account was created in. -
<API_key>
: Service account API key you got earlier required for authentication in the API.The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.
-
<file_path>
: Path to the file containing the ready-made examples for model tuning.
-
-
Run the created file:
python3 dataset-create.py
Result:
new tuning_dataset=Dataset(id='fdsv21bmp09v********', folder_id='b1gt6g8ht345********', name= 'YandexGPT Lite tuning', description=None, metadata=None, created_by='ajegtlf2q28a********', created_at=datetime.datetime(2025, 3, 12, 19, 31, 44), updated_at=datetime.datetime(2025, 3, 12, 19, 32, 20), labels=None, allow_data_logging=False, status=<DatasetStatus.READY: 3>, task_type='TextToTextGeneration', rows=3, size_bytes=3514, validation_errors=())
Save the new dataset's ID (the
id
field value): you will need it when fine-tuning the model.
-
Create a dataset:
grpcurl \ -H "Authorization: Bearer <IAM_token>" \ -d @ \ llm.api.cloud.yandex.net:443 yandex.cloud.ai.dataset.v1.DatasetService/Create <<EOM { "folder_id": "<folder_ID>", "name": "My awesome dataset", "task_type": "TextToTextGeneration", "upload_format": "jsonlines" } EOM
Where:
<IAM_token>
: IAM token of the service account you got before you started.<folder_ID>
: ID of the folder you are creating the dataset in.
Result:
{ "datasetId": "fdso08c1u1cq********", "dataset": { "datasetId": "fdso08c1u1cq********", "folderId": "b1gt6g8ht345********", "name": "My awesome dataset", "status": "DRAFT", "taskType": "TextToTextGeneration", "createdAt": "2025-01-20T10:36:50Z", "updatedAt": "2025-01-20T10:36:50Z", "createdById": "ajeg2b2et02f********", "createdBy": "ajeg2b2et02f********" } }
Save the new dataset's ID (the
datasetId
field value): you will need it to upload data to the dataset. -
Get a link to upload data into the dataset:
grpcurl \ -H "Authorization: Bearer <IAM_token>" \ -d '{"dataset_id": "<dataset_ID>", "size_bytes": <dataset_size>}' \ llm.api.cloud.yandex.net:443 yandex.cloud.ai.dataset.v1.DatasetService/GetUploadDraftUrl | jq
Where:
<IAM_token>
: IAM token of the service account you got before you started.<dataset_ID>
: Dataset ID you saved in the previous step.<dataset_size>
: Size in bytes of the file with data for tuning. In the terminal, you can get the file size using thels -l <file_path>
command.
Result:
{ "datasetId": "fdso08c1u1cq********", "uploadUrl": "https://storage.yandexcloud.net/ai-fomo-drafts-prod/b1gt6g8ht345********/fdso08c1u1cq********?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20250120T105352Z&X-Amz-SignedHeaders=content-length%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=YCAJE_WuJJ9D1r6huCoc8I3yO%2F20250120%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=611d7951994ae939acf4d32cc0c154c738d02adb2a04707a704f34ca********" }
The
uploadUrl
field of the response contains a link you can use to upload your data into the dataset.Tip
If you did not use jq, replace all
\u0026
occurrences with&
in the link to use it to upload the dataset. -
Upload your data by specifying the link you got in the previous step and the path to the fine-tuning data file:
curl \ --request PUT \ --upload-file <path_to_file> \ "<link>"
-
After the data upload is complete, run the dataset validation:
grpcurl \ -H "Authorization: Bearer <IAM_token>" \ -d '{"dataset_id": "<dataset_ID>"}' \ llm.api.cloud.yandex.net:443 yandex.cloud.ai.dataset.v1.DatasetService/Validate
Where:
<IAM_token>
: IAM token of the service account you got before you started.<dataset_ID>
: Dataset ID you saved in the previous step.
Result:
{ "id": "fdso01v2jdd4********", "createdAt": "2025-01-20T11:03:48Z", "modifiedAt": "2025-01-20T11:03:48Z" }
Save the validation operation ID (
id
field). You will need it in the next step. -
Dataset validation may take some time. To find out validation status and get an error report (if any), send this request:
grpcurl \ -H "Authorization: Bearer <IAM_token>" \ -d '{"operation_id": "<validation_operation_ID>"}' \ llm.api.cloud.yandex.net:443 yandex.cloud.operation.OperationService/Get
Where:
<IAM_token>
: IAM token of the service account you got before you started.<validation_operation_ID>
: ID of the validation operation you saved in the previous step.
Result:
{ "id": "fdso01v2jdd4********", "createdAt": "2025-01-20T11:03:48Z", "modifiedAt": "2025-01-20T11:04:46Z", "done": true, "response": { "@type": "type.googleapis.com/yandex.cloud.ai.dataset.v1.ValidateDatasetResponse", "datasetId": "fdso08c1u1cq********", "isValid": true } }
The
isValid
field is set totrue
. This means the loaded dataset was validated successfully.
Start tuning
-
In the management console
, select the folder for which your account has theai.playground.user
,ai.datasets.user
, andai.models.editor
roles or higher. -
From the list of services, select Foundation Models.
-
In the left-hand panel, click
Fine-tuning models. -
Click Fine-tune model.
-
Enter a name and descriptions for the dataset. Follow these naming requirements:
- It must be from 2 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- It must start with a letter and cannot end with a hyphen.
-
In the Task field, select Generation.
-
Optionally, add or delete the tuning labels. You can use them to split or join resources into logical groups.
-
In the Model field, select the model you need.
-
In the Dataset field, click Add.
-
In the window that opens, go to the Select from existings tab and select the dataset you created earlier.
-
Click Advanced settings to do advanced fine-tuning setup.
-
Click Start fine-tuning.
-
Create a file named
start-tuning.py
and add the following code to it:#!/usr/bin/env python3 from __future__ import annotations import pathlib import uuid from yandex_cloud_ml_sdk import YCloudML def main(): sdk = YCloudML( folder_id="<folder_ID>", auth="<API_key>", ) # Viewing the list of valid datasets for dataset in sdk.datasets.list(status="READY", name_pattern="completions"): print(f"List of existing datasets {dataset=}") # Setting the tuning dataset and the base model train_dataset = sdk.datasets.get("<dataset_ID>") base_model = sdk.models.completions("yandexgpt-lite") # Starting the tuning # Tuning can last up to several hours tuned_model = base_model.tune( train_dataset, name=str(uuid.uuid4()), n_samples=10000 ) print(f"Resulting {tuned_model}") # You can access the fine-tuned model using the run() method completion_result = tuned_model.run("Hello!") print(f"{completion_result=}") # Or you can save the URI of the fine-tuned model # And call the fine-tuned model by its URI tuned_uri = tuned_model.uri model = sdk.models.completions(tuned_uri) completion_result = model.run("How are you doing?") print(f"{completion_result=}") if __name__ == "__main__": main()
Where:
-
<folder_ID>
: ID of the folder the service account was created in. -
<API_key>
: Service account API key you got earlier required for authentication in the API.The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.
-
<dataset_ID>
: The new dataset's ID you saved in the previous step.
-
-
Run the created file:
python3 start-tuning.py
Result:
Resulting GPTModel(uri=gpt://b1gt6g8ht345********/yandexgpt-lite/latest@tamrmgia22979********, config=GPTModelConfig(temperature=None, max_tokens=None, reasoning_mode=None, response_format=None)) completion_result=GPTModelResult(alternatives=(Alternative(role='assistant', text='Hi! How's it going?', status=<AlternativeStatus.FINAL: 3>),), usage=Usage(input_text_tokens=12, completion_tokens=5, total_tokens=17), model_version='23.10.2024') completion_result=GPTModelResult(alternatives=(Alternative(role='assistant', text='Hello! I'm fine, thank you. How are you doing?', status=<AlternativeStatus.FINAL: 3>),), usage=Usage (input_text_tokens=13, completion_tokens=13, total_tokens=26), model_version='23.10.2024')
Model tuning may take up to 1 day depending on the size of the dataset and the system load.
Use the fine-tuned model's URI you got (the
uri
field value) when accessing the model. -
Fine-tuning metrics are available in TensorBoard format. You can open the downloaded file, for example, in the Yandex DataSphere
project:metrics_url = new_model.get_metrics_url() download_tensorboard(metrics_url)
-
Start tuning:
grpcurl \ -H "Authorization: Bearer <IAM_token>" \ -d @ \ llm.api.cloud.yandex.net:443 yandex.cloud.ai.tuning.v1.TuningService/Tune <<EOM { "base_model_uri": "gpt://<folder_ID>/yandexgpt-lite/latest", "train_datasets": [{"dataset_id": "<dataset_ID>", "weight": 1.0}], "name": "my first model", "text_to_text_completion": {} } EOM
Where:
<IAM_token>
: IAM token of the service account you got before you started.<folder_ID>
: ID of the folder you are fine-tuning the model in.<dataset_ID>
: Dataset ID you saved in the previous step.
Result:
{ "id": "ftnlljf53kil********", "createdAt": "2025-01-20T11:17:33Z", "modifiedAt": "2025-01-20T11:17:33Z", "metadata": { "@type": "type.googleapis.com/yandex.cloud.ai.tuning.v1.TuningMetadata" } }
You will get the Operation object in response. Save the operation
id
you get in the response. -
Model tuning may take up to one day depending on the dataset size and the system load. To check if the fine-tuning is complete, request the operation status:
grpcurl \ -H "Authorization: Bearer <IAM_token>" \ -d '{"operation_id": "<operation_ID>"}' \ llm.api.cloud.yandex.net:443 yandex.cloud.operation.OperationService/Get
Where:
<IAM_token>
: IAM token of the service account you got before you started.<operation_ID>
: Model fine-tuning operation ID you got in the previous step.
If the fine-tuning process is over, the Operation object will contain the tuned model's URI in the
targetModelUri
field:{ "id": "ftnlljf53kil********", "createdAt": "2025-01-20T11:17:33Z", "modifiedAt": "2025-01-20T11:25:40Z", "done": true, "metadata": { "@type": "type.googleapis.com/yandex.cloud.ai.tuning.v1.TuningMetadata", "status": "COMPLETED", "tuningTaskId": "ftnlljf53kil********" }, "response": { "@type": "type.googleapis.com/yandex.cloud.ai.tuning.v1.TuningResponse", "status": "COMPLETED", "targetModelUri": "gpt://b1gt6g8ht345********/yandexgpt-lite/latest@tamr2nc6pev5e********", "tuningTaskId": "ftnlljf53kil********" } }
Use the fine-tuned model's URI you got (the
targetModelUri
field value) when accessing the model. -
Fine-tuning metrics are available in TensorBoard format. Get the link to download the file:
grpcurl \ -H "Authorization: Bearer <IAM_token>" \ -d '{"task_id": "<job_ID>"}' \ llm.api.cloud.yandex.net:443 yandex.cloud.ai.tuning.v1.TuningService/GetMetricsUrl
You can open the downloaded file, for example, in the Yandex DataSphere
project:
Accessing a fine-tuned model
Once the model is fine-tuned, save its URI in gpt://<base_model_URI>/<version>@<tuning_suffix>
format. Use it to send synchronous and asynchronous requests or create an AI assistant based on the fine-tuned model.
See also
- Model tuning
- Tuning a text classification model
- Model tuning in DataSphere
- For more SDK examples, see our GitHub repository
.