Data, training, models, and releases

The requirements of this section apply to lifecycle objects that actually exist. For managed inference without custom tuning, there are no requirements for the training environment or loaded weights; however, there are still requirements for input data, RAG corpus, application configuration, and model version. For RAG without tuning, a model release is a coordinated combination of the model, index, system instructions, and applied configuration.

Data intake and preprocessingData intake and preprocessing

Each dataset has an owner, provenance, and allowed purposeEach dataset has an owner, provenance, and allowed purpose

Requirement ID: AI-DATA1

Requirement

Before ingestion or use, the system owner must register the dataset or RAG corpus: source, owner, lifecycle stage, sensitivity category, purpose, version, transformations, use limitations, permitted recipients, and communication with the system, model, index, or release. For an external source, the receipt and acceptance conditions must be additionally recorded. Synthetic data is marked separately as per AI-DATA7.

The record is created before the set is ingested; if the set is changed, a new record version is formed. If the system uses only the managed model API and does not maintain its own sets, permitted data categories and the evaluation set are recorded anyway.

Implementation

Yandex Cloud provides storages and access control tools but does not form the registry of origin and permitted use for the organization. DataSphere delimits access to communities and projects; a published DataSphere access model does not replace the record of origin for a specific set. The record is associated with the exact object version in the actual storage: Object Storage object version, Vector Store ID or index ID, location in Yandex Managed Service for OpenSearch or Yandex Managed Service for YDB, or a DataSphere set.

Manual check

1. Select at least one internal, one external, and one synthetic set, if used.
2. Trace the process from a registry record to the actual object and version.
3. Additionally, select sample objects from the expanded corpus and trace each one up to the record; an object without a record gets rejected during ingestion or release, quarantined, and gets an owner, source, and classification, or gets deleted according to storage rules.

An unaccounted or unauthorized source must not end up in a training, evaluation, or index.

Artifact

• Registry record.
• Tracing up to object and version.
• Card version.
• Object version or hash.
• Approver.

RAG loader is restricted to an approved source and destinationRAG loader is restricted to an approved source and destination

Requirement ID: AI-RAG-INGEST1

Scope

This requirement applies if the system creates or updates a RAG corpus or a search index.

Requirement

The loader's service identity must read only from approved sources and write only to the designated staging area and the target index. It should not have the permission to modify another production corpus simply because it resides in the same cloud or folder.

The source, version, acceptance check result, loading entity, and publication decision are linked to the update operation. Loading and search identities are separated.

Prior to recording, the loader verifies the source card, type, size, hash, and mandatory tenant and access metadata; an unapproved package is placed in quarantine.

Implementation

For AI Search, the source object, file, vector store, and management roles are recorded.

For Managed Service for OpenSearch, YDB, or another storage solution, the specific roles, network boundaries, and configuration objects of the selected service are used.

The architecture can distribute reading from the source, writing to an intermediate area, and publishing across multiple identities.

In AI Search, an index is created by a documented flow (AI Studio → AI Search → search indexes); Terraform manages the infrastructure but not corpus admission.

Check

1. Read from the approved source, write to the designated staging area, and publish a test version.
2. Then, verify that reading from an unapproved source and writing to an unauthorized index or area are rejected.
3. Rejection due to a modified hash or an invalid tenant is checked separately, prior to indexing.

The record must link the published version to the source, verification, and change subject. In the event of a failure, the loader pauses, the package is quarantined, the affected index segment is deleted or rebuilt, the compromised identity is replaced, and the test is then repeated.

Artifact

• Successful upload of an approved object.
• Rejection of an unapproved source.
• Loader version.
• Acceptance and rejection events.
• Reindexing record.

Data stages are separatedData stages are separated

Requirement ID: AI-DATA3

Requirement

Source, cleaned, marked-up, training, evaluation, or published data must be distinguishable by objects or versions, write permissions, and advancement rules. Transition between stages takes place only after a relevant check and decision.

Updating an already approved object requires a new version and re-admission. The production environment must not change the production corpus directly. By default, the network and IAM must not give the record back or between parallel stages.

If one immutable input is used directly without derivative stages, the N/A result is allowed only for absent branches.

For Object Storage, you can use separate buckets or prefixes, access policies, and versioning.

For network storages available online, VPC security groups are used where the service supports them. The particular scheme must suit the selected service, not be transferred from one storage to another.

In AI Search, uploading a file and creating an index are treated as different lifecycle objects.

Check

Check the path of one record from the source object up to the published version, subjects authorized to write at each stage, and the impossibility to write from the production environment to the source or evaluation set. The derived results created during the violation are recognized invalid, and the pipeline is repeated after narrowing down the access.

Artifact

• Bucket/prefix access policies.
• Record test from the production environment.
• Stage schema.
• Input and output versions.
• Transition log.

Data was classified and minimized prior to being forwarded for AI processingData was classified and minimized prior to being forwarded for AI processing

Requirement ID: AI-DATA4

Requirement

For each field or object type, the data owner must define a sensitivity category, allowed recipients, and an allowed action: retain, delete, aggregate, mask, or tokenize. Fields, records, attachments, and secrets that are irrelevant for the stated scenario must be removed before transmission.

If using reversible transformation, its execution location, acceptable reversibility, and subjects with access to the key or mapping table must be defined.

Fields are mapped based on the destination: prompt, RAG context, tool data, external call, and log event; data must be deleted or transformed before crossing the boundary, not after delivery to the provider or model.

Implementation

Data Security Posture Management (DSPM) can be used to search for specific data categories in supported sources. It does not transform the source data or establish the lawful basis for processing.

For info on the source scope, service account with the dspm.worker role, and extra permissions for KMS-encrypted buckets, see the DSPM guide on creating a scan. Discoveries from DSPM or NER are not transformations: the detected candidate must confirmed by a human reviewer or application logic.

Check

1. Take a sample from the actual input dataset post-transformation and check it against the approved schema.
2. Try to upload prohibited fields or object.

A detection report alone, without source remediation, does not confirm minimization. Any additional sensitive field must be blocked or modified before the call and must not appear in logs. If the test fails, the affected flow must be stopped, and all cached, indexed, and logged copies must be deleted according to applicable retention policies and legal requirements. After that, the test is must be repeated.

Artifact

• Field schema.
• Post-transformation selection.
• Rejection of prohibited fields.
• Field mapping by destination.
• Record of copy deletion.

Data quality and independence of assessment are verified prior to releaseData quality and independence of assessment are verified prior to release

Requirement ID: AI-DATA5

Requirement

The organization must define the input versions, metrics, thresholds, and known limits for checking quality, completeness, and acceptable bias for a specific scenario. Training and evaluation samples must be separated so that leakage between them does not compromise check results. For RAG, chunking quality, relevance, metadata completeness, and search reproducibility must be checked. Checks must evaluate completeness, duplicates, schema, distribution, sources, and overlap between the training and evaluation samples.

If synthetic data is used, the report must specify its proportion, retrieval method, and impact on the score. There is no single cloud-wide quality threshold. The product owner must approve the criteria in coordination with the risk officer.

The result must be approved by an independent reviewer, non-affiliated with the data producer. The report must be linked to the exact version of the dataset, model, or index. If there is no managed dataset, the N/A result is only acceptable if the model or API selection undergoes independent review.

Manual check

1. Repeat the approved assessment on a fixed data version and configuration.
2. Make sure there are no matching IDs or other signs of leakage. Also, check the report you saved specifying the initial version, criteria, and result.

Any data batch containing duplicates, corrupted schemas, prohibited data, or overlaps between the training and evaluation samples must block the release. Such batches must be quarantined, the source or transformation fixed, a new report generated, and check repeated.

Artifact

• Saved report with version, criteria, and result.
• Code version for checks and tests.
• Dataset hashes.
• Independent reviewer.

Data and/or RAG corpus changes are controlledData and/or RAG corpus changes are controlled

Requirement ID: AI-DATA6

Requirement

Source, intermediate, and published versions must have verifiable IDs or checksums. For every process, the system must record the owner, change creator, code or configuration version, performed checks, decision to publish, and rollback procedure to restore the previous secure version. Ingestion, cleansing, chunking, indexing, and publication must be handled by a trusted identity through a repeatable workflow. An non-verified object must not become available to the production system.

The acceptance check must align with the source risk level and, prior to publication, detect unsupported formats, corrupted structure, unexpected executable content, and categories prohibited by the organization. Model traffic moderation does not replace a full review of the dataset.

Implementation

In Object Storage, you can enable versioning to track object history, and, to enforce immutability, use object locks considering their rules and timelines.

In AI Search, uploading files and creating a vector store is part of the documented indexing process. The application must separately implement document authorization and content admission.

A manifest maps a logical release to the exact IDs and hashes of objects, files, and indexes. Each change creates a new version and enforces a new approval; the only copy is never overwritten. The object lock is enabled only after the retention period is approved: locked versions may become undeletable. For storages without versioning, an immutable release copy and change record are saved.

Check

1. Trace a published version from the source object to down to the index or dataset.
2. Attempt to bypass a check and publish a modified object.
3. As a separate step, use a synthetic object to check that a new version is created while you retain access to the previous version and can use it for recovery.

The requirement is met if the bypass attempt is rejected, and the saved data allow you to identify the active version and change initiator.

Artifact

• Version-to-source mapping.
• Check bypass test.
• Release manifest with hashes.
• Change event.
• Recovery test.

Synthetic data is accounted for separatelySynthetic data is accounted for separately

Requirement ID: AI-DATA7

Scope

This requirement applies if synthetic data is used for training, fine-tuning, or evaluation.

Requirement

For the synthetic portion of the dataset, the following must be registered: generation model or method, generator version and parameters, purpose, source data, share of synthetic records, and known limitations. Synthetic records go through the same admission and quality assurance stages as other data; their provenance must not be mixed with observed data.

Each synthetic record or batch gets the synthetic=true label; unprocessed generated data is not included into training or evaluation unless explicitly approved. The final batch receives a separate ID and dataset hash.

Implementation

When generated via AI Studio or DataSphere, the record contains the available IDs of the model, API, project, or task. Yandex Cloud does not maintain a universal registry of synthetic data for the organization.

Check

1. Trace the synthetic sample version back to the generator, source data, and parameters.
2. Check that the report shows its share and separately evaluates the required properties versus a non-synthetic control sample.

A generated test case without a label must be rejected at ingestion.

Artifact

• Synthetic portion record.
• Share and control sample report.
• Generator version.
• Unlabeled data rejection test result.

Training and model artifactsTraining and model artifacts

Development, training, and production execution are separatedDevelopment, training, and production execution are separated

Requirement ID: AI-TRAIN1

Requirement

If the organization trains, fine-tunes, or runs experimental models, such workflows should be isolated from the production system at the project, namespace, service account, access control, storage, and network path levels in line with the selected environment. Experimental workloads must never have access to production secrets or permissions to change production data simply because they run in the same cloud.

For each job, a secret must only be provided to its workload identity with minimal privileges. Do not store secrets in notebooks, datasets, container images, or logs. Limit outgoing traffic to approved sources, recipients, and scopes. Open internet access may allowed as a rare exception that has an owner, time limit, and compensatory control.

The secret delivery method and network controls are environment-specific and must be reviewed against the requirements from IAM, network, encryption, secrets, and audit and Implementation profiles in Yandex Cloud.

Implementation

In DataSphere, permissions are granted at the community and project levels based on this access management guide.

Managed Service for Kubernetes also separates IAM cluster access and Kubernetes RBAC. For namespace, network policy, and metadata security best practices, refer to this Kubernetes security guide.

For systems that only access a managed model API without training, the N/A result applies to this entire requirement.

For DataSphere, the service account, project network and security groups as well as job evidence are recorded. Management events do not prove the origin of data within a notebook or job.

For Kubernetes, a separate namespace or cluster with a workload identity, RBAC, and network policy is used.

For Compute Cloud, a dedicated service account, security groups, encrypted disk, and secret delivery path are configured.

A workload identity does not modify the production artifact pointer; the release copies the exact approved artifact.

Check

1. Compare the users and service accounts across the experimental and production environments.
2. Attempt to read a production or another subject’s test secret, modify a production object, or access a restricted endpoint from the experimental environment and unrelated job.
3. Also confirm that a development or training subject cannot deploy or overwrite a production version.

An allowed job must only get the assigned secret and source; unauthorized actions must be denied, and logs must not contain the secret value. Acceptable variance must be documented as an exception. Artifacts created within a compromised boundary are considered invalid and must be rebuilt once isolated.

Artifact

• Contour mapping.
• Unauthorized secret read and production write test.
• Hashes of job input and output objects.

Model package has verifiable provenanceModel package has verifiable provenance

Requirement ID: AI-MODEL1

Requirement

For a self-hosted or third-party model, the owner must record its source, version, checksum, licensing and operational limitations, weight format, tokenizer, configuration, ingestion code, dependent software components, related data versions, approval status, and check results.

For a managed API, the model vendor, API, ID, or version as well as available parameters affecting its behavior are recorded instead of unavailable weights.

The record must uniquely identify the package that has passed the check and is now used in the release. Approved artifacts are copied to a controlled storage; in a production environment, direct ingestion based on a dynamic external URL is not permitted because the production environment only ingests an exact digest.

Object Storage can store the package files, Yandex Container Registry can store Docker images, and DataSphere can store documented resource types, including models, Docker images, and file storages. The organization must link together the actual objects and versions in a separate provenance record no matter where they are stored. See the description of resource boundaries in the Container Registry and DataSphere guides.

Manual check

1. Find out the model package of the running instance and compare it with the admission record.
2. Recalculate the digest of deployed files and images and compare it against the record. A modified artifact or unknown source would block admission be quarantined.

A mismatch of weight, configuration, loader, or dependency requires a new check.

Artifact

• Provenance record.
• Mapping to a running instance.
• Model card.
• Hashes and versions of objects in storage.

Model format and loader do not allow arbitrary executionModel format and loader do not allow arbitrary execution

Requirement ID: AI-MODEL2

Requirement

Before using an external model package for the first time, the organization must assess the properties of its format, supporting code, custom loader and dependencies, eliminate unnecessary code execution, verify the checksum, and run the first launch in an isolated environment without production secrets or write access to production stores. Choosing a format depends on the risk. The extension name alone does not prove security.

The admission rules must list the allowed formats and exact versions of loaders for the framework in use. Executable deserialization of an untrusted artifact must not be used if a compatible secure format exists.

Implementation

The Container Registry scanner analyzes Docker image vulnerabilities rather than model behavior, weight contents, or data poisoning. For information on scanner scope, refer to this guide.

Check

Repeat running the fixed package in an isolated environment while observing file, network, and process activities. If you observe unexpected code execution, external state changes, or checksum mismatch, using the package is prohibited. The initial check concludes upon an expected smoke test result, after which the check’s one-time environment is wiped out.

Artifact

• Isolated execution result.
• Activity monitoring.
• Format and loader versions.
• Isolated environment configuration.
• Admission decision.

Release, rollback, and revocationRelease, rollback, and revocation

Only the exact verified version is admitted into the production environmentOnly the exact verified version is admitted into the production environment

Requirement ID: AI-REL1

Requirement

The release decision must bind the exact versions of data or index, model package, system instructions, app configuration, image, check results, current exceptions, and the approvers. Building and publishing must be handled by a trusted process. The release result must match the approved package; manual component swapping after the check is prohibited.

The deployment must use the release manifest, not the default tag or name. For serverless runtimes, use the exact revision or version, for Kubernetes, image digest and configuration, and for AI Studio, IDs of the model instance, guardrail, and index.

Implementation

For a containerized release, an image scanning result may be part of the release decision, but the organization must set the acceptance threshold and deadline for fixing vulnerabilities. Image scanning does not replace the model package and data checks.

Check

Use a production revision to restore all listed versions and the release decision. The requirement is met if the model package is unambiguous, reproducible, and it matches the approved one. The latest tag or an unapproved discrepancy between the actual revision and manifest means failure.

Artifact

• Release decision.
• Production revision mapping.
• Release manifest hash.
• Deployment event.

Version replacement, rollback, and revocation are managed explicitlyVersion replacement, rollback, and revocation are managed explicitly

Requirement ID: AI-REL2

Requirement

The model, index, and associated configuration must each have defined compatibility requirements, phased replacement sequence, rollback criteria, previous safe compatible set, and method for revoking an unsafe version. A rollback must not reintroduce a known vulnerability, prohibited data, or a revoked secret. Revocation must cover every production endpoint, alias, direct reference, and workaround that could have been used to select the version.

Implementation

Service-specific tools are used only within their documented scope: Serverless Containers revisions and deployment and rollback events from the Audit Trails reference, Object Storage object versions, or the application's own versioning scheme. The availability of object history does not guarantee a functional rollback of the entire AI system.

The rollback target is defined per environment: for serverless environments, it is the previous verified revision or version; for Kubernetes, the previous image and configuration; for Compute Cloud, the image, disk, and configuration recovery plan; for RAG, the previous corpus or index manifest, or a rebuild procedure; for AI Studio, the previous instance, guardrail, and index configurations.

Before release, backward compatibility of the data and index is verified. A reference to a previous version without a specific ID and verified compatibility is not considered a rollback plan.

Check

In the test environment, perform the replacement and rollback of the related package.

1. Check that the active version can be identified, the log associates the initiator with the change, and the revoked version cannot be selected through the standard process.
2. The rollback rehearsal includes switching to the previous target, functional and security checks; reverting to the new version can only be performed after approval.

A rollback is not considered valid if it exposes data from a newer state.

Artifact

• Replacement and rollback test.
• Initiator log.
• Revoked version unavailability.
• Rollback target IDs.
• Return approval.