Standard implementation and monitoring metrics
Written by
Updated at October 20, 2026
Section status: Reference. The metrics are provided as a basis for building an indicator system and are not mandatory requirements of this standard. Before formal adoption, the calculation formula, measurement period, unit of measurement, related
AI-*requirements, and result interpretation must be defined for each metric.
Metrics are there to monitor how requirements are implemented, but as such they do not prove the system is secure. Their list, frequency, and target values are determined by the organization with risk and architecture in mind.
If a metric's value cannot be calculated because there is no applicability condition in the architecture (e.g., there are no agent scenarios), it is marked as N/A. If the denominator is below the organization-specific threshold, the metric is considered N/A with an explanation.
The absence of a registry, end-to-end tracing, alerts, or any other data source required to calculate an applicable metric is not considered N/A: this fact gets registered as lack of control or data, which needs to be fixed.
Standard coverage and compliance
| № | Metric | Scope | Owner | Data source | Verification artifact | N/A / zero denominator rule |
|---|---|---|---|---|---|---|
| 1 | Percentage of covered AI systems | The organization operates AI systems. | Information security owner / architect | Registry of AI systems, architecture description reports | Registry entry with the system ID, date, and approver | If there are no systems in production: N/A. |
| 2 | Compliance level | The organization performs compliance review. | Information security owner / reviewer | Non-compliance tracking system, corrective action plan | Compliance report with a list of requirements and conclusions | If the requirements are not defined: N/A. |
| 3 | Mean time to remediate non-compliance (MTTR compliance) | The review found unfulfilled requirements or exceptions. | Information security owner / risk manager | Non-compliance tracking system, opening/closing dates | Non-compliance log with dates, resolutions, and responsible employees | If no non-compliance is found: N/A (zero denominator). |
| 4 | Number and age of exceptions | There are approved exceptions in effect. | Information security owner / risk manager | Register of exceptions as described in Exceptions | Exception record with an expiration date, reason, and compensatory measure | If there are no exceptions: 0 with a note. If there is no register: N/A. |
| 5 | Legal assessment coverage | The system processes regulated data. | Persons responsible for legal matters / compliance | Compliance decision register, list of systems processing personal data | Approved legal and compliance approach, data flow diagram | If no regulated data is processed: N/A. |
Identification, access, and secret management
| № | Metric | Scope | Owner | Data source | Verification artifact | N/A / zero denominator rule |
|---|---|---|---|---|---|---|
| 6 | Excessive privileges index (IAM Hygiene) | The system uses workload identities | Information security owner / IAM | Export of effective IAM and/or CIEM (if availability is confirmed) federated credentials | Comparison table: subject, role, operation, decision | N/A if there are no workload identities |
| 7 | Share of keyless access (Workload Identity) | The runtime receives cloud credentials | DevSecOps / platform | Runtime configuration, presence of an authorized key | Configuration record: environment, method of obtaining credentials, or N/A for environments without cloud access |
N/A with justification if all environments use long-lived keys under an approved exception |
| 8 | Timeliness of secret rotation | Long-lived secrets are used (API keys, external provider tokens, passwords) | Information security owner / secret manager | Secret registry, rotation dates | Record: secret ID, rotation date, approved TTL, decision | Short-lived IAM tokens are not included in rotation; N/A if there are no long-lived secrets |
| 9 | Share of secrets in approved storage | The system uses secrets | Information security owner / secret manager | Secret registry, delivery configuration | Record: secret ID, storage method, delivery | N/A if there are no secrets; approved storage is named by the organization (does not have to be Lockbox) |
Data and pipeline protection
| № | Metric | Scope | Owner | Data source | Verification artifact | N/A / zero denominator rule |
|---|---|---|---|---|---|---|
| 10 | Personal data detection quality (Detector Quality) | The system uses protective transformation of regulated data | Data owner / ML engineer | Results of synthetic tests of the detector | Test report: rule version, accuracy or completeness, or other approved measure, date, test set | The metric evaluates the detector, not the masking itself; if no transformation is used: N/A |
| 11 | Share of checked data ingestions | The system creates or updates a RAG corpus, trains or fine-tunes models | Data owner / ML Ops | Acceptance check execution log, quarantine, decisions | Record: ingestion ID, check start event, date, initiator, decision | Denominator: all applicable ingestions during the period, including ingestions without a check; if there are no applicable ingestions for the period: N/A |
| 12 | Training data isolation level | The organization trains models or experiments with them | ML platform owner | DataSphere logs, project registries, information security incidents | Incident record describing the detection of production data in the experimental environment | If there are no experimental environments: N/A |
| 13 | Share of checked third-party models | Third-party model packages are used | AI product owner | Model registry, acceptance records | Acceptance record: version, checksum, decision, date | If there are no external models: N/A |
Security of models, agents, and RAG
| № | Metric | Scope | Owner | Data source | Verification artifact | N/A / zero denominator rule |
|---|---|---|---|---|---|---|
| 14 | Guardrails activation rate | Guardrails are used (AI-CONTENT1). |
Product owner / content manager | Guardrail events, application logs | Record: category, action, date, session | If guardrails are not used as a security measure: N/A |
| 15 | Share of stopped agent loops | The system runs agent scenarios. | Agent owner / orchestrator | Agent logs, orchestrator metrics | Record: reason for termination, step number, limit | If there are no agent scenarios: N/A |
| 16 | RAG ACL effectiveness | RAG contains documents with different permissions. | RAG owner / developer | Automated test results for document authorization | Protocol: allowed and prohibited subject, result before model call | If RAG operates without access control: N/A |
| 17 | HITL rate | The system identifies actions with significant consequences. | Agent / security owner | Confirmation logs, agent traces | Record: action, confirmation subject, parameter version | If there are no actions requiring HITL: N/A |
Vulnerability management and supply chain
| № | Metric | Scope | Owner | Data source | Verification artifact | N/A / zero denominator rule |
|---|---|---|---|---|---|---|
| 18 | Mean time to remediate critical vulnerabilities (MMTR Critical Vulns) | Software images or dependencies are used | DevSecOps / application owner | Vulnerability scanner, issue tracker, release history | Record: vulnerability ID, detection date, fix date, release | If no critical vulnerabilities are found for the period: N/A (zero denominator) |
| 19 | Share of components with an up-to-date SBOM | Organization requires to generate a SBOM for releases | Information security owner / DevSecOps | Release registry, SBOM manifests | Saved SBOM, connection with release digest | SBOM counts only where it is required by the policy; if not required, N/A |
| 20 | Incident response speed by vendors | External components are used (models, data, libraries) | Information security owner / supply chain | Vendor notification registry, issue tracker | Record: notification, date of receipt, date of impact assessment, date of restrictive measures | If there are no external components: N/A |
Monitoring, incidents, and observability
| № | Metric | Scope | Owner | Data source | Verification artifact | N/A / zero denominator rule |
|---|---|---|---|---|---|---|
| 21 | Application trace completeness | The system processes requests requiring end-to-end tracing | Monitoring owner / DevOps | Application logs, correlation records | Record sample with correlation ID, policies, and tools | If no end-to-end trace is declared: N/A |
| 22 | Mean time to detect AI anomalies (MTTD AI) | The system has active alerts or YCDR with confirmed availability | SOC / monitoring owner | Alert events, Monitoring logs, YCDR (with availability and a configured collector) | Record: anomaly start time, alert trigger time, delivery channel | YCDR counts only for systems in which it is available and collects events; if no alerts are configured: N/A |
| 23 | Mean time to respond to AI incidents (MTTR AI) | AI incident requiring a response is registered | SOC / incident manager | Incident tracker, action logs | Record: alert time, time of first isolation step, action | If there are no incidents during the period: N/A (zero denominator) |
| 24 | Share of incidents with saved investigation materials | Incidents requiring investigation are registered | SOC / information security owner | Incident registry, archives of investigation materials | Incident report with a list of materials, period, owner | If there are no incidents during the period: N/A (zero denominator) |