Creating a trail to upload audit logs

1. Create a log group the audit logs will be uploaded to.

2. Create a service account for the trail.

3. Assign roles to the service account for the trail to be able to collect and upload logs:

   • logging.writer for a log group.

   • audit-trails.viewer for one of the resources that defines the required log collection scope:

     • Organization: To collect logs in selected clouds of the organization.
     • Cloud: To collect logs in selected folders of the cloud.
     • Folder: To collect logs in this folder.

     Child resources inherit access permissions from their parent resources. For example, if a service account gets a role for a cloud, then the trail that uses this account will be able to collect logs for resources across all folders of this cloud. However, the trail will not be able to collect logs in other clouds owned by the organization: a role for the organization is required for that.

4. Make sure your account has the roles required for creating a trail:

   • audit-trails.editor for the folder to host the trail.
   • iam.serviceAccounts.user for the trail’s service account.

1. Create a data stream the audit logs will be uploaded to.

2. Create a service account for the trail.

3. Assign roles to the service account for the trail to be able to collect and upload logs:

   • yds.editor for a data stream.

   • audit-trails.viewer for one of the resources that defines the required log collection scope:

     • Organization: To collect logs in selected clouds of the organization.
     • Cloud: To collect logs in selected folders of the cloud.
     • Folder: To collect logs in this folder.

     Child resources inherit access permissions from their parent resources. For example, if a service account gets a role for a cloud, then the trail that uses this account will be able to collect logs for resources across all folders of this cloud. However, the trail will not be able to collect logs in other clouds owned by the organization: a role for the organization is required for that.

4. Make sure your account has the roles required for creating a trail:

   • audit-trails.editor for the folder to host the trail.
   • iam.serviceAccounts.user for the trail’s service account.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameters.

See the description of the CLI trail creation command for details about the arguments you can use:

yc audit-trails trail create --help

You can create a trail by specifying its parameters in one of these two ways:

yc audit-trails trail create \
  --name <trail_name> \
  --description <trail_description> \
  --labels <label_list> \
  --service-account-id <service_account_ID> \
  --destination-bucket <bucket_name> \
  --destination-bucket-object-prefix <prefix_for_objects> \
  --destination-log-group-id <log_group_ID> \
  --destination-yds-stream <YDS_name> \
  --destination-yds-database-id <YDS_database_ID> \
  --filter-all-folder-id <folder_ID> \
  --filter-all-cloud-id <cloud_ID> \
  --filter-all-organisation-id <organization_ID> \
  --filter-some-folder-ids <cloud_folder_list> \
  --filter-from-cloud-id <cloud_ID_with_selected_folders> \
  --filter-some-cloud-ids <list_of_clouds_in_organization> \
  --filter-from-organisation-id <organization_ID_with_selected_clouds>

With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.

Terraform is distributed under the Business Source License. The Yandex Cloud provider for Terraform is distributed under the MPL-2.0 license.

For more information about the provider resources, see the documentation on the Terraform website or mirror website.

If you don't have Terraform, install it and configure the Yandex Cloud provider.

1. In the configuration file, describe the parameters of the trail to collect audit logs:

   resource "yandex_audit_trails_trail" "basic_trail" {
     name        = "<trail_name>"
     folder_id   = "<folder_ID>"
     description = "<trail_description>"
     labels = {
       key = "value"
     }
     service_account_id = "<service_account_ID>"
   
   
     # Only one destination must be specified:
     # storage_destination , logging_destination, data_stream_destination
     # Settings for all destinations are provided for illustration purposes.
   
     logging_destination {
       log_group_id = "<log_group_ID>"
     }
     storage_destination {
       bucket_name   = "<bucket_ID>"
       object_prefix = "<prefix>"
     }
     data_stream_destination {
       database_id = "<YDS_database_ID>"
       stream_name = "<YDS_name>"
     }
   
     # Filtering policy settings
   
     filtering_policy {
       management_events_filter {
         resource_scope {
           resource_id   = "<organization_ID>"
           resource_type = "resource-manager.organization"
         }
       }  
       data_events_filter {
         service = "<service>"
         included_events = ["<service_event_type>","<service_2_event_type>"]
         resource_scope {
           resource_id   = "<cloud_ID>"
           resource_type = "resource-manager.cloud"
         }
         resource_scope {
           resource_id   = "<folder_ID>"
           resource_type = "resource-manager.folder"
         }
       }
       data_events_filter {
         service = "<service_2>"
         resource_scope {
           resource_id   = "<ID_of_cloud_2>"
           resource_type = "resource-manager.cloud"
         }
         resource_scope {
           resource_id   = "<cloud_3_ID>"
           resource_type = "resource-manager.cloud"
         }
       }
       data_events_filter {
         service = "<service_3>"
         resource_scope {
           resource_id   = "<folder_2_ID>"
           resource_type = "resource-manager.folder"
         }
         resource_scope {
           resource_id   = "<folder_3_ID>"
           resource_type = "resource-manager.folder"
         }
       }
     }
   }
   

   Where:

   • name: Name of the new trail. The naming requirements are as follows:

     • It must be 2 to 63 characters long.
     • It may contain lowercase Latin letters, numbers, and hyphens.
     • It must start with a letter and cannot end with a hyphen.

   • folder_id: ID of the folder to create the trail in.

   • description: Description of the trail to distinguish it from other trails, e.g., My very first trail. This is an optional parameter.

   • labels: List of labels in key=value format. This is an optional parameter.

   • service_account_id: ID of the service account the trail will use to upload audit log files to the bucket.

     Based on the audit log collection scope, the service account must have the audit-trails.viewer role for the organization, cloud, or folder whose audit logs the trail will collect.

   Warning

   Only one destination must be specified: storage_destination, logging_destination, or data_stream_destination.

   • logging_destination: Uploading logs to a Yandex Cloud Logging group.

     • log_group_id: ID of the log group for the trail to save audit logs to.

   • storage_destination: Uploading logs to a Yandex Object Storage bucket:

     • log_group_id: Name of the bucket for the trail to save audit logs to.

     • object_prefix: Prefix that will be assigned to the objects with audit logs in the bucket. It is an optional parameter used in the full name of the audit log file.

       Note

       Use a prefix to store audit logs and third-party data in the same bucket. Do not use the same prefix for logs and other bucket objects because that may cause logs and third-party objects to overwrite each other.

   • data_stream_destination: Uploading logs to a data stream in Yandex Data Streams:

     • stream_name: Name of the data stream for the trail to save audit logs to.
     • database_id: ID of the Yandex Managed Service for YDB database used by Data Streams.

   • filtering_policy: Settings of the filtering policy that determines which events to collect and include in the audit logs. The policy consists of filters pertaining to different levels of events. It contains the management_events_filter and data_events_filters objects.

     • management_events_filter: Management event filter.

     • resource_scopes: Log collection scope. You can combine several scopes belonging to the same organization in one resource_scopes parameter. For example, you can collect logs from one entire cloud and only from particular folders in another cloud. Service account permissions must allow collecting logs from the specified scopes.

       • resource_id: ID of the resource for whose resources audit logs will be collected. Based on the audit log collection scope, specify the organization ID or the cloud ID in this parameter.

       • resource_type: Scope type according to the specified ID:

         • organization-manager.organization: Organization.
         • resource-manager.cloud: Cloud.
         • resource-manager.folder: Folder.

     • data_events_filters: Data event filters. You can configure several filters of this type, one filter per service. A filter for one service has the following structure:

       • service: Name of the service in which the trail will process events. You can get it from the data event reference.
       • resource_scopes: Places to collect data events from. You can configure this parameter the same way as the management event filter.
       • included_events: Collect only specified events. This is an optional parameter. If not specified, all events will be collected. To collect all events except the specified ones, replace included_events with excluded_events. These parameters are mutually exclusive.
         You can get a full list of events from the data event reference.

   For more information about the yandex_audit_trails_trail resource parameters in Terraform, see the provider documentation.

2. Create the resources:

   1. In the terminal, change to the folder where you edited the configuration file.

   2. Make sure the configuration file is correct using the command:

      terraform validate
      

      If the configuration is correct, the following message is returned:

      Success! The configuration is valid.
      

   3. Run the command:

      terraform plan
      

      The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.

   4. Apply the configuration changes:

      terraform apply
      

   5. Confirm the changes: type yes in the terminal and press Enter.

   Terraform will create all the required resources. You can check the new resources and their settings using the management console or this CLI command:

   yc audit-trails trail get <trail_name>
   

Use the create REST API method for the Trail resource or the TrailService/Create gRPC API call.

1. In the Terraform configuration file, describe the parameters of the trail you want to create:

   resource "yandex_audit_trails_trail" "basic_trail" {
     name               = "sample-trail-all-planes"
     folder_id          = "folder0***"
     service_account_id = "service0***"
   
     storage_destination {
       bucket_name  = "sample-logs-bucket"
     }
   
     filtering_policy {
       management_events_filter {
         resource_scope {
           resource_id   = "org1***"
           resource_type = "resource-manager.organization"
         }
       }  
       data_events_filter {
         service = "mdb.postgresql"
         excluded_events = ["yandex.cloud.audit.mdb.postgresql.CreateDatabase","yandex.cloud.audit.mdb.postgresql.UpdateDatabase"]
         resource_scope {
           resource_id   = "cloud1***"
           resource_type = "resource-manager.cloud"
         }
         resource_scope {
           resource_id   = "folder1***"
           resource_type = "resource-manager.folder"
         }
       }
       data_events_filter {
         service = "storage"
         resource_scope {
           resource_id   = "cloud2***"
           resource_type = "resource-manager.cloud"
         }
         resource_scope {
           resource_id   = "cloud3***"
           resource_type = "resource-manager.cloud"
         }
       }
       data_events_filter {
         service = "compute"
         resource_scope {
           resource_id   = "folder2***"
           resource_type = "resource-manager.folder"
         }
         resource_scope {
           resource_id   = "folder3***"
           resource_type = "resource-manager.folder"
         }
       }
     }
   }
   

2. Create the resources:

   1. In the terminal, change to the folder where you edited the configuration file.

   2. Make sure the configuration file is correct using the command:

      terraform validate
      

      If the configuration is correct, the following message is returned:

      Success! The configuration is valid.
      

   3. Run the command:

      terraform plan
      

      The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.

   4. Apply the configuration changes:

      terraform apply
      

   5. Confirm the changes: type yes in the terminal and press Enter.

   A trail will be created with the specified parameters. You can check the new trail using the management console or this CLI command:

   yc audit-trails trail get sample-trail-all-planes
   

Use the create REST API method for the Trail resource.

To use the examples, install cURL.

The example below is intended to be run in MacOS and Linux. To run it in Windows, see how to work with Bash in Microsoft Windows.

1. Get an IAM token for authentication with the API.

2. Save the token to a variable and run this command in the terminal:

   export IAM_TOKEN=<IAM_token>
   

3. Create a file named body.json with the following request body and a description of the new trail:

   {
     "folderId": "folder0**",
     "name": "sample-trail-all-planes",
     "description": "sample-trail",
     "destination": {
       "objectStorage": {
         "bucketId": "sample-logs-bucket"
       }
     },
     "serviceAccountId": "service0***",
     "filteringPolicy": {
       "managementEventsFilter": {
         "resourceScopes": [
           {
             "id": "org1***",
             "type": "resource-manager.organization"
           }
         ]
       },
       "dataEventsFilters": [
         {
           "service": "mdb.postgresql",
           "excludedEvents": {
             "eventTypes": [
               "yandex.cloud.audit.mdb.postgresql.CreateDatabase"
               ,"yandex.cloud.audit.mdb.postgresql.UpdateDatabase"
             ]
           },
           "resourceScopes": [
             {
               "id": "cloud1***",
               "type": "resource-manager.cloud"
             },
             {
               "id": "folder1***",
               "type": "resource-manager.folder"
             }
           ]
         },
         {
           "service": "storage",
           "resourceScopes": [
             {
               "id": "cloud2**",
               "type": "resource-manager.cloud"
             },
             {
               "id": "cloud3**",
               "type": "resource-manager.cloud"
             }
           ]
         },
         {
           "service": "compute",
           "resourceScopes": [
             {
               "id": "folder2**",
               "type": "resource-manager.folder"
             },
             {
               "id": "folder3**",
               "type": "resource-manager.folder"
             }
           ]
         }
       ]
     }
   }
   

4. Run this request in your terminal:

   curl \
     --request POST \
     --header "Authorization: Bearer ${IAM_TOKEN}" \
     --data "@<request_body_file>" \
     https://audittrails.api.cloud.yandex.net/audit-trails/v1/trails
   

   Where:

   • <request_body_file>: Path to the previously created request body file (body.json).

   Result:

   {
   "done": true,
   "metadata": {
     "@type": "type.googleapis.com/yandex.cloud.audittrails.v1.CreateTrailMetadata",
     "trailId": "cnpvprd5pa66********"
   },
   "id": "cnp9qb9g8ldb********",
   "description": "operation_create",
   "createdAt": "2025-02-20T07:06:18.547321903Z",
   "createdBy": "ajevfb0tjfts********",
   "modifiedAt": "2025-02-20T07:06:18.547321903Z"
   }