5. Collecting, monitoring, and analyzing audit logs

IntroductionIntroduction

An audit log is a record of all events in the system, including access to it and operations performed. By collecting and verifying audit logs, you can monitor compliance with the established security procedures and standards and identify vulnerabilities in your security mechanisms.

There are different levels of audit log events:

• Yandex Cloud level: Events related to Yandex Cloud resources.
• OS level.
• Application level.
• Network level (Flow Logs).

5.1 Yandex Audit Trails is enabled at the organization level5.1 Yandex Audit Trails is enabled at the organization level

The main tool for collecting Yandex Cloud level logs is Yandex Audit Trails. This service allows you to collect audit logs about events happening to Yandex Cloud resources and upload these logs to Yandex Object Storage buckets or Cloud Logging log groups for further analysis or export. For information on how to start collecting logs, see this guide.

Audit Trails audit logs may contain two types of events: management events and data events.

Management events are actions you take to configure Yandex Cloud resources, such as creating, updating, or deleting infrastructure components, users, or policies. Data events are updates and actions performed on data and resources within Yandex Cloud services. By default, Audit Trails does not log data events. You need to enable collection of data event audit logs individually for each supported service.

For more information, see Comparing management and data event logs.

To collect metrics, analyze Yandex Cloud-level events, and set up notifications, we recommend using Yandex Monitoring. For example, it can help you track spikes in Compute Cloud workload, Application Load Balancer RPS, or significant changes in Identity and Access Management event statistics.

You can also use Monitoring to monitor the health of the Audit Trails service itself and track security events. You can export metrics to a SIEM system via the API, see the instructions.

Solution: Monitoring Audit Trails and security events using Monitoring

You can export audit logs to a Cloud Logging or Data Streams log group and to a customer's SIEM system to analyze information about events and incidents.

List of important Yandex Cloud-level events to search for in audit logs:

Solution: Searching for important security events in audit logs

You can enable Yandex Audit Trails at the folder, cloud, and organization level. We recommend enabling Yandex Audit Trails at the level of the entire organization. Thus you will be able to collect audit logs in a centralized manner, e.g., to a separate security cloud.

5.2 Yandex Audit Trails events are exported to SIEM systems5.2 Yandex Audit Trails events are exported to SIEM systems

Solutions for exporting Yandex Cloud audit logs are available for the following SIEM systems:

• ArcSight: Collecting, monitoring, and analyzing audit logs in ArcSight SIEM

• Splunk: Collecting, monitoring, and analyzing audit logs in Splunk SIEM

• MaxPatrol SIEM: Collecting, monitoring, and analyzing audit logs in MaxPatrol SIEM

• Wazuh: Collecting, monitoring, and analyzing audit logs in Wazuh

For more information about MaxPatrol, see this section.

You can set up export to any SIEM using GeeseFS or s3fs. These utilities allow mounting a Yandex Object Storage bucket as a VM local disk. Next, you need to install a SIEM connector on the VM and configure reading JSON files from the bucket. You can also use utilities compatible with AWS Kinesis datastreams if sending audit logs to Yandex Data Streams.

If you have no SIEM, you can also analyze audit logs manually using one of the following methods (in descending order of convenience):

• Searching for Yandex Cloud events in Yandex Query.

• Searching for Yandex Cloud events in Cloud Logging.

• Searching for Yandex Cloud events in Object Storage.

5.3 Responding to Yandex Audit Trails events is set up5.3 Responding to Yandex Audit Trails events is set up

You can respond to Yandex Audit Trails events using your SIEM tools or manually. You can also use automatic responses.

Using Yandex Cloud Functions, you can configure alerts about Audit Trails events, as well as automatic responses to malicious actions, including removing dangerous rules or revoking access rights.

Solution: Notifications and responses to Audit Trails information security events using IAM / Cloud Functions + Telegram

5.4 Hardening of the Object Storage bucket that stores Yandex Audit Trails audit logs is done5.4 Hardening of the Object Storage bucket that stores Yandex Audit Trails audit logs is done

If you write Yandex Audit Trails audit logs to a Yandex Object Storage bucket, make sure the bucket is set up using best security practices, such as:

• 4.1 In Yandex Object Storage, encryption of data at rest using KMS keys is enabled.
• 3.8 In Yandex Object Storage, logging of actions with buckets is enabled.
• 3.8 In Yandex Object Storage, the Object locks feature is enabled.
• 3.7 In Yandex Object Storage, Bucket Policies are used.
• 3.6 No public access to the Yandex Object Storage bucket is allowed.

You can use a solution for secure Yandex Object Storage bucket setup with Terraform.

5.5 Audit logs are collected at the OS level5.5 Audit logs are collected at the OS level

When using IaaS cloud services and Kubernetes node groups, the customer is responsible for ensuring OS security and collecting OS-level events on their own. Free tools for collecting standard OS-generated events and exporting them to the customer's SIEM system include:

• Osquery
• Filebeat (ELK)
• Wazuh

Additional event generation options can be implemented using Auditd for Linux or Sysmon for Windows.

You can collect Linux system metrics (CPU, RAM, and disk space usage) with Unified Agent in Monitoring.

You can also export OS events to Cloud Logging using a Fluent Bit plugin or to Data Streams.

To describe events to be searched for in audit logs, we recommend using Sigma format, which is supported by popular SIEM systems. The Sigma repository contains a library of events described in this format.

To get the exact time of OS- and application-level events, configure clock synchronization by following this guide.

5.6 Audit logs are collected at the application level5.6 Audit logs are collected at the application level

Customers may collect events that occur at the level of applications deployed on Compute Cloud resources on their own. For example, save application logs to files and transfer them to a SIEM system using the tools listed in the subsection above.

5.7 Logs are collected at the network level5.7 Logs are collected at the network level

Currently, VPC network traffic event logs (Flow Logs) can only be collected by customers. You can use Yandex Cloud Marketplace solutions (such as NGFW, IDS/IPS, or network products) or free software for collecting and transmitting events. You can also collect network-level logs using different agents, e.g., HIDS.

5.8 Data events are monitored5.8 Data events are monitored

A data event audit log is a JSON object with a record of events related to Yandex Cloud resources. Data event monitoring makes it easier for you to collect additional events from cloud services and, as a result, effectively respond to security incidents in clouds. This also helps you ensure your cloud infrastructure meets regulatory requirements and industry standards. For example, you can keep track of your employees' access permissions to sensitive data stored in buckets.

ClickHouse® is a registered trademark of ClickHouse, Inc.