5. Collecting, monitoring, and analyzing audit logs
Introduction
An audit log is a record of all events in the system, including access to it and operations performed. By collecting and verifying audit logs, you can monitor compliance with the established security procedures and standards and identify vulnerabilities in your security mechanisms.
Events in audit logs occur on different levels:
- Yandex Cloud level: Events that occur with Yandex Cloud resources.
- OS level.
- Application level.
- Network level (Flow Logs).
Note
For more information about Kubernetes events, see Collecting, monitoring, and analyzing audit logs in Yandex Managed Service for Kubernetes.
5.1 Yandex Audit Trails is enabled at the organization level
The main tool for collecting Yandex Cloud level logs is Yandex Audit Trails. This service allows you to collect audit logs about events happening to Yandex Cloud resources and upload these logs to Yandex Object Storage buckets or Cloud Logging log groups for further analysis or export. See this guide on how to start collecting logs. You can also learn more about the event format or check out the event reference.
To collect metrics, analyze Yandex Cloud-level events, and set up notifications, we recommend using Yandex Monitoring. It helps you track, for example, a sharp increase in the load on Compute Cloud, the number of Application Load Balancer requests per second (RPS), or significant changes in event statistics in Identity and Access Management.
You can also use Monitoring to monitor the health of the Audit Trails service itself and track security events. You can export metrics to a SIEM system via the API, see the instructions.
Solution: Monitoring Audit Trails and security events using Monitoring
You can export audit logs to a Cloud Logging or Data Streams log group and to a customer's SIEM system to analyze information about events and incidents.
List of important Yandex Cloud-level events to search for in audit logs:
Solution: Searching for important security events in audit logs
You can enable Yandex Audit Trails at the folder, cloud, and organization level. We recommend enabling Yandex Audit Trails at the level of the entire organization. Thus you will be able to collect audit logs in a centralized manner, e.g., to a separate security cloud.
- In the management console, select the cloud or folder to check the functions in.
- In the list of services, select Yandex Audit Trails.
- Make sure the Filter parameter is set to Organization.
- In addition, check that the destination of logs is Yandex Object Storage bucket, Cloud Logging log group, and Data Streams, that they are up and running, and that the logs are available for further analysis.
5.2 Yandex Audit Trails events are exported to SIEM systems
Solutions for exporting Yandex Cloud audit logs are available for the following SIEM systems:
-
ArcSight: Collecting, monitoring, and analyzing audit logs in ArcSight SIEM
-
Splunk: Collecting, monitoring, and analyzing audit logs in Splunk SIEM
-
MaxPatrol SIEM: Collecting, monitoring, and analyzing audit logs in MaxPatrol SIEM
-
Wazuh: Collecting, monitoring, and analyzing audit logs in Wazuh
For more information about MaxPatrol, see this section.
You can set up export to any SIEM using GeeseFS or s3fs. These utilities allow mounting a Yandex Object Storage bucket as a VM local disk. Next, you need to install a SIEM connector on your VM and configure reading JSON files from the bucket. You can also use utilities compatible with AWS Kinesis datastreams if sending audit logs to Yandex Data Streams.
If you have no SIEM, you can also analyze audit logs manually using one of the following methods (in descending order of convenience):
-
Searching for Yandex Cloud events in Yandex Query.
-
Searching for Yandex Cloud events in Cloud Logging.
-
Searching for Yandex Cloud events in Object Storage.
Make sure that audit logs from Yandex Audit Trails are exported for analysis to a SIEM system or analyzed in the cloud using one of the available methods.
5.3 Responding to Yandex Audit Trails events is set up
You can respond to Yandex Audit Trails events using your SIEM tools or manually. You can also use automatic responses.
Using Yandex Cloud Functions, you can configure alerts about Audit Trails events, as well as automatic responses to malicious actions, including removing dangerous rules or revoking access rights.
5.4 Hardening of the Object Storage bucket that stores Yandex Audit Trails audit logs is done
If you write Yandex Audit Trails audit logs to a Yandex Object Storage bucket, make sure the bucket is set up using best security practices, such as:
- 4.1 In Yandex Object Storage, encryption of data at rest using KMS keys is enabled.
- 3.8 In Yandex Object Storage, logging of actions with buckets is enabled.
- 3.8 In Yandex Object Storage, the Object locks feature is enabled.
- 3.7 In Yandex Object Storage, Bucket Policies are used.
- 3.6 No public access to the Yandex Object Storage bucket is allowed.
You can use a solution for secure Yandex Object Storage bucket setup with Terraform.
Run a manual check.
5.5 Audit logs are collected at the OS level
When using IaaS cloud services and Kubernetes node groups, the customer is responsible for ensuring OS security and collecting OS-level events on their own. Free tools for collecting standard OS-generated events and exporting them to the customer's SIEM system include:
Additional event generation options can be implemented using Auditd for Linux or Sysmon for Windows.
You can collect Linux system metrics (CPU, RAM, and disk space usage) with Unified Agent in Monitoring.
You can also export OS events to Cloud Logging using a Fluent Bit plugin
To describe events to be searched for in audit logs, we recommend using Sigma
To get the exact time of OS- and application-level events, configure clock synchronization by following this guide.
Run a manual check.
5.6 Audit logs are collected at the application level
Customers may collect events that occur at the level of applications deployed on Compute Cloud resources on their own. For example, save application logs to files and transfer them to a SIEM system using the tools listed in the subsection above.
Run a manual check.
5.7 Logs are collected at the network level
Currently, VPC network traffic event logs (Flow Logs) can only be collected by customers. You can use Yandex Cloud Marketplace solutions (such as NGFW, IDS/IPS, or network products) or free software for collecting and transmitting events. You can also collect network-level logs using different agents, e.g., HIDS.
Run a manual check.
ClickHouse® is a registered trademark of ClickHouse, Inc