Configuring logging via Application Load Balancer
You can write logs using either Yandex Cloud Logging or Yandex Audit Trails. For Cloud Logging, there are two log collection options available: via Smart Web Security and via the L7 Application Load Balancer the security profile is connected to.
This section provides info on log collection via Application Load Balancer. For information on logging via Smart Web Security, see this section.
Analyzing Yandex Smart Web Security logs enables you to:
-
Test security rules, WAF, and ARL in Logging only (dry run) mode.
In this mode, the system does not block user requests but logs rule matches.
-
View the number of blocked and allowed requests, evaluate and adjust rule performance.
-
View detailed request information and and identify false positives.
-
Investigate security incidents.
Features of log collection services:
-
Cloud Logging: Collects basic logs for traffic and rule matches from security profiles, WAF, and ARL.
Smart Web Security logs are sent through the L7 load balancer your security profile is connected to and are written to a log group.
-
Audit Trails: Records security events and collects more detailed audit logs on WAF and ARL rules.
In Audit Trails, events are delivered directly from Smart Web Security without using an L7 load balancer. There are two types of events in Audit Trails:
- Management events, which include actions related to Yandex Cloud resource configuration, such as creating or deleting a security profile.
- Data events, which include actions performed on resources within Yandex Cloud services, e.g., triggering a rule from a WAF profile.
You can log Audit Trails events to a bucket in Object Storage, log group in Cloud Logging, or data stream in Data Streams.
To get started with Smart Web Security logs:
Enable logging
- In the management console
, select the folder containing the Smart Web Security profile. - Select Application Load Balancer.
- In the list, select the load balancer linked to the security profile.
- Click
and select Edit. - Under Log settings:
- Click Save.
For other ways to enable logging, see Setting up L7 load balancer logging.
You can log Audit Trails events to a bucket in Object Storage, log group in Cloud Logging, or data stream in Data Streams. In this guide, we will set up logging of events to a log group.
-
In the management console
, select the folder containing the Smart Web Security profile. -
Select Audit Trails.
-
Click Create trail.
-
Enter a name for the trail, e.g.,
trail-sws. -
Under Destination, select Cloud Logging as the destination object.
-
Select or create a Cloud Logging log group to store Smart Web Security events.
-
Under Collecting data events, enable event collection and select Smart Web Security.
For the rest of the settings in this section, leave the default values. The system will log all data events from Smart Web Security in the current folder, ignoring management events.
-
Under Service account, create or select an account with the
logging.writerrole. -
Click Create.
For other ways to enable event logging, see Creating a trail to upload audit logs.
Viewing logs
-
In the management console
, select the folder containing the Smart Web Security profile. -
Select Application Load Balancer.
-
Go to the Logs section.
-
Select the number of messages per page and the time interval: 1 hour, 3 hours, 1 day, 1 week, 2 weeks.
-
In the Query row, specify you query using the filter expression language and click Run.
You can find examples of queries below.
-
To view log details, expand it.
For other ways to view logs, see Viewing L7 load balancer logs and Reading records.
Examples of preset log filters
Logs are delivered in JSON format. A single log entry maps to a single client request to the L7 load balancer.
Requests for log filtering are based on the relationship between Smart Web Security profiles and rules. You can view logs for active, running rules, or rules in Logging only (dry run) mode.
Filters for active rules
-
Show requests blocked by basic rules based on specific conditions, e.g., by IP list or region:
json_payload.smartwebsecurity.matched_rule.rule_type = RULE_CONDITION and json_payload.smartwebsecurity.matched_rule.verdict = DENY -
Show requests that have triggered the Smart Protection rules with a CAPTCHA challenge:
json_payload.smartwebsecurity.matched_rule.rule_type = SMART_PROTECTION and json_payload.smartwebsecurity.matched_rule.verdict = CAPTCHA -
Show requests blocked based on the WAF profile, i.e., by the security profile WAF rules:
json_payload.smartwebsecurity.matched_rule.rule_type = WAF and json_payload.smartwebsecurity.matched_rule.verdict = DENY -
Show requests blocked by the ARL profile rules:
json_payload.smartwebsecurity.advanced_rate_limiter.verdict = DENY -
Show requests which triggered a specific ARL rule,
arl-rule-1:json_payload.smartwebsecurity.advanced_rate_limiter.verdict = DENY and json_payload.smartwebsecurity.advanced_rate_limiter.applied_quota_name = "arl-rule-1"
You can similarly add other conditions to the filters and adjust them to fit your traffic flow.
Filters for rules in logging mode
-
Show requests that have triggered the Smart Protection rules with a CAPTCHA challenge:
json_payload.smartwebsecurity.dry_run_matched_rule.rule_type = SMART_PROTECTION and json_payload.smartwebsecurity.dry_run_matched_rule.verdict = CAPTCHA -
View requests that have triggered the ARL rules (limits on requests).
For the Logging only mode, you cannot use a request filtered by the
DENYverdict, because this mode does not block requests. The rule verdict will beALLOWeven after the limit is exceeded. To debug the rules, use thedry_run_exceeded_quota_namesparameter. This parameter shows which ARL rules were triggered by the request. If this parameter contains no rules, no limits were exceeded.Here is an example of a log fragment with the
dry_run_exceeded_quota_namesparameter:"smartwebsecurity": { "advanced_rate_limiter": { "applied_quota_name": "", "dry_run_exceeded_quota_names": [ "<rule_name_1>", "<rule_name_2>" ], "profile_id": "<profile_id>", "verdict": "ALLOW" },
In this fragment, limits were exceeded for the <rule_name_1> and <rule_name_2> rules in the <profile_id> profile.
-
In the management console
, select the folder containing the Smart Web Security profile. -
Select Cloud Logging.
-
Select the log group receiving your Audit Trails events.
-
Select the number of messages per page and the time interval: 1 hour, 3 hours, 1 day, 1 week, 2 weeks.
-
In the Query field, specify you query using the filter expression language and click Run.
Audit Trails logs are written in JSON format. To find a specific event, provide its name in the following format:
yandex.cloud.audit.smartwebsecurity.<event_name>For examples of how to create queries, see Examples of requests for searching events in audit logs.
-
To view log details, expand it.