KEDA with Yandex Monitoring

KEDA is an application that autoscales Kubernetes resources in a Managed Service for Kubernetes cluster based on load monitoring.

KEDA application:

• Integrates Kubernetes resource autoscaling with Yandex Cloud Monitoring.
• Allows you to scale your applications based on Yandex Monitoring metrics, e.g., Application Load Balancer or managed database metrics.
• Supports time window configuration, as well as aggregation and processing of NaN values.

1. Create a service account for KEDA.

2. Assign the monitoring.viewer role to the service account.

3. Create an authorized key for the service account and save it to a file named sa-key.json:

   yc iam key create \
      --service-account-name <service_account_name> \
      --output sa-key.json
   

4. Configure the application:

   • Namespace: Create a new namespace named keda-system. If you keep the default namespace, KEDA may work incorrectly.
   • Application name: Specify the application name.
   • Service account key: Copy the contents of the sa-key.json file to this field.

5. Click Install.

6. Wait for the application to change its status to Deployed.

7. Create the ScaledObject resource with these resource autoscaling settings:

   apiVersion: keda.sh/v1alpha1
   kind: ScaledObject
   metadata:
     name: <resource_name>
   spec:
     scaleTargetRef:
       name: <autoscaling_resource_name>
     pollingInterval: <monitoring_event_check_interval>
     cooldownPeriod: <waiting_period_after_event>
     minReplicaCount: <minimum_number_of_replicas_for_autoscaling>
     maxReplicaCount: <maximum_number_of_replicas_for_autoscaling>
     triggers:
     - type: external
       metadata:
         scalerAddress: keda-external-scaler-yc.keda-system.svc.cluster.local:8080
         query: <Yandex_Monitoring_metric_request>
         folderId: "<folder_ID>"
         targetValue: "<target_metric_value>"
         downsampling.disabled: <enable_data_decimation_mode>
         downsampling.gridAggregation: "<data_aggregation_function>"
         downsampling.gridInterval: "<downsampling_time_window>"
         downsampling.maxPoints: <maximum_number_of_points>
         downsampling.gapFilling: <data_filling>
   
         timeWindow: "<time_interval_window>"
         timeWindowOffset: "<time_window_indent>"
    
         logLevel: "<logging_level>"
         logMetrics: "<allow_Monitoring_request_logging>"
         logAggregation: "<allow_data_aggregation_logging>"
   

   Required metadata in the triggers field:

   • scalerAddress: Autoscaling service address. It is always keda-external-scaler-yc.keda-system.svc.cluster.local:8080.
   • query: Monitoring metric request.
   • folderId: ID of the folder where the provider will run.
   • targetValue: Target metric value, exceeding which adds a pod for the replica.

   Downsampling parameters. For autoscaling to work, you need to select at least one of these parameters:

   • downsampling.gridAggregation: Data aggregation function. The possible values are MAX, MIN, SUM, AVG, LAST, or COUNT. The default value is AVG.

   • downsampling.gapFilling: Settings for filling in missing data:

     • NULL: Returns null as the metric value, and timestamp as the timestamp value.
     • NONE: Returns no values.
     • PREVIOUS: Returns the value from the previous data point.

   • downsampling.maxPoints: Maximum number of points to receive in a request response. The value of this parameter must be greater than 10.

   • downsampling.gridInterval: Time window (grid) width in milliseconds. It is used for downsampling: points inside the window are merged into a single one using the aggregation function. The value of this parameter must be greater than 0.

   • downsampling.disabled: Disable data downsampling; The possible values are true or false.

     Use only one of these parameters: downsampling.maxPoints, downsampling.gridInterval, or downsampling.disabled. For more information about downsampling parameters, see this API guide.

   Time window parameters:

   • timeWindow: Time period for calculating the aggregation function (in DdHhMmSs format, e.g., 5d10h30m20s).
   • timeWindowOffset: Delay in calculating the function in the time window.

   Logging parameters:

   • logLevel: Logging level, The possible values are debug, info, warn, error, and none. The default value is info.
   • logMetrics: Allow logging of Monitoring requests, true or false. The default value is false.
   • logAggregation: Allow data aggregation logging, true or false. The default value is false.

   For more information on the ScaledObject resource parameters, see the project’s guide on Github.

1. Deploy nginx in a Managed Service for Kubernetes cluster.

2. Install ALB Ingress Controller and the Ingress resource for nginx.

3. Create the ScaledObject resource with these settings for autoscaling web application pods based on Application Load Balancer metrics:

   apiVersion: keda.sh/v1alpha1
   kind: ScaledObject
   metadata:
     name: demo-app-scaler
   spec:
     scaleTargetRef:
       name: <Deployment_resource_name>
     pollingInterval: 30
     cooldownPeriod: 60
     minReplicaCount: 1
     maxReplicaCount: 20
     triggers:
     - type: external
       metadata:
         scalerAddress: keda-external-scaler-yc.keda-system.svc.cluster.local:8080
         query: |
          alias(replace_nan(series_sum("load_balancer.requests_count_per_second"{
          service="application-load-balancer", 
          load_balancer="<load_balancer_ID>", 
          http_router="*", virtual_host="*", 
          route="<route_ID>", 
          backend_group="*", backend="*", 
          zone="*", code="total"}), 0), "Total requests count")
         folderId: "<folder_ID>"
         targetValue: "<target_metric_value>"
         downsampling.gridAggregation: "AVG"
         downsampling.gridInterval: "120000"
   
         timeWindow: "2m"
         timeWindowOffset: "30s"
    
         logLevel: "debug"
         logMetrics: "true"
         logAggregation: "true"
   

Yandex Cloud technical support is available 24/7. The types of requests you can submit and the relevant response times depend on your pricing plan. You can switch to the paid support plan in the management console. You can learn more about the technical support terms here.