Installing KEDA with Yandex Monitoring support

KEDA is an application that performs autoscaling of Kubernetes resources based on load monitoring.

KEDA application:

• Integrates Kubernetes autoscaling with Yandex Monitoring.
• Allows you to scale applications based on metrics from Monitoring, such as Application Load Balancer metrics or managed databases.
• Supports time window configuration, as well as aggregation and processing of NaN values.

Getting startedGetting started

1. Create a service account:

   yc iam service-account create --name keda-sa
   

2. Assign the monitoring.viewer role to the service account you created previously.

   yc resource-manager folder add-access-binding \
     --id <folder_ID> \
     --service-account-name keda-sa \
     --role monitoring.viewer
   

   You can fetch the folder ID with the list of folders.

3. Create an authorized key for the service account you created earlier and save it to a file named authorized-key.json:

   yc iam key create \
     --service-account-name keda-sa \
     --output authorized-key.json
   

4. Make sure the security groups for the Managed Service for Kubernetes cluster and its node groups are configured correctly. If a rule is missing, add it.

   Warning

   The configuration of security groups determines performance and availability of the cluster and the services and applications running in it.

5. Install kubect and configure it to work with the new cluster.

Installation from Yandex Cloud MarketplaceInstallation from Yandex Cloud Marketplace

1. Navigate to the folder page and select Managed Service for Kubernetes.

2. Click the name of the Managed Service for Kubernetes cluster you need and select the Marketplace tab.

3. Under Application available for installation, select KEDA with Yandex Monitoring support and click Go to install.

4. Configure the application:

   • Namespace: Create a new namespace, keda-system. If you keep the default namespace, KEDA may work incorrectly.
   • Application name: Specify the application name.
   • Service account key: Copy the contents of the authorized-key.json file to this field.

5. Click Install.

6. Wait for the application status to change to Deployed.

7. Create the ScaledObject resource with these resource autoscaling settings:

   apiVersion: keda.sh/v1alpha1
   kind: ScaledObject
   metadata:
     name: <resource_name>
   spec:
     scaleTargetRef:
       name: <autoscaling_resource_name>
     pollingInterval: <monitoring_event_check_interval>
     cooldownPeriod: <waiting_period_after_event>
     minReplicaCount: <minimum_number_of_replicas_for_autoscaling>
     maxReplicaCount: <maximum_number_of_replicas_for_autoscaling>
     triggers:
     - type: external
       metadata:
         scalerAddress: keda-external-scaler-yc.keda-system.svc.cluster.local:8080
         query: <Yandex_Monitoring_metrics_request>
         folderId: "<folder_ID>"
         targetValue: "<target_metric_value>"
         downsampling.disabled: <enable_data_decimation_mode>
         downsampling.gridAggregation: "<data_aggregation_function>"
         downsampling.gridInterval: "<decimation_time_window>"
         downsampling.maxPoints: <maximum_number_of_points>
         downsampling.gapFilling: <data_filling>
   
         timeWindow: "<time_interval_window>"
         timeWindowOffset: "<time_window_indent>"
    
         logLevel: "<logging_level>"
         logMetrics: "<allow_Monitoring_request_logging>"
         logAggregation: "<enable_data_aggregation_logging>"
   

   Required metadata in the triggers field:

   • scalerAddress: Autoscaling service address. It is always keda-external-scaler-yc.keda-system.svc.cluster.local:8080.
   • query: Monitoring metrics request.
   • folderId: ID of the folder where the provider will run.
   • targetValue: Target metric value, exceeding which adds a pod for the replica.

   Decimation (downsampling) parameters. For autoscaling to work, you need to select at least one of these parameters:

   • downsampling.gridAggregation: Data aggregation function. The possible values are: MAX, MIN, SUM, AVG, LAST, COUNT. The default value is AVG.

   • downsampling.gapFilling: Settings for filling in missing data:

     • NULL: Returns null as the metric value and timestamp as the timestamp value.
     • NONE: Returns no values.
     • PREVIOUS: Returns the value from the previous data point.

   • downsampling.maxPoints: Maximum number of points to receive in response to a request. The value of this parameter must be greater than 10.

   • downsampling.gridInterval: Time window (grid) width in milliseconds. It is used for decimating: Data points within this time window are merged into a single value using the selected aggregation function. The value of this parameter must be greater than 0.

   • downsampling.disabled: Disable data decimation. The possible values are: true or false.

     Note

     Use only one of these parameters: downsampling.maxPoints, downsampling.gridInterval, or downsampling.disabled. For more information about decimation parameters, see the API documentation.

   Time window parameters:

   • timeWindow: Time period for calculating the aggregation function (in DdHhMmSs format, e.g., 5d10h30m20s).
   • timeWindowOffset: Delay in calculating the function in the time window.

   Logging parameters:

   • logLevel: Logging level. The possible values are: debug, info, warn, error, none. The default value is info.
   • logMetrics: Enable Monitoring request logging: true or false. The default value is false.
   • logAggregation: Enable data aggregation logging: true or false. The default value is false.

   For more information on the ScaledObject resource parameters, see the project's guide on Github.

Installation using a Helm chartInstallation using a Helm chart

1. Install Helm v3.8.0 or higher.

2. To install a Helm chart with KEDA, run this command:

   helm pull oci://cr.yandex/yc-marketplace/yandex-cloud/keda/chart/keda-external-scaler-yc \
     --version 1.1.3 \
     --untar && \
   helm install \
     --namespace keda-system \
     --create-namespace \
     --set-file secret.data=./authorized-key.json \
     --set keda.enabled=true \
     keda ./keda
   

   If you set namespace to the default namespace, KEDA may work incorrectly.

   Note

   If you are using a Helm version below 3.8.0, add the export HELM_EXPERIMENTAL_OCI=1 && \ string at the beginning of the command to enable Open Container Initiative (OCI) support in the Helm client.

An example of using KEDAAn example of using KEDA

1. Deploy nginx in the Managed Service for Kubernetes cluster.

2. Install ALB Ingress Controller and the Ingress resource for nginx.

3. Create the ScaledObject resource with these settings for autoscaling web application pods based on Application Load Balancer metrics:

   apiVersion: keda.sh/v1alpha1
   kind: ScaledObject
   metadata:
     name: demo-app-scaler
   spec:
     scaleTargetRef:
       name: <Deployment_resource_name>
     pollingInterval: 30
     cooldownPeriod: 60
     minReplicaCount: 1
     maxReplicaCount: 20
     triggers:
     - type: external
       metadata:
         scalerAddress: keda-external-scaler-yc.keda-system.svc.cluster.local:8080
         query: |
          alias(replace_nan(series_sum("load_balancer.requests_count_per_second"{
          service="application-load-balancer", 
          load_balancer="<load_balancer_ID>", 
          http_router="*", virtual_host="*", 
          route="<route_ID>", 
          backend_group="*", backend="*", 
          zone="*", code="total"}), 0), "Total requests count")
         folderId: "<folder_ID>"
         targetValue: "<target_metric_value>"
         downsampling.gridAggregation: "AVG"
         downsampling.gridInterval: "120000"
   
         timeWindow: "2m"
         timeWindowOffset: "30s"
    
         logLevel: "debug"
         logMetrics: "true"
         logAggregation: "true"
   

See alsoSee also

• KEDA documentation
• KEDA with Yandex Monitoring support on GitHub