Managed Service for Apache Airflow™ API, REST: Cluster.List

Written by

Updated at December 17, 2024

HTTP request
Query parameters
Response
Cluster
Monitoring
ClusterConfig
AirflowConfig
WebserverConfig
Resources
SchedulerConfig
TriggererConfig
WorkerConfig
Dependencies
LockboxConfig
NetworkConfig
CodeSyncConfig
S3Config
LoggingConfig

Retrieves a list of Apache Airflow Cluster resources.

HTTP request

GET https://airflow.api.cloud.yandex.net/managed-airflow/v1/clusters

Query parameters

Field	Description
folderId	string Required field. ID of the folder to list Apache Airflow clusters in.
pageSize	string (int64) The maximum number of results per page to return. If the number of available results is larger than `pageSize`, the service returns a ListClustersResponse.nextPageToken that can be used to get the next page of results in subsequent list requests.
pageToken	string Page token. To get the next page of results, set `pageToken` to the ListClustersResponse.nextPageToken returned by the previous list request.
filter	string A filter expression that filters resources listed in the response. The expression must specify: The field name. Currently you can only use filtering with the Cluster.name field. An `=` operator. The value in double quotes (`"`). Must be 1-63 characters long and match the regular expression `[a-zA-Z0-9_-]+`.

Response

HTTP Code: 200 - OK

{
  "clusters": [
    {
      "id": "string",
      "folderId": "string",
      "createdAt": "string",
      "name": "string",
      "description": "string",
      "labels": "object",
      "monitoring": [
        {
          "name": "string",
          "description": "string",
          "link": "string"
        }
      ],
      "config": {
        "versionId": "string",
        "airflow": {
          "config": "object"
        },
        "webserver": {
          "count": "string",
          "resources": {
            "resourcePresetId": "string"
          }
        },
        "scheduler": {
          "count": "string",
          "resources": {
            "resourcePresetId": "string"
          }
        },
        "triggerer": {
          "count": "string",
          "resources": {
            "resourcePresetId": "string"
          }
        },
        "worker": {
          "minCount": "string",
          "maxCount": "string",
          "resources": {
            "resourcePresetId": "string"
          }
        },
        "dependencies": {
          "pipPackages": [
            "string"
          ],
          "debPackages": [
            "string"
          ]
        },
        "lockbox": {
          "enabled": "boolean"
        },
        "airflowVersion": "string",
        "pythonVersion": "string"
      },
      "health": "string",
      "status": "string",
      "network": {
        "subnetIds": [
          "string"
        ],
        "securityGroupIds": [
          "string"
        ]
      },
      "codeSync": {
        // Includes only one of the fields `s3`
        "s3": {
          "bucket": "string"
        }
        // end of the list of possible fields
      },
      "deletionProtection": "boolean",
      "webserverUrl": "string",
      "serviceAccountId": "string",
      "logging": {
        "enabled": "boolean",
        // Includes only one of the fields `folderId`, `logGroupId`
        "folderId": "string",
        "logGroupId": "string",
        // end of the list of possible fields
        "minLevel": "string"
      }
    }
  ],
  "nextPageToken": "string"
}

Field

Description

clusters[]

Cluster

List of Apache Airflow Cluster resources.

nextPageToken

string

This token allows you to get the next page of results for list requests. If the number of results
is larger than ListClustersRequest.pageSize, use the nextPageToken as the value
for the ListClustersRequest.pageToken parameter in the next list request. Each subsequent
list request will have its own nextPageToken to continue paging through the results.

Cluster

Apache Airflow cluster.

Field	Description
id	string Unique ID of the Apache Airflow cluster. This ID is assigned by Cloud during cluster creation.
folderId	string ID of the folder that the Apache Airflow cluster belongs to.
createdAt	string (date-time) The time when the Apache Airflow cluster was created. String in RFC3339 text format. The range of possible values is from `0001-01-01T00:00:00Z` to `9999-12-31T23:59:59.999999999Z`, i.e. from 0 to 9 digits for fractions of a second. To work with values in this field, use the APIs described in the Protocol Buffers reference. In some languages, built-in datetime utilities do not support nanosecond precision (9 digits).
name	string Name of the Apache Airflow cluster. The name is unique within the folder. 1-64 characters long.
description	string Description of the Apache Airflow cluster. 0-256 characters long.
labels	object (map<string, string>) Resource labels as `key:value` pairs. Maximum of 64 per resource.
monitoring[]	Monitoring Monitoring systems relevant to the Apache Airflow cluster.
config	ClusterConfig Configuration of Apache Airflow components.
health	enum (Health) Aggregated cluster health. `HEALTH_UNKNOWN`: Cluster is in unknown state (we have no data). `ALIVE`: Cluster is alive and well. `DEAD`: Cluster is inoperable (it cannot perform any of its essential functions). `DEGRADED`: Cluster is partially alive (it can perform some of its essential functions).
status	enum (Status) Cluster status. `STATUS_UNKNOWN`: Cluster state is unknown. `CREATING`: Cluster is being created. `RUNNING`: Cluster is running normally. `ERROR`: Cluster encountered a problem and cannot operate. `STOPPING`: Cluster is stopping. `STOPPED`: Cluster is stopped. `STARTING`: Cluster is starting. `UPDATING`: Cluster is being updated.
network	NetworkConfig Network related configuration options.
codeSync	CodeSyncConfig Parameters of the location and access to the code that will be executed in the cluster.
deletionProtection	boolean Deletion Protection inhibits deletion of the cluster.
webserverUrl	string Address of Apache Airflow web UI.
serviceAccountId	string Service account used to access Cloud resources. For more information, see documentation.
logging	LoggingConfig Cloud Logging configuration.

Monitoring

Monitoring system.

Field	Description
name	string Name of the monitoring system.
description	string Description of the monitoring system.
link	string Link to the monitoring system.

ClusterConfig

Field	Description
versionId	string Version of Apache Airflow that runs on the cluster. Use `airlow_version` instead.
airflow	AirflowConfig Configuration of the Apache Airflow application itself.
webserver	WebserverConfig Required field. Configuration of webserver instances.
scheduler	SchedulerConfig Required field. Configuration of scheduler instances.
triggerer	TriggererConfig Configuration of triggerer instances.
worker	WorkerConfig Required field. Configuration of worker instances.
dependencies	Dependencies The list of additional packages installed in the cluster.
lockbox	LockboxConfig Configuration of Lockbox Secret Backend.
airflowVersion	string Apache Airflow version. Format: "Major.Minor"
pythonVersion	string Python version. Format: "Major.Minor"

AirflowConfig

Field

Description

config

object (map<string, string>)

Properties to be passed to Apache Airflow configuration file.

WebserverConfig

Field

Description

count

string (int64)

The number of webserver instances in the cluster.

resources

Resources

Resources allocated to webserver instances.

Resources

Field

Description

resourcePresetId

string

ID of the preset for computational resources available to an instance (CPU, memory etc.).

SchedulerConfig

Field

Description

count

string (int64)

The number of scheduler instances in the cluster.

resources

Resources

Resources allocated to scheduler instances.

TriggererConfig

Field

Description

count

string (int64)

The number of triggerer instances in the cluster.

resources

Resources

Resources allocated to triggerer instances.

WorkerConfig

Field	Description
minCount	string (int64) The minimum number of worker instances in the cluster.
maxCount	string (int64) The maximum number of worker instances in the cluster.
resources	Resources Resources allocated to worker instances.

Dependencies

Field

Description

pipPackages[]

string

Python packages that are installed in the cluster.

debPackages[]

string

System packages that are installed in the cluster.

LockboxConfig

Field

Description

enabled

boolean

The setting allows to enable Lockbox Secret Backend.

NetworkConfig

Field

Description

subnetIds[]

string

IDs of VPC network subnets where instances of the cluster are attached.

securityGroupIds[]

string

User security groups.

CodeSyncConfig

Field

Description

S3Config

Includes only one of the fields s3.

S3Config

Field

Description

bucket

string

The name of the Object Storage bucket that stores DAG files used in the cluster.

LoggingConfig

Field	Description
enabled	boolean Logs generated by the Airflow components are delivered to Cloud Logging.
folderId	string Logs should be written to default log group for specified folder. Includes only one of the fields `folderId`, `logGroupId`. Destination of log records.
logGroupId	string Logs should be written to log group resolved by ID. Includes only one of the fields `folderId`, `logGroupId`. Destination of log records.
minLevel	enum (Level) Minimum log entry level. See LogLevel.Level for details. `LEVEL_UNSPECIFIED`: Default log level. Equivalent to not specifying log level at all. `TRACE`: Trace log level. Possible use case: verbose logging of some business logic. `DEBUG`: Debug log level. Possible use case: debugging special cases in application logic. `INFO`: Info log level. Mostly used for information messages. `WARN`: Warn log level. May be used to alert about significant events. `ERROR`: Error log level. May be used to alert about errors in infrastructure, logic, etc. `FATAL`: Fatal log level. May be used to alert about unrecoverable failures and events.

Managed Service for Apache Airflow™ API, REST: Cluster.List

HTTP requestHTTP request

Query parametersQuery parameters

ResponseResponse

ClusterCluster

MonitoringMonitoring

ClusterConfigClusterConfig

AirflowConfigAirflowConfig

WebserverConfigWebserverConfig

ResourcesResources

SchedulerConfigSchedulerConfig

TriggererConfigTriggererConfig

WorkerConfigWorkerConfig

DependenciesDependencies

LockboxConfigLockboxConfig

NetworkConfigNetworkConfig

CodeSyncConfigCodeSyncConfig

S3ConfigS3Config

LoggingConfigLoggingConfig

Was the article helpful?

HTTP request

Query parameters

Response

Cluster

Monitoring

ClusterConfig

AirflowConfig

WebserverConfig

Resources

SchedulerConfig

TriggererConfig

WorkerConfig

Dependencies

LockboxConfig

NetworkConfig

CodeSyncConfig

S3Config

LoggingConfig