Managed Service for Apache Airflow™ API, REST: Cluster.Create
Creates an Apache Airflow cluster.
HTTP request
POST https://airflow.api.yandexcloud.kz/managed-airflow/v1/clusters
Body parameters
{
"folderId": "string",
"name": "string",
"description": "string",
"labels": "object",
"config": {
"versionId": "string",
"airflow": {
"config": "object"
},
"webserver": {
"count": "string",
"resources": {
"resourcePresetId": "string"
}
},
"scheduler": {
"count": "string",
"resources": {
"resourcePresetId": "string"
}
},
"dagProcessor": {
"count": "string",
"resources": {
"resourcePresetId": "string"
}
},
"triggerer": {
"count": "string",
"resources": {
"resourcePresetId": "string"
}
},
"worker": {
"minCount": "string",
"maxCount": "string",
"resources": {
"resourcePresetId": "string"
}
},
"dependencies": {
"pipPackages": [
"string"
],
"debPackages": [
"string"
]
},
"lockbox": {
"enabled": "boolean"
},
"airflowVersion": "string",
"pythonVersion": "string",
"datacatalog": {
"enabled": "boolean"
}
},
"network": {
"subnetIds": [
"string"
],
"securityGroupIds": [
"string"
]
},
"codeSync": {
// Includes only one of the fields `s3`, `gitSync`
"s3": {
"bucket": "string"
},
"gitSync": {
"repo": "string",
"branch": "string",
"subPath": "string",
"sshKey": "string"
}
// end of the list of possible fields
},
"deletionProtection": "boolean",
"serviceAccountId": "string",
"logging": {
// Includes only one of the fields `folderId`, `logGroupId`
"folderId": "string",
"logGroupId": "string",
// end of the list of possible fields
"enabled": "boolean",
"minLevel": "string"
},
"maintenanceWindow": {
// Includes only one of the fields `anytime`, `weeklyMaintenanceWindow`
"anytime": "object",
"weeklyMaintenanceWindow": {
"day": "string",
"hour": "string"
}
// end of the list of possible fields
},
"adminPassword": "string"
}
|
Field |
Description |
|
folderId |
string Required field. ID of the folder to create Apache Airflow cluster in. The maximum string length in characters is 50. |
|
name |
string Required field. Name of the Apache Airflow cluster. The name must be unique within the folder. The maximum string length in characters is 63. Value must match the regular expression |
|
description |
string Description of the Apache Airflow cluster. The maximum string length in characters is 256. |
|
labels |
object (map<string, string>) Custom labels for the Apache Airflow cluster as The maximum string length in characters for each value is 63. The maximum string length in characters for each key is 63. Each key must match the regular expression |
|
config |
Required field. Configuration of Apache Airflow components. |
|
network |
Network related configuration options. |
|
codeSync |
Parameters of the location and access to the code that will be executed in the cluster. |
|
deletionProtection |
boolean Deletion Protection inhibits deletion of the cluster. |
|
serviceAccountId |
string Service account used to access Cloud resources. The maximum string length in characters is 50. |
|
logging |
Cloud Logging configuration. |
|
maintenanceWindow |
Window of maintenance operations. |
|
adminPassword |
string Required field. Password of user The string length in characters must be 8-128. |
ClusterConfig
|
Field |
Description |
|
versionId |
string Version of Apache Airflow that runs on the cluster. |
|
airflow |
Configuration of the Apache Airflow application itself. |
|
webserver |
Required field. Configuration of webserver instances. |
|
scheduler |
Required field. Configuration of scheduler instances. |
|
dagProcessor |
Configuration of dag-processor instances. |
|
triggerer |
Configuration of triggerer instances. |
|
worker |
Required field. Configuration of worker instances. |
|
dependencies |
The list of additional packages installed in the cluster. |
|
lockbox |
Configuration of Lockbox Secret Backend. |
|
airflowVersion |
string Apache Airflow version. Format: "Major.Minor" |
|
pythonVersion |
string Python version. Format: "Major.Minor" |
|
datacatalog |
Configuration for datacatalog integration. |
AirflowConfig
|
Field |
Description |
|
config |
object (map<string, string>) Properties to be passed to Apache Airflow configuration file. |
WebserverConfig
|
Field |
Description |
|
count |
string (int64) The number of webserver instances in the cluster. Acceptable values are 1 to 512, inclusive. |
|
resources |
Resources allocated to webserver instances. |
Resources
|
Field |
Description |
|
resourcePresetId |
string ID of the preset for computational resources available to an instance (CPU, memory etc.). |
SchedulerConfig
|
Field |
Description |
|
count |
string (int64) The number of scheduler instances in the cluster. Acceptable values are 1 to 512, inclusive. |
|
resources |
Resources allocated to scheduler instances. |
DagProcessorConfig
|
Field |
Description |
|
count |
string (int64) The number of dag-processor instances in the cluster. Acceptable values are 1 to 512, inclusive. |
|
resources |
Resources allocated to dag-processor instances. |
TriggererConfig
|
Field |
Description |
|
count |
string (int64) The number of triggerer instances in the cluster. Acceptable values are 0 to 512, inclusive. |
|
resources |
Resources allocated to triggerer instances. |
WorkerConfig
|
Field |
Description |
|
minCount |
string (int64) The minimum number of worker instances in the cluster. Acceptable values are 0 to 512, inclusive. |
|
maxCount |
string (int64) The maximum number of worker instances in the cluster. Acceptable values are 1 to 512, inclusive. |
|
resources |
Resources allocated to worker instances. |
Dependencies
|
Field |
Description |
|
pipPackages[] |
string Python packages that are installed in the cluster. |
|
debPackages[] |
string System packages that are installed in the cluster. |
LockboxConfig
|
Field |
Description |
|
enabled |
boolean The setting allows to enable Lockbox Secret Backend. |
DatacatalogConfig
|
Field |
Description |
|
enabled |
boolean The setting allows to enable sending data to Datacatalog Backend. |
NetworkConfig
|
Field |
Description |
|
subnetIds[] |
string IDs of VPC network subnets where instances of the cluster are attached. |
|
securityGroupIds[] |
string User security groups. |
CodeSyncConfig
|
Field |
Description |
|
s3 |
Configuration for s3 folder for dags Includes only one of the fields |
|
gitSync |
Configuration for git repository for dags Includes only one of the fields |
S3Config
|
Field |
Description |
|
bucket |
string The name of the Object Storage bucket that stores DAG files used in the cluster. |
GitSyncConfig
|
Field |
Description |
|
repo |
string Required field. Git repository URL. |
|
branch |
string Required field. Git branch name to sync from. |
|
subPath |
string Subdirectory path within the repository containing DAG files. |
|
sshKey |
string SSH private key for repository authentication. |
LoggingConfig
|
Field |
Description |
|
folderId |
string Logs should be written to default log group for specified folder. Value must match the regular expression Includes only one of the fields Destination of log records. |
|
logGroupId |
string Logs should be written to log group resolved by ID. Value must match the regular expression Includes only one of the fields Destination of log records. |
|
enabled |
boolean Logs generated by the Airflow components are delivered to Cloud Logging. |
|
minLevel |
enum (Level) Minimum log entry level.
|
MaintenanceWindow
|
Field |
Description |
|
anytime |
object Includes only one of the fields |
|
weeklyMaintenanceWindow |
Includes only one of the fields |
WeeklyMaintenanceWindow
|
Field |
Description |
|
day |
enum (WeekDay)
|
|
hour |
string (int64) Hour of the day in UTC. Acceptable values are 1 to 24, inclusive. |
Response
HTTP Code: 200 - OK
{
"id": "string",
"description": "string",
"createdAt": "string",
"createdBy": "string",
"modifiedAt": "string",
"done": "boolean",
"metadata": "object",
// Includes only one of the fields `error`, `response`
"error": {
"code": "integer",
"message": "string",
"details": [
"object"
]
},
"response": "object"
// end of the list of possible fields
}
An Operation resource. For more information, see Operation.
|
Field |
Description |
|
id |
string ID of the operation. |
|
description |
string Description of the operation. 0-256 characters long. |
|
createdAt |
string (date-time) Creation timestamp. String in RFC3339 To work with values in this field, use the APIs described in the |
|
createdBy |
string ID of the user or service account who initiated the operation. |
|
modifiedAt |
string (date-time) The time when the Operation resource was last modified. String in RFC3339 To work with values in this field, use the APIs described in the |
|
done |
boolean If the value is |
|
metadata |
object Service-specific metadata associated with the operation. |
|
error |
The error result of the operation in case of failure or cancellation. Includes only one of the fields The operation result. |
|
response |
object The normal response of the operation in case of success. Includes only one of the fields The operation result. |
Status
The error result of the operation in case of failure or cancellation.
|
Field |
Description |
|
code |
integer (int32) Error code. An enum value of google.rpc.Code |
|
message |
string An error message. |
|
details[] |
object A list of messages that carry the error details. |