Managed Service for Apache Airflow™ API, gRPC: ClusterService.Update
- gRPC request
- UpdateClusterRequest
- UpdateClusterConfigSpec
- AirflowConfig
- WebserverConfig
- Resources
- SchedulerConfig
- TriggererConfig
- WorkerConfig
- Dependencies
- LockboxConfig
- DagProcessorConfig
- DatacatalogConfig
- CodeSyncConfig
- S3Config
- GitSyncConfig
- UpdateNetworkConfigSpec
- LoggingConfig
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- operation.Operation
Updates the specified Apache Airflow cluster.
gRPC request
rpc Update (UpdateClusterRequest) returns (operation.Operation)
UpdateClusterRequest
{
"cluster_id": "string",
"update_mask": "google.protobuf.FieldMask",
"name": "string",
"description": "string",
"labels": "map<string, string>",
"config_spec": {
"airflow": {
"config": "map<string, string>"
},
"webserver": {
"count": "int64",
"resources": {
"resource_preset_id": "string"
}
},
"scheduler": {
"count": "int64",
"resources": {
"resource_preset_id": "string"
}
},
"triggerer": {
"count": "int64",
"resources": {
"resource_preset_id": "string"
}
},
"worker": {
"min_count": "int64",
"max_count": "int64",
"resources": {
"resource_preset_id": "string"
}
},
"dependencies": {
"pip_packages": [
"string"
],
"deb_packages": [
"string"
]
},
"lockbox": {
"enabled": "bool"
},
"airflow_version": "string",
"python_version": "string",
"dag_processor": {
"count": "int64",
"resources": {
"resource_preset_id": "string"
}
},
"datacatalog": {
"enabled": "bool"
}
},
"code_sync": {
// Includes only one of the fields `s3`, `git_sync`
"s3": {
"bucket": "string"
},
"git_sync": {
"repo": "string",
"branch": "string",
"sub_path": "string",
"ssh_key": "string"
}
// end of the list of possible fields
},
"network_spec": {
"security_group_ids": [
"string"
]
},
"deletion_protection": "bool",
"service_account_id": "string",
"logging": {
// Includes only one of the fields `folder_id`, `log_group_id`
"folder_id": "string",
"log_group_id": "string",
// end of the list of possible fields
"enabled": "bool",
"min_level": "Level"
},
"maintenance_window": {
// Includes only one of the fields `anytime`, `weekly_maintenance_window`
"anytime": "AnytimeMaintenanceWindow",
"weekly_maintenance_window": {
"day": "WeekDay",
"hour": "int64"
}
// end of the list of possible fields
}
}
|
Field |
Description |
|
cluster_id |
string Required field. ID of the Apache Airflow Cluster resource to update. The maximum string length in characters is 50. |
|
update_mask |
Field mask that specifies which fields of the Apache Airflow Cluster resource should be updated. |
|
name |
string New name of the cluster. Value must match the regular expression |
|
description |
string New description of the Apache Airflow cluster. The maximum string length in characters is 256. |
|
labels |
object (map<string, string>) Custom labels for the Apache Airflow cluster as The maximum string length in characters for each value is 63. The string length in characters for each key must be 1-63. Each key must match the regular expression |
|
config_spec |
Configuration of Apache Airflow components. |
|
code_sync |
Parameters of the location and access to the code that will be executed in the cluster. |
|
network_spec |
Network related configuration options. |
|
deletion_protection |
bool Deletion Protection inhibits deletion of the cluster |
|
service_account_id |
string Service account used to access Cloud resources. The maximum string length in characters is 50. |
|
logging |
Cloud Logging configuration. |
|
maintenance_window |
Window of maintenance operations. |
UpdateClusterConfigSpec
|
Field |
Description |
|
airflow |
Configuration of the Apache Airflow application itself. |
|
webserver |
Configuration of webserver instances. |
|
scheduler |
Configuration of scheduler instances. |
|
triggerer |
Configuration of triggerer instances. |
|
worker |
Configuration of worker instances. |
|
dependencies |
The list of additional packages installed in the cluster. |
|
lockbox |
Configuration of Lockbox Secret Backend. |
|
airflow_version |
string Apache Airflow version. Format: "Major.Minor" |
|
python_version |
string Python version. Format: "Major.Minor" |
|
dag_processor |
Configuration of dag-processor instances. |
|
datacatalog |
Configuration for datacatalog integration. |
AirflowConfig
|
Field |
Description |
|
config |
object (map<string, string>) Properties to be passed to Apache Airflow configuration file. |
WebserverConfig
|
Field |
Description |
|
count |
int64 The number of webserver instances in the cluster. Acceptable values are 1 to 512, inclusive. |
|
resources |
Resources allocated to webserver instances. |
Resources
|
Field |
Description |
|
resource_preset_id |
string ID of the preset for computational resources available to an instance (CPU, memory etc.). |
SchedulerConfig
|
Field |
Description |
|
count |
int64 The number of scheduler instances in the cluster. Acceptable values are 1 to 512, inclusive. |
|
resources |
Resources allocated to scheduler instances. |
TriggererConfig
|
Field |
Description |
|
count |
int64 The number of triggerer instances in the cluster. Acceptable values are 0 to 512, inclusive. |
|
resources |
Resources allocated to triggerer instances. |
WorkerConfig
|
Field |
Description |
|
min_count |
int64 The minimum number of worker instances in the cluster. Acceptable values are 0 to 512, inclusive. |
|
max_count |
int64 The maximum number of worker instances in the cluster. Acceptable values are 1 to 512, inclusive. |
|
resources |
Resources allocated to worker instances. |
Dependencies
|
Field |
Description |
|
pip_packages[] |
string Python packages that are installed in the cluster. |
|
deb_packages[] |
string System packages that are installed in the cluster. |
LockboxConfig
|
Field |
Description |
|
enabled |
bool The setting allows to enable Lockbox Secret Backend. |
DagProcessorConfig
|
Field |
Description |
|
count |
int64 The number of dag-processor instances in the cluster. Acceptable values are 1 to 512, inclusive. |
|
resources |
Resources allocated to dag-processor instances. |
DatacatalogConfig
|
Field |
Description |
|
enabled |
bool The setting allows to enable sending data to Datacatalog Backend. |
CodeSyncConfig
|
Field |
Description |
|
s3 |
Configuration for s3 folder for dags Includes only one of the fields |
|
git_sync |
Configuration for git repository for dags Includes only one of the fields |
S3Config
|
Field |
Description |
|
bucket |
string The name of the Object Storage bucket that stores DAG files used in the cluster. |
GitSyncConfig
|
Field |
Description |
|
repo |
string Required field. Git repository URL. |
|
branch |
string Required field. Git branch name to sync from. |
|
sub_path |
string Subdirectory path within the repository containing DAG files. |
|
ssh_key |
string SSH private key for repository authentication. |
UpdateNetworkConfigSpec
|
Field |
Description |
|
security_group_ids[] |
string User security groups. |
LoggingConfig
|
Field |
Description |
|
folder_id |
string Logs should be written to default log group for specified folder. Value must match the regular expression Includes only one of the fields Destination of log records. |
|
log_group_id |
string Logs should be written to log group resolved by ID. Value must match the regular expression Includes only one of the fields Destination of log records. |
|
enabled |
bool Logs generated by the Airflow components are delivered to Cloud Logging. |
|
min_level |
enum Level Minimum log entry level.
|
MaintenanceWindow
|
Field |
Description |
|
anytime |
Includes only one of the fields |
|
weekly_maintenance_window |
Includes only one of the fields |
AnytimeMaintenanceWindow
|
Field |
Description |
|
Empty |
|
WeeklyMaintenanceWindow
|
Field |
Description |
|
day |
enum WeekDay
|
|
hour |
int64 Hour of the day in UTC. Acceptable values are 1 to 24, inclusive. |
operation.Operation
{
"id": "string",
"description": "string",
"created_at": "google.protobuf.Timestamp",
"created_by": "string",
"modified_at": "google.protobuf.Timestamp",
"done": "bool",
"metadata": "google.protobuf.Any",
// Includes only one of the fields `error`, `response`
"error": "google.rpc.Status",
"response": "google.protobuf.Any"
// end of the list of possible fields
}
An Operation resource. For more information, see Operation.
|
Field |
Description |
|
id |
string ID of the operation. |
|
description |
string Description of the operation. 0-256 characters long. |
|
created_at |
Creation timestamp. |
|
created_by |
string ID of the user or service account who initiated the operation. |
|
modified_at |
The time when the Operation resource was last modified. |
|
done |
bool If the value is |
|
metadata |
Service-specific metadata associated with the operation. |
|
error |
The error result of the operation in case of failure or cancellation. Includes only one of the fields The operation result. |
|
response |
The normal response of the operation in case of success. Includes only one of the fields The operation result. |