Data Proc API, REST: Cluster.Update
Updates the configuration of the specified cluster.
HTTP request
PATCH https://dataproc.api.cloud.yandex.net/dataproc/v1/clusters/{clusterId}
Path parameters
Field |
Description |
clusterId |
string Required field. ID of the cluster to update. To get the cluster ID, make a ClusterService.List request. |
Body parameters
{
"updateMask": "string",
"description": "string",
"labels": "string",
"configSpec": {
"subclustersSpec": [
{
"id": "string",
"name": "string",
"resources": {
"resourcePresetId": "string",
"diskTypeId": "string",
"diskSize": "string"
},
"hostsCount": "string",
"autoscalingConfig": {
"maxHostsCount": "string",
"preemptible": "boolean",
"measurementDuration": "string",
"warmupDuration": "string",
"stabilizationDuration": "string",
"cpuUtilizationTarget": "string",
"decommissionTimeout": "string"
}
}
],
"hadoop": {
"services": [
"string"
],
"properties": "string",
"sshPublicKeys": [
"string"
],
"initializationActions": [
{
"uri": "string",
"args": [
"string"
],
"timeout": "string"
}
]
}
},
"name": "string",
"serviceAccountId": "string",
"bucket": "string",
"decommissionTimeout": "string",
"uiProxy": "boolean",
"securityGroupIds": [
"string"
],
"deletionProtection": "boolean",
"logGroupId": "string"
}
Field |
Description |
updateMask |
string (field-mask) A comma-separated names off ALL fields to be updated. If |
description |
string New description for the cluster. |
labels |
string A new set of cluster labels as |
configSpec |
Configuration and resources for hosts that should be created with the Data Proc cluster. |
name |
string New name for the Data Proc cluster. The name must be unique within the folder. |
serviceAccountId |
string ID of the new service account to be used by the Data Proc manager agent. |
bucket |
string Name of the new Object Storage bucket to use for Data Proc jobs. |
decommissionTimeout |
string (int64) Timeout to gracefully decommission nodes. In seconds. Default value: 0 |
uiProxy |
boolean Enable UI Proxy feature. |
securityGroupIds[] |
string User security groups. |
deletionProtection |
boolean Deletion Protection inhibits deletion of the cluster |
logGroupId |
string ID of the cloud logging log group to write logs. If not set, logs will not be sent to logging service |
UpdateClusterConfigSpec
Field |
Description |
subclustersSpec[] |
New configuration for subclusters in a cluster. |
hadoop |
Hadoop specific options |
UpdateSubclusterConfigSpec
Field |
Description |
id |
string ID of the subcluster to update. To get the subcluster ID make a SubclusterService.List request. |
name |
string Name of the subcluster. |
resources |
Resource configuration for each host in the subcluster. |
hostsCount |
string (int64) Number of hosts in the subcluster. |
autoscalingConfig |
Configuration for instance group based subclusters |
Resources
Field |
Description |
resourcePresetId |
string ID of the resource preset for computational resources available to a host (CPU, memory etc.). |
diskTypeId |
string Type of the storage environment for the host.
|
diskSize |
string (int64) Volume of the storage available to a host, in bytes. |
AutoscalingConfig
Field |
Description |
maxHostsCount |
string (int64) Upper limit for total instance subcluster count. |
preemptible |
boolean Preemptible instances are stopped at least once every 24 hours, and can be stopped at any time |
measurementDuration |
string (duration) Required field. Time in seconds allotted for averaging metrics. |
warmupDuration |
string (duration) The warmup time of the instance in seconds. During this time, |
stabilizationDuration |
string (duration) Minimum amount of time in seconds allotted for monitoring before |
cpuUtilizationTarget |
string Defines an autoscaling rule based on the average CPU utilization of the instance group. |
decommissionTimeout |
string (int64) Timeout to gracefully decommission nodes during downscaling. In seconds. Default value: 120 |
HadoopConfig
Hadoop configuration that describes services installed in a cluster,
their properties and settings.
Field |
Description |
services[] |
enum (Service) Set of services used in the cluster (if empty, the default set is used).
|
properties |
string Properties set for all hosts in For example, use the key 'hdfs:dfs.replication' to set the |
sshPublicKeys[] |
string List of public SSH keys to access to cluster hosts. |
initializationActions[] |
Set of init-actions |
InitializationAction
Field |
Description |
uri |
string URI of the executable file |
args[] |
string Arguments to the initialization action |
timeout |
string (int64) Execution timeout |
Response
HTTP Code: 200 - OK
{
"id": "string",
"description": "string",
"createdAt": "string",
"createdBy": "string",
"modifiedAt": "string",
"done": "boolean",
"metadata": {
"clusterId": "string"
},
// Includes only one of the fields `error`, `response`
"error": {
"code": "integer",
"message": "string",
"details": [
"object"
]
},
"response": {
"id": "string",
"folderId": "string",
"createdAt": "string",
"name": "string",
"description": "string",
"labels": "string",
"monitoring": [
{
"name": "string",
"description": "string",
"link": "string"
}
],
"config": {
"versionId": "string",
"hadoop": {
"services": [
"string"
],
"properties": "string",
"sshPublicKeys": [
"string"
],
"initializationActions": [
{
"uri": "string",
"args": [
"string"
],
"timeout": "string"
}
]
}
},
"health": "string",
"status": "string",
"zoneId": "string",
"serviceAccountId": "string",
"bucket": "string",
"uiProxy": "boolean",
"securityGroupIds": [
"string"
],
"hostGroupIds": [
"string"
],
"deletionProtection": "boolean",
"logGroupId": "string"
}
// end of the list of possible fields
}
An Operation resource. For more information, see Operation.
Field |
Description |
id |
string ID of the operation. |
description |
string Description of the operation. 0-256 characters long. |
createdAt |
string (date-time) Creation timestamp. String in RFC3339 To work with values in this field, use the APIs described in the |
createdBy |
string ID of the user or service account who initiated the operation. |
modifiedAt |
string (date-time) The time when the Operation resource was last modified. String in RFC3339 To work with values in this field, use the APIs described in the |
done |
boolean If the value is |
metadata |
Service-specific metadata associated with the operation. |
error |
The error result of the operation in case of failure or cancellation. Includes only one of the fields The operation result. |
response |
The normal response of the operation in case of success. Includes only one of the fields The operation result. |
UpdateClusterMetadata
Field |
Description |
clusterId |
string ID of the cluster that is being updated. |
Status
The error result of the operation in case of failure or cancellation.
Field |
Description |
code |
integer (int32) Error code. An enum value of google.rpc.Code |
message |
string An error message. |
details[] |
object A list of messages that carry the error details. |
Cluster
A Data Proc cluster. For details about the concept, see documentation.
Field |
Description |
id |
string ID of the cluster. Generated at creation time. |
folderId |
string ID of the folder that the cluster belongs to. |
createdAt |
string (date-time) Creation timestamp. String in RFC3339 To work with values in this field, use the APIs described in the |
name |
string Name of the cluster. The name is unique within the folder. |
description |
string Description of the cluster. |
labels |
string Cluster labels as |
monitoring[] |
Monitoring systems relevant to the cluster. |
config |
Configuration of the cluster. |
health |
enum (Health) Aggregated cluster health.
|
status |
enum (Status) Cluster status.
|
zoneId |
string ID of the availability zone where the cluster resides. |
serviceAccountId |
string ID of service account for the Data Proc manager agent. |
bucket |
string Object Storage bucket to be used for Data Proc jobs that are run in the cluster. |
uiProxy |
boolean Whether UI Proxy feature is enabled. |
securityGroupIds[] |
string User security groups. |
hostGroupIds[] |
string Host groups hosting VMs of the cluster. |
deletionProtection |
boolean Deletion Protection inhibits deletion of the cluster |
logGroupId |
string ID of the cloud logging log group to write logs. If not set, default log group for the folder will be used. |
Monitoring
Metadata of a monitoring system for a Data Proc cluster.
Field |
Description |
name |
string Name of the monitoring system. |
description |
string Description of the monitoring system. |
link |
string Link to the monitoring system. |
ClusterConfig
Field |
Description |
versionId |
string Image version for cluster provisioning. |
hadoop |
Data Proc specific configuration options. |
HadoopConfig
Hadoop configuration that describes services installed in a cluster,
their properties and settings.
Field |
Description |
services[] |
enum (Service) Set of services used in the cluster (if empty, the default set is used).
|
properties |
string Properties set for all hosts in For example, use the key 'hdfs:dfs.replication' to set the |
sshPublicKeys[] |
string List of public SSH keys to access to cluster hosts. |
initializationActions[] |
Set of init-actions |
InitializationAction
Field |
Description |
uri |
string URI of the executable file |
args[] |
string Arguments to the initialization action |
timeout |
string (int64) Execution timeout |