Updating a Yandex Data Processing cluster

Written by

Updated at December 16, 2024

After creating a cluster, you can change its basic and advanced settings.

You can disable sending cluster logs to Yandex Cloud Logging. For more information, see Working with logs.

You can also move a Yandex Data Processing cluster to a different availability zone. The process depends on the cluster type:

Management console

CLI

Terraform

To change Yandex Data Processing cluster settings:

Go to the folder page and select Yandex Data Processing.
Select the cluster and click Edit in the top panel.
Change the name and description of the cluster in the Cluster name and Description fields.
Add or delete cluster labels in the Labels field:
Update cluster settings:
- Service account: Service account to which you need to grant access to the Yandex Data Processing cluster.
  
  Select an existing service account or create a new one.
  
  To change your service account in a Yandex Data Processing cluster, make sure your Yandex Cloud account has the iam.serviceAccounts.user role or higher.
  
  Warning
  
  If the cluster already uses a service account to access objects from Object Storage, then changing it to a different service account may make these objects unavailable and interrupt the cluster operation. Before changing the service account settings, make sure that the cluster doesn't use the objects in question.
- Properties: Cluster component properties.
  
  Add, edit, or delete the required properties:
  
  Tip
  
  If you want a property to be included into a configuration file relating to a specific component, specify a prefix for the key.
- Bucket ID format: Bucket name selection format, List or ID.
- Bucket name: Name of the bucket that will be used by the cluster.
  
  Depending on the format you selected, either pick a name from the list or specify it manually. You can request the name of the bucket with the list of buckets in the folder.
- Security groups: Security groups that will be used by the cluster.
  
  Select one or more security groups. If the required security group is not in the list, create it.
  
  Warning
  
  Incorrect security group settings may cause Yandex Data Processing cluster performance issues. For more information on setting up security groups, see Connecting to a cluster.
- UI Proxy: Option that manages access to component web interfaces via UI Proxy.
  
  Warning
  
  You may need to additionally set up security groups to use UI Proxy.
- Log group: Yandex Cloud Logging log group the cluster will send logs to.
  
  Select the default log group or another existing log group. If the required log group is not in the list, create it.
  
  To enable the cluster to send logs, assign the logging.writer role to its service account. For more information, see the Yandex Cloud Logging documentation.
In the advanced cluster settings, select the required Deletion protection value.

This option manages cluster protection against accidental deletion by a user.

Enabled protection will not prevent a manual connection to the cluster and deletion of data.
Click Save changes.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To change Yandex Data Processing cluster settings:

View the description of the update cluster CLI command:
```
yc dataproc cluster update --help
```
To edit the log group the cluster logs go to, provide the log group ID in the --log-group-id parameter:
```
yc dataproc cluster update <cluster_name_or_ID> \
   --log-group-id=<log_group_ID>
```
You can request the log group ID with a list of log groups in the folder.
To protect a cluster from accidental deletion by a user of your cloud, add the --deletion-protection parameter:
```
yc dataproc cluster update <cluster_name_or_ID> \
   --deletion-protection
```
Enabled protection will not prevent a manual connection to the cluster and deletion of data.
To update component properties, provide the required properties in the --property parameter:
```
yc dataproc cluster update <cluster_name_or_ID> \
   --property "<key_1_prefix>:<key_1>=<value>", "<key_2_prefix>:<key_2>=<value>", ...
```
Warning

Using the --property parameter will reset all component properties that were not explicitly provided in the parameter to their defaults. To save the previously updated properties, list them in your request along with the properties you want to update.

You can get the cluster ID and name with a list of clusters in the folder.

To change Yandex Data Processing cluster settings:

Open the current Terraform configuration file with an infrastructure plan.

For more information about creating this file, see Creating clusters.
To activate cluster deletion protection and access to Yandex Data Processing component web interfaces, update the values in the appropriate fields of the Yandex Data Processing cluster description:
```
resource "yandex_dataproc_cluster" "data_cluster" {
  ...
  deletion_protection = true
  ui_proxy            = true
  ...
}
```
Make sure the settings are correct.
1. Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
2. Run the command:
```
terraform validate
```
  If there are errors in the configuration files, Terraform will point to them.
Confirm updating the resources.
1. Run the command to view planned changes:
```
terraform plan
```
  If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
2. If you are happy with the planned changes, apply them:
  1. Run the command:
```
terraform apply
```
  2. Confirm the update of resources.
  3. Wait for the operation to complete.

For more information about the resources you can create with Terraform, see the provider documentation.

Updating a Yandex Data Processing cluster

Was the article helpful?