Updating Apache Kafka® cluster settings
After creating a Managed Service for Apache Kafka® cluster, you can:
- Change the cluster name and description
- Change the class and number of broker hosts
- Change the ZooKeeper host class
- Change the KRaft host class
- Change security group and public access settings
- Change additional cluster settings
- Change Apache Kafka® settings
- Move a cluster to another folder
Learn more about other cluster updates:
- Apache Kafka® version upgrade.
- Managing disk space in a Managed Service for Apache Kafka® cluster.
- Migrating Apache Kafka® cluster hosts to a different availability zone.
- Kafka UI for Apache Kafka®.
Changing cluster name and description
- Navigate to the folder dashboard
and select Managed Service for Kafka. - In the cluster row, click
, then select Edit. - Under Basic parameters, enter a new name and description for the cluster.
- Click Save.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To change cluster name and description:
-
View the description of the CLI command for updating a cluster:
yc managed-kafka cluster update --help -
Specify a new name and description in the cluster update command:
yc managed-kafka cluster update <cluster_name_or_ID> \ --new-name <new_cluster_name> \ --description <new_cluster_description>To find out the cluster name or ID, get a list of clusters in the folder.
Alert
Do not change the cluster name using Terraform. This will delete the existing cluster and create a new one.
To update the cluster description:
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
Edit the
descriptionparameter in the Managed Service for Apache Kafka® cluster's description:resource "yandex_mdb_kafka_cluster" "<cluster_name>" { name = "<cluster_name>" description = "<new_cluster_description>" ... } -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider article.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Cluster.update method and send the following request, e.g., via cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "name,description", "name": "<cluster_name>", "description": "<new_cluster_description>" }'Where:
-
updateMask: List of parameters to update as a single string, separated by commas.Specify the relevant parameters:
name: To change the cluster name.description: To change the cluster description.
-
name: New cluster name. -
description: New cluster description.
You can request the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService/Update call and send the following request, e.g., via gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "name", "description" ] }, "name": "<cluster_name>", "description": "<new_cluster_description>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ClusterService.UpdateWhere:
-
update_mask: List of parameters to update as an array ofpaths[]strings.Specify the relevant parameters:
name: To change the cluster name.description: To change the cluster description.
-
name: New cluster name. -
description: New cluster description.
You can request the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
Changing the broker host class and number
You can increase the number of broker hosts if the following conditions are met:
- The cluster uses Apache Kafka® with ZooKeeper. You cannot change the number of broker hosts in clusters with Apache Kafka® and the KRaft protocol.
- The cluster contains at least two broker hosts in different availability zones.
You cannot have fewer broker hosts. To comply with cluster high availability conditions, at least three broker hosts are required.
When changing the broker host class:
- A single broker host cluster will be unavailable for a few minutes with topic connections terminated.
- In a multiple broker host cluster, hosts will be stopped and updated one at a time. Stopped hosts will be unavailable for a few minutes.
We recommend changing broker host class only when there is no active workload on the cluster.
To change the class and number of hosts:
-
Navigate to the folder dashboard
and select Managed Service for Kafka. -
In the cluster row, click
, then select Edit. -
Change the required settings:
- To edit the broker host class, select a new Host class.
- Change Number of brokers in zone.
-
Click Save.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To change the class and number of hosts:
-
Get information about the cluster:
yc managed-kafka cluster list yc managed-kafka cluster get <cluster_name_or_ID> -
View the description of the CLI command for updating a cluster:
yc managed-kafka cluster update --help -
To increase the number of broker hosts, run this command:
yc managed-kafka cluster update <cluster_name_or_ID> --brokers-count <number> -
To change the broker host class, run this command:
yc managed-kafka cluster update <cluster_name_or_ID> --resource-preset <host_class>
To find out the cluster name or ID, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
In the Managed Service for Apache Kafka® cluster description, change the
brokers_countparameter to increase the number of broker hosts:resource "yandex_mdb_kafka_cluster" "<cluster_name>" { config { brokers_count = <number_of_broker_hosts> ... } ... } -
In the Managed Service for Apache Kafka® cluster description, edit the value of the
resource_preset_idparameter underkafka.resourcesto specify a new broker host class:resource "yandex_mdb_kafka_cluster" "<cluster_name>" { ... kafka { resources { resource_preset_id = "<broker_host_class>" ... } } } -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider article.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Request a list of available host classes:
-
Use the ResourcePreset.list method and run the request, e.g., via cURL
:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/resourcePresets' -
View the server response to make sure your request was successful.
-
-
Change the class and number of broker hosts as appropriate:
-
Use the Cluster.update method and send the following request, e.g., via cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.kafka.resources.resourcePresetId,configSpec.brokersCount", "configSpec": { "kafka": { "resources": { "resourcePresetId": "<broker_host_class_ID>" } }, "brokersCount": "<number_of_broker_hosts>" } }'Where:
-
updateMask: List of parameters to update as a single string, separated by commas.Specify the relevant parameters:
configSpec.kafka.resources.resourcePresetId: To change the broker host class.configSpec.brokersCount: To change the number of broker hosts.
You can request the cluster ID with the list of clusters in the folder. The list of available host classes with their IDs came to you earlier.
-
-
View the server response to make sure your request was successful.
-
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Get the list of available host classes:
-
Use the ResourcePresetService/List call and run the request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/resource_preset_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ResourcePresetService.List -
View the server response to make sure your request was successful.
-
-
Change the host class as appropriate:
-
Use the ClusterService/Update call and send the following request, e.g., via gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "config_spec.kafka.resources.resource_preset_id", "config_spec.brokers_count" ] }, "config_spec": { "kafka": { "resources": { "resource_preset_id": "<broker_host_class_ID>" } }, "brokers_count": { "value": "<number_of_broker_hosts>" } } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ClusterService.UpdateWhere:
-
update_mask: List of parameters to update as an array ofpaths[]strings.Specify the relevant parameters:
config_spec.kafka.resources.resource_preset_id: To change the broker host class.config_spec.brokers_count: To change the number of broker hosts.
You can request the cluster ID with the list of clusters in the folder. Earlier, you already obtained the list of available host classes with their IDs.
-
-
View the server response to make sure your request was successful.
-
Changing the ZooKeeper host class
- Navigate to the folder dashboard
and select Managed Service for Kafka. - In the cluster row, click
, then select Edit. - Select a new ZooKeeper host class.
- Click Save.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To change the class of ZooKeeper hosts:
-
Get information about the cluster:
yc managed-kafka cluster list yc managed-kafka cluster get <cluster_name_or_ID> -
View the description of the CLI command for updating a cluster:
yc managed-kafka cluster update --help -
To change the ZooKeeper host class, run this command:
yc managed-kafka cluster update <cluster_name_or_ID> \ --zookeeper-resource-preset <host_class>
To find out the cluster name or ID, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
In the Managed Service for Apache Kafka® cluster description, edit the value of the
resource_preset_idparameter underzookeeper.resourcesto specify a new ZooKeeper host class:resource "yandex_mdb_kafka_cluster" "<cluster_name>" { ... zookeeper { resources { resource_preset_id = "<ZooKeeper_host_class>" ... } } } -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider article.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Request a list of available host classes:
-
Use the ResourcePreset.list method and run the request, e.g., via cURL
:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/resourcePresets' -
View the server response to make sure your request was successful.
-
-
Change the host class as appropriate:
-
Use the Cluster.update method and send the following request, e.g., via cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.zookeeper.resources.resourcePresetId", "configSpec": { "zookeeper": { "resources": { "resourcePresetId": "<ZooKeeper_host_class_ID>" } } } }'Where:
-
updateMask: List of parameters to update as a single string, separated by commas.Specify the relevant parameters:
configSpec.zookeeper.resources.resourcePresetId: To change the ZooKeeper host class.
You can get the cluster ID with the list of clusters in the folder. Earlier, you already obtained the list of available host classes with their IDs.
-
-
View the server response to make sure your request was successful.
-
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Get the list of available host classes:
-
Use the ResourcePresetService/List call and run the request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/resource_preset_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ResourcePresetService.List -
View the server response to make sure your request was successful.
-
-
Change the host class as appropriate:
-
Use the ClusterService/Update call and send the following request, e.g., via gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "config_spec.zookeeper.resources.resource_preset_id" ] }, "config_spec": { "zookeeper": { "resources": { "resource_preset_id": "<ZooKeeper_host_class_ID>" } } } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ClusterService.UpdateWhere:
-
update_mask: List of parameters to update as an array ofpaths[]strings.Specify the relevant parameters:
config_spec.zookeeper.resources.resource_preset_id: To change the ZooKeeper host class.
You can request the cluster ID with the list of clusters in the folder. Earlier, you already obtained the list of available host classes with their IDs.
-
-
View the server response to make sure your request was successful.
-
Changing the KRaft host class
Note
The KRaft host class is used only in clusters with Apache Kafka® 3.6 or higher.
- Navigate to the folder dashboard
and select Managed Service for Kafka. - In the cluster row, click
, then select Edit. - Select a new KRaft host class.
- Click Save.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To change the class of KRaft hosts:
-
Get information about the cluster:
yc managed-kafka cluster list yc managed-kafka cluster get <cluster_name_or_ID> -
View the description of the CLI command for updating a cluster:
yc managed-kafka cluster update --help -
To change the KRaft host class, run this command:
yc managed-kafka cluster update <cluster_name_or_ID> \ --controller-resource-preset <host_class>
To find out the cluster name or ID, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
In the Managed Service for Apache Kafka® cluster description, edit the value of the
resource_preset_idparameter underkraft.resourcesto specify a new KRaft host class:resource "yandex_mdb_kafka_cluster" "<cluster_name>" { ... kraft { resources { resource_preset_id = "<KRaft_host_class>" ... } } } -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider article.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Request a list of available host classes:
-
Use the ResourcePreset.list method and run the request, e.g., via cURL
:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/resourcePresets' -
View the server response to make sure your request was successful.
-
-
Change the host class as appropriate:
-
Use the Cluster.update method and send the following request, e.g., via cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.kraft.resources.resourcePresetId", "configSpec": { "kraft": { "resources": { "resourcePresetId": "<KRaft_host_class_ID>" } } } }'Where:
-
updateMask: List of parameters to update as a single string, separated by commas.Specify the relevant parameters:
configSpec.kraft.resources.resourcePresetId: To change the KRaft host class.
You can get the cluster ID with the list of clusters in the folder. Earlier, you already obtained the list of available host classes with their IDs.
-
-
View the server response to make sure your request was successful.
-
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Get the list of available host classes:
-
Use the ResourcePresetService/List call and run the request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/resource_preset_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ResourcePresetService.List -
View the server response to make sure your request was successful.
-
-
Change the host class as appropriate:
-
Use the ClusterService/Update call and send the following request, e.g., via gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "config_spec.kraft.resources.resource_preset_id" ] }, "config_spec": { "kraft": { "resources": { "resource_preset_id": "<KRaft_host_class_ID>" } } } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ClusterService.UpdateWhere:
-
update_mask: List of parameters to update as an array ofpaths[]strings.Specify the relevant parameters:
config_spec.kraft.resources.resource_preset_id: To change the KRaft host class.
You can get the cluster ID with the list of clusters in the folder. Earlier, you already obtained the list of available host classes with their IDs.
-
-
View the server response to make sure your request was successful.
-
Changing security group and public access settings
- Navigate to the folder dashboard
and select Managed Service for Kafka. - In the cluster row, click
, then select Edit. - Under Network settings, select security groups for cluster network traffic.
- Enable or disable public access to a cluster via the Public access option.
- Click Save.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To edit the list of security groups for your cluster:
-
View the description of the CLI command for updating a cluster:
yc managed-kafka cluster update --help -
Specify the security groups and public access settings in the update cluster command:
yc managed-kafka cluster update <cluster_name_or_ID> \ --security-group-ids <list_of_security_groups> \ --assign-public-ip=<enable_public_access_to_cluster>Where:
--security-group-ids: List of cluster security group IDs.--assign-public-ip: Public access to the cluster,trueorfalse.
To find out the cluster name or ID, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
Change the values of the
security_group_idsandassign_public_ipparameters in the cluster description:resource "yandex_mdb_kafka_cluster" "<cluster_name>" { ... security_group_ids = [ <list_of_security_groups> ] ... config { assign_public_ip = "<enable_public_access_to_cluster>" ... } }Where:
security_group_ids: List of cluster security group IDs.assign_public_ip: Public access to the cluster,trueorfalse.
-
Validate your configuration.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider article.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Cluster.update method and send the following request, e.g., via cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ -url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "securityGroupIds,configSpec.assignPublicIp", "securityGroupIds": [ <list_of_security_group_IDs> ], "configSpec": { "assignPublicIp": "<enable_public_access_to_cluster>" } }'Where:
-
updateMask: List of parameters to update as a single string, separated by commas.Specify the relevant parameters:
securityGroupIds: To change the list of security groups.configSpec.assignPublicIp: To change the public access setting.
-
securityGroupIds: Security group IDs as an array of strings. Each string is a security group ID.Warning
The list of security groups assigned to the cluster will be completely overwritten by the list in the
securityGroupIdsparameter.Before running your request, make sure the list includes all the required security group IDs, including existing ones.
-
assignPublicIp: Access to broker hosts from the internet,trueorfalse.
You can request the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService/Update call and send the following request, e.g., via gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "security_group_ids", "config_spec.assign_public_ip" ] }, "security_group_ids": [ <list_of_security_group_IDs> ], "config_spec": { "assign_public_ip": "<enable_public_access_to_cluster>" } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ClusterService.UpdateWhere:
-
update_mask: List of parameters to update as an array ofpaths[]strings.Specify the relevant parameters:
security_group_ids: To change the list of security groups.config_spec.assign_public_ip: To change the public access setting.
-
security_group_ids: Security group IDs as an array of strings. Each string is a security group ID.Warning
The list of security groups assigned to the cluster will be completely overwritten by the list in the
security_group_idsparameter.Before running your request, make sure the list includes all the required security group IDs, including existing ones.
-
assign_public_ip: Access to broker hosts from the internet,trueorfalse.
You can request the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
Restart the cluster for the new public access settings to take effect.
You may need to additionally set up security groups to connect to the cluster.
Changing additional cluster settings
-
Navigate to the folder dashboard
and select Managed Service for Kafka. -
In the cluster row, click
, then select Edit. -
Change additional cluster settings:
-
Maintenance window: Maintenance window settings:
- To enable maintenance at any time, select arbitrary (default).
- To specify the preferred maintenance start time, select by schedule and specify the desired day of the week and UTC hour. For example, you can choose a time when the cluster is least loaded.
Maintenance operations are carried out both on enabled and disabled clusters. They may include updating the DBMS, applying patches, and so on.
-
Deletion protection: Manages cluster protection against accidental deletion.
Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.
-
To manage data schemas using Managed Schema Registry, enable the Schema registry setting.
Warning
You cannot disable data schema management using Managed Schema Registry after connecting it.
-
To allow sending requests to the Apache Kafka® API, enable Kafka Rest API.
It is implemented based on the Karapace
open-source tool. The Karapace API is compatible with the Confluent REST Proxy API with only minor exceptions.Warning
You cannot disable Kafka Rest API once it is enabled.
-
To use the Apache Kafka® web interface, enable the Kafka UI setting.
-
-
Click Save.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To change additional cluster settings:
-
View the description of the CLI command for updating a cluster:
yc managed-kafka cluster update --help -
Run the following command with the list of settings to update:
yc managed-kafka cluster update <cluster_name_or_ID> \ --maintenance-window type=<maintenance_type>,` `day=<day_of_week>,` `hour=<hour> \ --datatransfer-access=<allow_access_from_Data_Transfer> \ --deletion-protection \ --schema-registry=<data_schema_management>
You can update the following settings:
-
--maintenance-window: Maintenance window settings (including for disabled clusters), wheretypeis the maintenance type:anytime: At any time (default).weekly: On a schedule. For this value, also specify the following:day: Day of week, i.e.,MON,TUE,WED,THU,FRI,SAT, orSUN.hour: Hour of day (UTC), from1to24.
-
--deletion-protection: Cluster protection from accidental deletion,trueorfalse.Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.
-
--schema-registry: Manage data schemas using Managed Schema Registry,trueorfalse.Warning
You cannot disable data schema management using Managed Schema Registry after connecting it.
To find out the cluster name or ID, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
To set up the maintenance window (for disabled clusters as well), add the
maintenance_windowsection to the cluster description:resource "yandex_mdb_kafka_cluster" "<cluster_name>" { ... maintenance_window { type = <maintenance_type> day = <day_of_week> hour = <hour> } ... }Where:
type: Maintenance type. The possible values include:ANYTIME: AnytimeWEEKLY: On a schedule
day: Day of week for theWEEKLYtype, i.e.,MON,TUE,WED,THU,FRI,SAT, orSUN.hour: UTC hour for theWEEKLYtype, from1to24.
-
To enable cluster protection against accidental deletion by a user of your cloud, add the
deletion_protectionfield set totrueto your cluster description:resource "yandex_mdb_kafka_cluster" "<cluster_name>" { ... deletion_protection = <protect_cluster_from_deletion> }Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.
-
To enable data schema management using Managed Schema Registry, add the
config.schema_registryfield set totrueto the cluster description:resource "yandex_mdb_kafka_cluster" "<cluster_name>" { ... config { ... schema_registry = <data_schema_management> } }Warning
You cannot disable data schema management using Managed Schema Registry after connecting it.
-
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider article.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Cluster.update method and send the following request, e.g., via cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ -url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.restApiConfig.enabled,configSpec.schemaRegistry,maintenanceWindow,deletionProtection", "configSpec": { "schemaRegistry": true, "restApiConfig": { "enabled": true } }, "maintenanceWindow": { "anytime": {}, "weeklyMaintenanceWindow": { "day": "<day_of_week>", "hour": "<hour_UTC>" } }, "deletionProtection": <protect_cluster_from_deletion> }'Where:
-
updateMask: List of parameters to update as a single string, separated by commas.Specify the relevant parameters:
-
configSpec.schemaRegistry: To enable the settings for data schema management using Managed Schema Registry. -
configSpec.restApiConfig.enabled: To enable access to sending requests to the Apache Kafka® REST API. -
maintenanceWindow: To change the maintenance window settings, including for disabled clusters. -
deletionProtection: To enable or disable cluster protection against accidental deletion.Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.
-
-
configSpec.schemaRegistry: Set totrueto manage data schemas using Managed Schema Registry. You will not be able to edit this setting once it is enabled. -
configSpec.restApiConfig.enabled: Set totruefor access to sending requests to the Apache Kafka® REST API. You will not be able to edit this setting once it is enabled. -
maintenanceWindow: Maintenance window settings (including for disabled clusters). Select one of these options:anytime: At any time (default).weeklyMaintenanceWindow: On schedule:day: Day of week inDDDformat:MON,TUE,WED,THU,FRI,SAT, orSUN.hour: Time of day (UTC) inHHformat, from1to24.
-
deletionProtection: To enable (true) or disable (false) cluster protection against accidental deletion.Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.
You can get the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService/Update call and send the following request, e.g., via gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "maintenance_window", "deletion_protection", "config_spec.schema_registry", "config_spec.rest_api_config.enabled" ] }, "config_spec": { "schema_registry": true, "rest_api_config": { "enabled": true } }, "maintenance_window": { "anytime": {}, "weekly_maintenance_window": { "day": "<day_of_week>", "hour": "<hour_UTC>" } }, "deletion_protection": <protect_cluster_from_deletion> }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ClusterService.UpdateWhere:
-
update_mask: List of parameters to update as an array ofpaths[]strings.Specify the relevant parameters:
-
config_spec.schema_registry: To manage data schemas using Managed Schema Registry. -
config_spec.rest_api_config.enabled: To enable access to sending requests to the Apache Kafka® REST API. -
maintenance_window: To change the maintenance window settings, including for disabled clusters. -
deletion_protection: To enable or disable protection of the cluster, its databases, and users against accidental deletion.
-
-
schema_registry: Set totrueto manage data schemas using Managed Schema Registry. You will not be able to edit this setting once it is enabled. -
rest_api_config.enabled: Set totruefor access to sending requests to the Apache Kafka® REST API. You will not be able to edit this setting once it is enabled. -
maintenance_window: Maintenance window settings. Select one of these options:anytime: Any time.weekly_maintenance_window: On schedule:day: Day of week inDDDformat:MON,TUE,WED,THU,FRI,SAT, orSUN.hour: Time of day (UTC) inHHformat, from1to24.
-
deletion_protection: To enable (true) or disable (false) cluster protection against accidental deletion.Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.
You can get the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
Changing Apache Kafka® settings
-
Navigate to the folder dashboard
and select Managed Service for Kafka. -
In the cluster row, click
, then select Edit. -
Under Kafka Settings, click Settings.
For more information, see Apache Kafka® settings.
-
Click Save.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To update the Apache Kafka® settings:
-
See the description of the CLI command for updating cluster settings:
yc managed-kafka cluster update --help -
Change the Apache Kafka® settings in the cluster update command (the example below does not list all possible parameters):
yc managed-kafka cluster update <cluster_name_or_ID> \ --compression-type <compression_type> \ --log-flush-interval-messages <number_of_messages_in_log> \ --log-flush-interval-ms <maximum_time_to_store_messages>Where:
--log-flush-interval-messages: Number of messages in the log to trigger flushing to disk.--log-flush-interval-ms: Maximum time a message can be stored in memory before flushing to disk.
To find out the cluster name or ID, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
In the Managed Service for Apache Kafka® cluster description, modify the values of the parameters in the
kafka.kafka_configsection (the example below does not list all possible settings):resource "yandex_mdb_kafka_cluster" "<cluster_name>" { ... config { kafka { ... kafka_config { compression_type = "<compression_type>" log_flush_interval_messages = <maximum_number_of_messages_in_memory> ... } } } } -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider article.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Cluster.update method and send the following request, e.g., via cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ -url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.kafka.kafkaConfig_2_8.<setting_1_for_Apache Kafka®_2.8_configuration>,...,configSpec.kafka.kafkaConfig_2_8.<setting_N_for_Apache Kafka®_2.8_configuration>,configSpec.kafka.kafkaConfig_3.<setting_1_for_Apache Kafka®_3.x_configuration>,...,configSpec.kafka.kafkaConfig_3.<setting_N_for_Apache Kafka®_3.x_configuration>", "configSpec": { "kafka": { "kafkaConfig_2_8": { "<setting_1_for_Apache Kafka®_2.8_configuration>": "<setting_value>", "<setting_2_for_Apache Kafka®_2.8_configuration>": "<setting_value>", ..., "<setting_N_for_Apache Kafka®_2.8_configuration>": "<setting_value>" }, "kafkaConfig_3": { "<setting_1_for_Apache Kafka®_3.x_configuration>": "<setting_value>", "<setting_2_for_Apache Kafka®_3.x_configuration>": "<setting_value>", ..., "<setting_N_for_Apache Kafka®_3.x_configuration>": "<setting_value>" } } } }'Where:
-
updateMask: List of parameters to update as a single string, separated by commas.Specify the relevant parameters:
configSpec.kafka.kafkaConfig_2_8.<setting_for_Apache Kafka®_2.8_configuration>: For Apache Kafka®2.8.configSpec.kafka.kafkaConfig_3.<setting_for_Apache Kafka®_3.x_configuration>: For Apache Kafka®3.x.
-
configSpec.kafka.kafkaConfig_2_8.<setting_for_Apache Kafka®_2.8_configuration: Specify a new value for the setting if using Apache Kafka®2.8. -
configSpec.kafka.kafkaConfig_3.<setting_for_Apache Kafka®_3.x_configuration: Specify a new value for the setting if using Apache Kafka®3.x.
You can request the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService/Update call and send the following request, e.g., via gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "config_spec.kafka.kafka_config_2_8.<setting_1_for_Apache Kafka®_2.8_configuration>", ..., "config_spec.kafka.kafka_config_2_8.<setting_N_for_Apache Kafka®_2.8_configuration>", "config_spec.kafka.kafka_config_3.<setting_1_for_Apache Kafka®_3.x_configuration>", ..., "config_spec.kafka.kafka_config_3.<setting_N_for_Apache Kafka®_3.x_configuration>" ] } "config_spec": { "kafka": { "kafka_config_2_8": { "<setting_1_for_Apache Kafka®_2.8_configuration>": "<setting_value>", "<setting_2_for_Apache Kafka®_2.8_configuration>": "<setting_value>", ..., "<setting_N_for_Apache Kafka®_2.8_configuration>": "<setting_value>" }, "kafka_config_3": { "<setting_1_for_Apache Kafka®_3.x_configuration>": "<setting_value>", "<setting_2_for_Apache Kafka®_3.x_configuration>": "<setting_value>", ..., "<setting_N_for_Apache Kafka®_3.x_configuration>": "<setting_value>" } } } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ClusterService.UpdateWhere:
-
update_mask: List of parameters to update as an array ofpaths[]strings.Specify the relevant parameters:
config_spec.kafka.kafka_config_2_8.<setting_for_Apache Kafka®_2.8_configuration>: For Apache Kafka®2.8.config_spec.kafka.kafka_config_3.<setting_for_Apache Kafka®_3.x_configuration>: For Apache Kafka®3.x.
-
config_spec.kafka.kafka_config_2_8.<setting_for_Apache Kafka®_2.8_configuration: Specify a new value for the setting if using Apache Kafka®2.8. -
config_spec.kafka.kafka_config_3.<setting_for_Apache Kafka®_3.x_configuration: Specify a new value for the setting if using Apache Kafka®3.x.
You can request the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
Moving a cluster to another folder
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To move a cluster:
-
View the description of the CLI move cluster command:
yc managed-kafka cluster move --help -
Specify the destination folder in that command:
yc managed-kafka cluster move <cluster_name_or_ID> \ --destination-folder-name=<destination_folder_name>To find out the cluster name or ID, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see this guide.
-
In the Managed Service for Apache Kafka® cluster description, edit or add the
folder_idvalue:resource "yandex_mdb_kafka_cluster" "<cluster_name>" { ... folder_id = "<destination_folder_ID>" } -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider article.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Cluster.move method and send the following request, e.g., via cURL
:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ -url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>:move' \ --data '{ "destinationFolderId": "<folder_ID>" }'You can request the cluster ID with the list of clusters in the folder.
-
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService/Move call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "destination_folder_id": "<folder_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.ClusterService.MoveYou can request the cluster ID with the list of clusters in the folder.
-
View the server response to make sure your request was successful.