Updating Apache Kafka® cluster settings

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

To change the name and description of a cluster:

1. View the description of the update cluster CLI command:

   yc managed-kafka cluster update --help
   

2. Specify a new name and description in the cluster update command:

   yc managed-kafka cluster update <cluster_name_or_ID> \
     --new-name <new_cluster_name> \
     --description <new_cluster_description>
   

   To find out the cluster name or ID, get a list of clusters in the folder.

To update the cluster description:

1. Open the current Terraform configuration file with an infrastructure plan.

   For more information about creating this file, see Creating clusters.

2. In the Managed Service for Apache Kafka® cluster description, change the description parameter value:

   resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
     name        = "<cluster_name>"
     description = "<new_cluster_description>"
     ...
   }
   

3. Make sure the settings are correct.

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

4. Confirm updating the resources.

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

For more information, see the Terraform provider documentation.

resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Use the Cluster.update method and send the following request, e.g., via cURL:

   Warning

   The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.

   curl \
       --request PATCH \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --header "Content-Type: application/json" \
       --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \
       --data '{
                 "updateMask": "name,description",
                 "name": "<cluster_name>",
                 "description": "<new_cluster_description>"
               }'
   

   Where:

   • updateMask: List of parameters to update as a single string, separated by commas.

     Specify the relevant parameters:

     • name: To change the cluster name.
     • description: To change the cluster description.

   • name: New cluster name.

   • description: New cluster description.

   You can request the cluster ID with the list of clusters in the folder.

3. View the server response to make sure the request was successful.

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService/Update call and send the following request, e.g., via gRPCurl:

   Warning

   The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the update_mask parameter as an array of paths[] strings.

   Format for listing settings

   "update_mask": {
       "paths": [
           "<setting_1>",
           "<setting_2>",
           ...
           "<setting_N>"
       ]
   }
   

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
             "cluster_id": "<cluster_ID>",
             "update_mask": {
               "paths": [
                 "name",
                 "description"
               ]
             },
             "name": "<cluster_name>",
             "description": "<new_cluster_description>"
           }' \
       mdb.api.cloud.yandex.net:443 \
       yandex.cloud.mdb.kafka.v1.ClusterService.Update
   

   Where:

   • update_mask: List of parameters to update as an array of paths[] strings.

     Specify the relevant parameters:

     • name: To change the cluster name.
     • description: To change the cluster description.

   • name: New cluster name.

   • description: New cluster description.

   You can request the cluster ID with the list of clusters in the folder.

4. View the server response to make sure the request was successful.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

To change the class and number of hosts:

1. Get information about the cluster:

   yc managed-kafka cluster list
   yc managed-kafka cluster get <cluster_name_or_ID>
   

2. View the description of the CLI command to update the cluster:

   yc managed-kafka cluster update --help
   

3. To increase the number of broker hosts, run this command:

   yc managed-kafka cluster update <cluster_name_or_ID> --brokers-count <number>
   

4. To change the broker host class, run this command:

   yc managed-kafka cluster update <cluster_name_or_ID> --resource-preset <host_class>
   

To find out the cluster name or ID, get a list of clusters in the folder.

1. Open the current Terraform configuration file with an infrastructure plan.

   For more information about creating this file, see Creating clusters.

2. In the Managed Service for Apache Kafka® cluster description, change the brokers_count parameter to increase the number of broker hosts:

   resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
     config {
       brokers_count = <number_of_broker_hosts>
       ...
     }
     ...
   }
   

3. In the Managed Service for Apache Kafka® cluster description, edit the value of the resource_preset_id parameter under kafka.resources to specify a new broker host class:

   resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
     ...
     kafka {
       resources {
         resource_preset_id = "<broker_host_class>"
         ...
       }
     }
   }
   

4. Make sure the settings are correct.

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

5. Confirm updating the resources.

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

For more information, see the Terraform provider documentation.

resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Request a list of available host classes:

   1. Use the ResourcePreset.list method and run the request, e.g., via cURL:

      curl \
          --request GET \
          --header "Authorization: Bearer $IAM_TOKEN" \
          --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/resourcePresets'
      

   2. View the server response to make sure the request was successful.

3. Change the class and number of broker hosts as appropriate:

   1. Use the Cluster.update method and send the following request, e.g., via cURL:

      Warning

      The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.

      curl \
          --request PATCH \
          --header "Authorization: Bearer $IAM_TOKEN" \
          --header "Content-Type: application/json" \
          --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \
          --data '{
                    "updateMask": "configSpec.kafka.resources.resourcePresetId,configSpec.brokersCount",
                    "configSpec": {
                      "kafka": {
                        "resources": {
                          "resourcePresetId": "<broker_host_class_ID>"
                        }
                      },
                      "brokersCount": "<number_of_broker_hosts>"
                    }
                  }'
      

      Where:

      • updateMask: List of parameters to update as a single string, separated by commas.

        Specify the relevant parameters:

        • configSpec.kafka.resources.resourcePresetId: To change the broker host class.
        • configSpec.brokersCount: To change the number of broker hosts.

      You can request the cluster ID with the list of clusters in the folder. The list of available host classes with their IDs came to you earlier.

   2. View the server response to make sure the request was successful.

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Request a list of available host classes:

   1. Use the ResourcePresetService/List call and run the request, e.g., via gRPCurl:

      grpcurl \
          -format json \
          -import-path ~/cloudapi/ \
          -import-path ~/cloudapi/third_party/googleapis/ \
          -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/resource_preset_service.proto \
          -rpc-header "Authorization: Bearer $IAM_TOKEN" \
          mdb.api.cloud.yandex.net:443 \
          yandex.cloud.mdb.kafka.v1.ResourcePresetService.List
      

   2. View the server response to make sure the request was successful.

4. Change the host class as appropriate:

   1. Use the ClusterService/Update call and send the following request, e.g., via gRPCurl:

      Warning

      The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the update_mask parameter as an array of paths[] strings.

      Format for listing settings

      "update_mask": {
          "paths": [
              "<setting_1>",
              "<setting_2>",
              ...
              "<setting_N>"
          ]
      }
      

      grpcurl \
          -format json \
          -import-path ~/cloudapi/ \
          -import-path ~/cloudapi/third_party/googleapis/ \
          -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \
          -rpc-header "Authorization: Bearer $IAM_TOKEN" \
          -d '{
                "cluster_id": "<cluster_ID>",
                "update_mask": {
                  "paths": [
                    "config_spec.kafka.resources.resource_preset_id",
                    "config_spec.brokers_count"
                  ]
                },
                "config_spec": {
                  "kafka": {
                    "resources": {
                      "resource_preset_id": "<broker_host_class_ID>"
                    }
                  },
                  "brokers_count": {
                    "value": "<number_of_broker_hosts>"
                  }
                }
              }' \
          mdb.api.cloud.yandex.net:443 \
          yandex.cloud.mdb.kafka.v1.ClusterService.Update
      

      Where:

      • update_mask: List of parameters to update as an array of paths[] strings.

        Specify the relevant parameters:

        • config_spec.kafka.resources.resource_preset_id: To change the broker host class.
        • config_spec.brokers_count: To change the number of broker hosts.

      You can request the cluster ID with the list of clusters in the folder. The list of available host classes with their IDs came to you earlier.

   2. View the server response to make sure the request was successful.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

To change the class of ZooKeeper hosts:

1. Get information about the cluster:

   yc managed-kafka cluster list
   yc managed-kafka cluster get <cluster_name_or_ID>
   

2. View the description of the CLI command to update the cluster:

   yc managed-kafka cluster update --help
   

3. To change the ZooKeeper host class, run this command:

   yc managed-kafka cluster update <cluster_name_or_ID> \
     --zookeeper-resource-preset <host_class>
   

To find out the cluster name or ID, get a list of clusters in the folder.

1. Open the current Terraform configuration file with an infrastructure plan.

   For more information about creating this file, see Creating clusters.

2. In the Managed Service for Apache Kafka® cluster description, edit the value of the resource_preset_id parameter under zookeeper.resources to specify a new ZooKeeper host class:

   resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
     ...
     zookeeper {
       resources {
         resource_preset_id = "<ZooKeeper_host_class>"
         ...
       }
     }
    }
   

3. Make sure the settings are correct.

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

4. Confirm updating the resources.

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

For more information, see the Terraform provider documentation.

resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Request a list of available host classes:

   1. Use the ResourcePreset.list method and run the request, e.g., via cURL:

      curl \
          --request GET \
          --header "Authorization: Bearer $IAM_TOKEN" \
          --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/resourcePresets'
      

   2. View the server response to make sure the request was successful.

3. Change the host class as appropriate:

   1. Use the Cluster.update method and send the following request, e.g., via cURL:

      Warning

      The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.

      curl \
          --request PATCH \
          --header "Authorization: Bearer $IAM_TOKEN" \
          --header "Content-Type: application/json" \
          --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \
          --data '{
                    "updateMask": "configSpec.zookeeper.resources.resourcePresetId",
                    "configSpec": {
                      "zookeeper": {
                        "resources": {
                          "resourcePresetId": "<ZooKeeper_host_class_ID>"
                        }
                      }
                    }
                  }'
      

      Where:

      • updateMask: List of parameters to update as a single string, separated by commas.

        Specify the relevant parameters:

        • configSpec.zookeeper.resources.resourcePresetId: To change the ZooKeeper host class.

      You can request the cluster ID with the list of clusters in the folder. The list of available host classes with their IDs came to you earlier.

   2. View the server response to make sure the request was successful.

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Request a list of available host classes:

   1. Use the ResourcePresetService/List call and run the request, e.g., via gRPCurl:

      grpcurl \
          -format json \
          -import-path ~/cloudapi/ \
          -import-path ~/cloudapi/third_party/googleapis/ \
          -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/resource_preset_service.proto \
          -rpc-header "Authorization: Bearer $IAM_TOKEN" \
          mdb.api.cloud.yandex.net:443 \
          yandex.cloud.mdb.kafka.v1.ResourcePresetService.List
      

   2. View the server response to make sure the request was successful.

4. Change the host class as appropriate:

   1. Use the ClusterService/Update call and send the following request, e.g., via gRPCurl:

      Warning

      The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the update_mask parameter as an array of paths[] strings.

      Format for listing settings

      "update_mask": {
          "paths": [
              "<setting_1>",
              "<setting_2>",
              ...
              "<setting_N>"
          ]
      }
      

      grpcurl \
          -format json \
          -import-path ~/cloudapi/ \
          -import-path ~/cloudapi/third_party/googleapis/ \
          -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \
          -rpc-header "Authorization: Bearer $IAM_TOKEN" \
          -d '{
                "cluster_id": "<cluster_ID>",
                "update_mask": {
                  "paths": [
                    "config_spec.zookeeper.resources.resource_preset_id"
                  ]
                },
                "config_spec": {
                  "zookeeper": {
                    "resources": {
                      "resource_preset_id": "<ZooKeeper_host_class_ID>"
                    }
                  }
                }
              }' \
          mdb.api.cloud.yandex.net:443 \
          yandex.cloud.mdb.kafka.v1.ClusterService.Update
      

      Where:

      • update_mask: List of parameters to update as an array of paths[] strings.

        Specify the relevant parameters:

        • config_spec.zookeeper.resources.resource_preset_id: To change the ZooKeeper host class.

      You can request the cluster ID with the list of clusters in the folder. The list of available host classes with their IDs came to you earlier.

   2. View the server response to make sure the request was successful.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

To edit the list of security groups for your cluster:

1. View the description of the update cluster CLI command:

   yc managed-kafka cluster update --help
   

2. Specify the security groups and public access settings in the update cluster command:

   yc managed-kafka cluster update <cluster_name_or_ID> \
      --security-group-ids <list_of_security_groups> \
      --assign-public-ip=<public_access>
   

   Where:

   • --security-group-ids: List of cluster security group IDs.
   • --assign-public-ip: Public access to the cluster, true or false.

   To find out the cluster name or ID, get a list of clusters in the folder.

1. Open the current Terraform configuration file with an infrastructure plan.

   For more information about creating this file, see Creating clusters.

2. Change the values of the security_group_ids and assign_public_ip parameters in the cluster description:

   resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
     ...
     security_group_ids = [ <list_of_security_groups> ]
     ...
     config {
       assign_public_ip = "<public_access>"
       ...
       }
   }
   

   Where:

   • security_group_ids: List of cluster security group IDs.
   • assign_public_ip: Public access to the cluster, true or false.

3. Make sure the settings are correct.

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

4. Confirm updating the resources.

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

For more information, see the Terraform provider documentation.

resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Use the Cluster.update method and send the following request, e.g., via cURL:

   Warning

   The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.

   curl \
       --request PATCH \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --header "Content-Type: application/json" \
       -url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \
       --data '{
                 "updateMask": "securityGroupIds,configSpec.assignPublicIp",
                 "securityGroupIds": [
                   <list_of_security_group_IDs>
                 ],
                 "configSpec": {
                   "assignPublicIp": "<public_access:_true_or_false>"
                 }
               }'
   

   Where:

   • updateMask: List of parameters to update as a single string, separated by commas.

     Specify the relevant parameters:

     • securityGroupIds: To change the list of security groups.
     • configSpec.assignPublicIp: To change the public access setting.

   • securityGroupIds: Security group IDs as an array of strings. Each string is a security group ID.

     Warning

     The list of security groups assigned to the cluster will be completely overwritten by the list in the securityGroupIds parameter.

     Before executing your request, make sure the list includes all the required security group IDs, including existing ones.

   • assignPublicIp: Internet access to the broker hosts, true or false.

   You can request the cluster ID with the list of clusters in the folder.

3. View the server response to make sure the request was successful.

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService/Update call and send the following request, e.g., via gRPCurl:

   Warning

   The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the update_mask parameter as an array of paths[] strings.

   Format for listing settings

   "update_mask": {
       "paths": [
           "<setting_1>",
           "<setting_2>",
           ...
           "<setting_N>"
       ]
   }
   

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
             "cluster_id": "<cluster_ID>",
             "update_mask": {
               "paths": [
                 "security_group_ids",
                 "config_spec.assign_public_ip"
               ]
             },
             "security_group_ids": [
               <list_of_security_group_IDs>
             ],
             "config_spec": {
               "assign_public_ip": "<public_access:_true_or_false>"
             }
           }' \
       mdb.api.cloud.yandex.net:443 \
       yandex.cloud.mdb.kafka.v1.ClusterService.Update
   

   Where:

   • update_mask: List of parameters to update as an array of paths[] strings.

     Specify the relevant parameters:

     • security_group_ids: To change the list of security groups.
     • config_spec.assign_public_ip: To change the public access setting.

   • security_group_ids: Security group IDs as an array of strings. Each string is a security group ID.

     Warning

     The list of security groups assigned to the cluster will be completely overwritten by the list in the security_group_ids parameter.

     Before executing your request, make sure the list includes all the required security group IDs, including existing ones.

   • assign_public_ip: Internet access to the broker hosts, true or false.

   You can request the cluster ID with the list of clusters in the folder.

4. View the server response to make sure the request was successful.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

To change additional cluster settings:

1. View the description of the update cluster CLI command:

   yc managed-kafka cluster update --help
   

2. Run the following command with the list of settings to update:

   yc managed-kafka cluster update <cluster_name_or_ID> \
      --maintenance-window type=<maintenance_type>,`
                          `day=<day_of_week>,`
                          `hour=<hour> \
      --datatransfer-access=<access_to_cluster> \
      --deletion-protection \
      --schema-registry=<data_schema_management>
   

You can change the following settings:

• --maintenance-window: Maintenance window settings (including for disabled clusters), where type is the maintenance type:

  • anytime (default): Any time.
  • weekly: On a schedule. If setting this value, specify the day of week and the hour:
    • day: Day of week in DDD format: MON, TUE, WED, THU, FRI, SAT, or SUN.
    • hour: Hour (UTC) in HH format: 1 to 24.

• --deletion-protection: Cluster protection from accidental deletion, true or false.

  Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.

• --schema-registry: Manage data schemas using Managed Schema Registry, true or false.

  Warning

  You cannot disable data schema management using Managed Schema Registry after connecting it.

To find out the cluster name or ID, get a list of clusters in the folder.

1. Open the current Terraform configuration file with an infrastructure plan.

   For more information about creating this file, see Creating clusters.

2. To set up the maintenance window (for disabled clusters as well), add the maintenance_window section to the cluster description:

   resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
     ...
     maintenance_window {
       type = <maintenance_type>
       day  = <day_of_week>
       hour = <hour>
     }
     ...
   }
   

   Where:

   • type: Maintenance type. The possible values include:
     • anytime: Anytime.
     • weekly: By schedule.
   • day: Day of the week for the weekly type in DDD format, e.g., MON.
   • hour: Hour of the day for the weekly type in the HH format, e.g., 21.

3. To enable cluster protection against accidental deletion by a user of your cloud, add the deletion_protection field set to true to your cluster description:

   resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
     ...
     deletion_protection = <cluster_deletion_protection>
   }
   

   Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.

4. To enable data schema management using Managed Schema Registry, add the config.schema_registry field set to true to the cluster description:

   resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
     ...
     config {
       ...
       schema_registry  = <data_schema_management>
     }
   }
   

   Warning

   You cannot disable data schema management using Managed Schema Registry after connecting it.

5. Make sure the settings are correct.

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

6. Confirm updating the resources.

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

For more information, see the Terraform provider documentation.

resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Use the Cluster.update method and send the following request, e.g., via cURL:

   Warning

   The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.

   curl \
       --request PATCH \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --header "Content-Type: application/json" \
       -url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \
       --data '{
                 "updateMask": "configSpec.restApiConfig.emabled,configSpec.schemaRegistry,maintenanceWindow,deletionProtection",
                 "configSpec": {
                   "schemaRegistry": true,
                   "restApiConfig": {
                     "enabled": true
                   }
                 },
                 "maintenanceWindow": {
                   "anytime": {},
                   "weeklyMaintenanceWindow": {
                     "day": "<day_of_week>",
                     "hour": "<hour_UTC>"
                   }
                 },
                 "deletionProtection": <cluster_deletion_protection:_true_or_false>
               }'
   

   Where:

   • updateMask: List of parameters to update as a single string, separated by commas.

     Specify the relevant parameters:

     • configSpec.schemaRegistry: To enable the settings for data schema management using Managed Schema Registry.

     • configSpec.restApiConfig.enabled: To enable access to sending requests to the Apache Kafka® REST API.

     • maintenanceWindow: To change the maintenance window settings, including for disabled clusters.

     • deletionProtection: To enable or disable cluster protection against accidental deletion.

       Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.

   • configSpec.schemaRegistry: Set to true to manage data schemas using Managed Schema Registry. You will not be able to edit this setting once it is enabled.

   • configSpec.restApiConfig.enabled: Set to true for access to sending requests to the Apache Kafka® REST API. You will not be able to edit this setting once it is enabled.

   • maintenanceWindow: Maintenance window settings (including for disabled clusters). Select one of the options:

     • anytime: At any time (default).
     • weeklyMaintenanceWindow: On schedule:
       • day: Day of week in DDD format: MON, TUE, WED, THU, FRI, SAT, or SUN.
       • hour: Hour of day (UTC) in HH format, from 1 to 24.

   • deletionProtection: To enable or disable cluster protection against accidental deletion.

     Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.

   You can request the cluster ID with the list of clusters in the folder.

3. View the server response to make sure the request was successful.

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService/Update call and send the following request, e.g., via gRPCurl:

   Warning

   The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the update_mask parameter as an array of paths[] strings.

   Format for listing settings

   "update_mask": {
       "paths": [
           "<setting_1>",
           "<setting_2>",
           ...
           "<setting_N>"
       ]
   }
   

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
             "cluster_id": "<cluster_ID>",
             "update_mask": {
               "paths": [
                 "maintenance_window",
                 "deletion_protection",
                 "config_spec.schema_registry",
                 "config_spec.rest_api_config.enabled"
               ]
             },
             "config_spec": {
               "schema_registry": true,
               "rest_api_config": {
                 "enabled": true
               }
             },
             "maintenance_window": {
               "anytime": {},
               "weekly_maintenance_window": {
                 "day": "<day_of_week>",
                 "hour": "<hour_UTC>"
               }
             },
             "deletion_protection": <cluster_deletion_protection:_true_or_false>
           }' \
       mdb.api.cloud.yandex.net:443 \
       yandex.cloud.mdb.kafka.v1.ClusterService.Update
   

   Where:

   • update_mask: List of parameters to update as an array of paths[] strings.

     Specify the relevant parameters:

     • config_spec.schema_registry: To manage data schemas using Managed Schema Registry.

     • config_spec.rest_api_config.enabled: To enable access to sending requests to the Apache Kafka® REST API.

     • maintenance_window: To change the maintenance window settings, including for disabled clusters.

     • deletion_protection: To enable or disable protection of the cluster, its databases, and users against accidental deletion.

   • schema_registry: Set to true to manage data schemas using Managed Schema Registry. You will not be able to edit this setting once it is enabled.

   • rest_api_config.enabled: Set to true for access to sending requests to the Apache Kafka® REST API. You will not be able to edit this setting once it is enabled.

   • maintenance_window: Maintenance window settings. Select one of these options:

     • anytime: Anytime.
     • weekly_maintenance_window: On schedule:
       • day: Day of week in DDD format: MON, TUE, WED, THU, FRI, SAT, or SUN.
       • hour: Hour of day (UTC) in HH format, from 1 to 24.

   • deletion_protection: To enable (true) or disable (false) cluster protection against accidental deletion.

     Even with cluster deletion protection enabled, one can still delete a user or topic or connect manually and delete the data.

   You can request the cluster ID with the list of clusters in the folder.

4. View the server response to make sure the request was successful.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

To update the Apache Kafka® settings:

1. See the description of the CLI command for updating cluster settings:

   yc managed-kafka cluster update --help
   

2. Change the Apache Kafka® settings in the cluster update command (the example below does not list all possible parameters):

   yc managed-kafka cluster update <cluster_name_or_ID> \
      --compression-type <compression_type> \
      --log-flush-interval-messages <number_of_messages_in_log> \
      --log-flush-interval-ms <maximum_time_to_store_messages>
   

   Where:

   • --log-flush-interval-messages: Number of messages in the log to trigger flushing to disk.
   • --log-flush-interval-ms: Maximum time a message can be stored in memory before flushing to disk.

   To find out the cluster name or ID, get a list of clusters in the folder.

1. Open the current Terraform configuration file with an infrastructure plan.

   For more information about creating this file, see Creating clusters.

2. In the Managed Service for Apache Kafka® cluster description, modify the values of the parameters in the kafka.kafka_config section (the example below does not list all possible settings):

   resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
     ...
     config {
       kafka {
         ...
         kafka_config {
           compression_type            = "<compression_type>"
           log_flush_interval_messages = <maximum_number_of_messages_in_memory>
           ...
         }
       }
     }
   }
   

3. Make sure the settings are correct.

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

4. Confirm updating the resources.

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

For more information, see the Terraform provider documentation.

resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Use the Cluster.update method and send the following request, e.g., via cURL:

   Warning

   The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.

   curl \
       --request PATCH \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --header "Content-Type: application/json" \
       -url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>' \
       --data '{
                 "updateMask": "configSpec.kafka.kafkaConfig_2_8.<setting_1_for_Apache Kafka®_2.8_configuration>,...,configSpec.kafka.kafkaConfig_2_8.<setting_N_for_Apache Kafka®_2.8_configuration>,configSpec.kafka.kafkaConfig_3.<setting_1_for_Apache Kafka®_3.x_configuration>,...,configSpec.kafka.kafkaConfig_3.<setting_N_for_Apache Kafka®_3.x_configuration>",
                 "configSpec": {
                   "kafka": {
                     "kafkaConfig_2_8": {
                       "<setting_1_for_Apache Kafka®_2.8_configuration>": "<setting_value>",
                       "<setting_2_for_Apache Kafka®_2.8_configuration>": "<setting_value>",
                       ...,
                       "<setting_N_for_Apache Kafka®_2.8_configuration>": "<setting_value>"
                     },
                     "kafkaConfig_3": {
                       "<setting_1_for_Apache Kafka®_3.x_configuration>": "<setting_value>",
                       "<setting_2_for_Apache Kafka®_3.x_configuration>": "<setting_value>",
                       ...,
                       "<setting_N_for_Apache Kafka®_3.x_configuration>": "<setting_value>"
                     }
                   }
                 }
               }'
   

   Where:

   • updateMask: List of parameters to update as a single string, separated by commas.

     Specify the relevant parameters:

     • configSpec.kafka.kafkaConfig_2_8.<setting_for_Apache Kafka®_2.8_configuration>: For Apache Kafka® 2.8.
     • configSpec.kafka.kafkaConfig_3.<setting_for_Apache Kafka®_3.x_configuration>: For Apache Kafka® 3.x.

   • configSpec.kafka.kafkaConfig_2_8.<setting_for_Apache Kafka®_2.8_configuration: Specify a new value for the setting if using Apache Kafka® 2.8.

   • configSpec.kafka.kafkaConfig_3.<setting_for_Apache Kafka®_3.x_configuration: Specify a new value for the setting if using Apache Kafka® 3.x.

   You can request the cluster ID with the list of clusters in the folder.

3. View the server response to make sure the request was successful.

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService/Update call and send the following request, e.g., via gRPCurl:

   Warning

   The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the update_mask parameter as an array of paths[] strings.

   Format for listing settings

   "update_mask": {
       "paths": [
           "<setting_1>",
           "<setting_2>",
           ...
           "<setting_N>"
       ]
   }
   

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
             "cluster_id": "<cluster_ID>",
             "update_mask": {
               "paths": [
                 "config_spec.kafka.kafka_config_2_8.<setting_1_for_Apache Kafka®_2.8_configuration>",
                 ...,
                 "config_spec.kafka.kafka_config_2_8.<setting_N_for_Apache Kafka®_2.8_configuration>",
                 "config_spec.kafka.kafka_config_3.<setting_1_for_Apache Kafka®_3.x_configuration>",
                 ...,
                 "config_spec.kafka.kafka_config_3.<setting_N_for_Apache Kafka®_3.x_configuration>"
               ]
             }
             "config_spec": {
               "kafka": {
                 "kafka_config_2_8": {
                   "<setting_1_for_Apache Kafka®_2.8_configuration>": "<setting_value>",
                   "<setting_2_for_Apache Kafka®_2.8_configuration>": "<setting_value>",
                   ...,
                   "<setting_N_for_Apache Kafka®_2.8_configuration>": "<setting_value>"
                 },
                 "kafka_config_3": {
                   "<setting_1_for_Apache Kafka®_3.x_configuration>": "<setting_value>",
                   "<setting_2_for_Apache Kafka®_3.x_configuration>": "<setting_value>",
                   ...,
                   "<setting_N_for_Apache Kafka®_3.x_configuration>": "<setting_value>"
                 }
               }
             }
           }' \
       mdb.api.cloud.yandex.net:443 \
       yandex.cloud.mdb.kafka.v1.ClusterService.Update
   

   Where:

   • update_mask: List of parameters to update as an array of paths[] strings.

     Specify the relevant parameters:

     • config_spec.kafka.kafka_config_2_8.<setting_for_Apache Kafka®_2.8_configuration>: For Apache Kafka® 2.8.
     • config_spec.kafka.kafka_config_3.<setting_for_Apache Kafka®_3.x_configuration>: For Apache Kafka® 3.x.

   • config_spec.kafka.kafka_config_2_8.<setting_for_Apache Kafka®_2.8_configuration: Specify a new value for the setting if using Apache Kafka® 2.8.

   • config_spec.kafka.kafka_config_3.<setting_for_Apache Kafka®_3.x_configuration: Specify a new value for the setting if using Apache Kafka® 3.x.

   You can request the cluster ID with the list of clusters in the folder.

4. View the server response to make sure the request was successful.

1. Open the current Terraform configuration file with an infrastructure plan.

   For more information about creating this file, see Creating clusters.

2. In the Managed Service for Apache Kafka® cluster description, edit or add the folder_id parameter value:

   resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
     ...
     folder_id = "<destination_folder_ID>"
   }
   

3. Make sure the settings are correct.

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

4. Confirm updating the resources.

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

For more information, see the Terraform provider documentation.

resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Use the Cluster.move method and send the following request, e.g., via cURL:

   curl \
       --request POST \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --header "Content-Type: application/json" \
       -url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>:move' \
       --data '{
                 "destinationFolderId": "<folder_ID>"
               }'
   

   You can request the cluster ID with the list of clusters in the folder.

3. View the server response to make sure the request was successful.

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService/Move call and send the following request, e.g., via gRPCurl:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
             "cluster_id": "<cluster_ID>",
             "destination_folder_id": "<folder_ID>"
           }' \
       mdb.api.cloud.yandex.net:443 \
       yandex.cloud.mdb.kafka.v1.ClusterService.Move
   

   You can request the cluster ID with the list of clusters in the folder.

4. View the server response to make sure the request was successful.