Managing Apache Kafka® users
Users in Apache Kafka®:
-
Keep the access permissions of data producers and consumers separate. Learn more about the permissions you get with each role.
A producer or consumer can only access topics that are allowed for their users. You can use the same user for multiple producers or consumers: the former get the rights to write data to certain topics and the latter get the read rights.
-
Manage topics. For more information, see Topics and partitions.
After creating an Apache Kafka® cluster, you can:
- Get a list of users in a cluster
- Create a user
- Change user settings:
- Import a user to Terraform
- Delete a user
Getting a list of users in a cluster
- In the management console
, go to the relevant folder. - In the list of services, select Managed Service for Kafka.
- Click the cluster name and go to the Users tab.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
-
To get a list of users, run the following command:
yc managed-kafka user list --cluster-name <cluster_name> -
To get detailed information for a specific user, run this command:
yc managed-kafka user get <user_name> --cluster-name <cluster_name>
To find out the cluster name, get a list of clusters in the folder.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the User.list method and send the following request, e.g., via cURL
:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>/users'You can request the cluster ID with the list of clusters in the folder.
-
View the server response to make sure the request was successful.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the UserService/List call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/user_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.UserService.ListYou can request the cluster ID with the list of clusters in the folder.
-
View the server response to make sure the request was successful.
Creating a user
Note
Use the CLI, API, or Terraform to create an admin user.
To create a user for a producer or consumer in a cluster:
-
In the management console
, go to the relevant folder. -
In the list of services, select Managed Service for Kafka.
-
Click the cluster name and go to the Users tab.
-
Click Create user.
-
Enter your username and password.
Note
The username may contain Latin letters, numbers, hyphens, and underscores but must begin with a letter or underscore.
The password must be from 8 to 128 characters long.
-
Grant access permissions for the relevant topics.
-
Click Create.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To create a user:
-
View the description of the CLI command to create users:
yc managed-kafka user create --help -
Create a user with the
producerrole for the producer or theconsumerrole for the consumer and grant access permissions for the relevant topics:yc managed-kafka user create <user_name> \ --cluster-name <cluster_name> \ --password <password> \ --permission topic=<topic_name>,role=<user's_role>,allow_host=<allowed_IP_address_1>,allow_host=<allowed_IP_address_2>,...,allow_host=<allowed_IP_address_N>The
--permissionparameter must include at least one topic-role pair, where:topic: Topic name.role: User’s role,producer,consumer,admin, ortopic_admin. Learn more about the permissions you get with each role.allow_host: (Optional) IP address the user is allowed to access the topic from. To specify multiple addresses, add the required number ofallow_hostoptions separated by commas.
To create an admin user to manage cluster topics:
-
View the description of the CLI command to create s user:
yc managed-kafka user create --help -
Create a user with the
adminrole valid for all (*) cluster topics:yc managed-kafka user create <user_name> \ --cluster-name <cluster_name> \ --password <password> \ --permission topic=*,role=admin,allow_host=<allowed_IP_address_1>,allow_host=<allowed_IP_address_2>,...,allow_host=<allowed_IP_address_N> -
Create a user with the
topic_adminrole for all cluster topics prefixed withpref:yc managed-kafka user create <user_name> \ --cluster-name <cluster_name> \ --password <password> \ --permission topic=pref*,role=topic_admin
Note
The username may contain Latin letters, numbers, hyphens, and underscores but must begin with a letter or underscore.
The password must be from 8 to 128 characters long.
To find out the cluster name, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
Add the
yandex_mdb_kafka_userresource:resource "yandex_mdb_kafka_user" "<user_name>" { cluster_id = "<cluster_ID>" name = "<user_name>" password = "<password>" ... }Note
The username may contain Latin letters, numbers, hyphens, and underscores but must begin with a letter or underscore.
The password must be from 8 to 128 characters long.
-
Grant access permissions for the relevant topics.
-
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the User.create method and send the following request, e.g., via cURL
:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>/users' \ --data '{ "userSpec": { "name": "<user_name>", "password": "<user_password>", "permissions": [ { "topicName": "<topic_name>", "role": "<level_of_topic_access_permissions>", "allowHosts": [ <list_of_IP_addresses> ] } ] } }'Where
userSpecrepresents the new Apache Kafka® user settings:-
name: Username. -
password: User password.Note
The username may contain Latin letters, numbers, hyphens, and underscores but must begin with a letter or underscore.
The password must be from 8 to 128 characters long.
-
permissions: Topic access permissions. Each array element is for a separate topic and has the following structure:topicName: Topic name or name template:*to allow access to any topics.- Full topic name to allow access to a specific topic. To find out the name, get a list of cluster topics.
<prefix>*to grant access to topics whose names start with the prefix. Let's assume you have topics namedtopic_a1,topic_a2, anda3. If you puttopic*, access will be granted totopic_a1andtopic_a2. To include all the cluster's topics, use the*mask.
role: User’s role,ACCESS_ROLE_CONSUMER,ACCESS_ROLE_PRODUCER,ACCESS_ROLE_TOPIC_ADMIN, orACCESS_ROLE_ADMIN. TheACCESS_ROLE_ADMINrole is only available if all topics are selected (topicName: "*"). Learn more about the permissions you get with each role.allowHosts: (Optional) List of IP addresses the user is allowed to access the topic from.
You can request the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure the request was successful.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the UserService/Create call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/user_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "user_spec": { "name": "<user_name>", "password": "<user_password>", "permissions": [ { "topic_name": "<topic_name>", "role": "<level_of_topic_access_permissions>", "allow_hosts": [ <list_of_IP_addresses> ] } ] } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.UserService.CreateWhere
user_specrepresents the new DB user settings:-
name: Username. -
password: User password.Note
The username may contain Latin letters, numbers, hyphens, and underscores but must begin with a letter or underscore.
The password must be from 8 to 128 characters long.
-
permissions: Topic access permissions. Each array element is for a separate topic and has the following structure:topic_name: Topic name or name template:*to allow access to any topics.- Full topic name to allow access to a specific topic. To find out the name, get a list of cluster topics.
<prefix>*to grant access to topics whose names start with the prefix. Let's assume you have topics namedtopic_a1,topic_a2, anda3. If you specifytopic*, access will be granted totopic_a1andtopic_a2.
role: User’s role,ACCESS_ROLE_CONSUMER,ACCESS_ROLE_PRODUCER,ACCESS_ROLE_TOPIC_ADMIN, orACCESS_ROLE_ADMIN. TheACCESS_ROLE_ADMINrole is only available if all topics are selected (topicName: "*"). Learn more about the permissions you get with each role.allow_hosts: (Optional) List of IP addresses the user is allowed to access the topic from.
You can request the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure the request was successful.
Changing user settings
-
In the management console
, go to the relevant folder. -
In the list of services, select Managed Service for Kafka.
-
Click the cluster name and go to the Users tab.
-
Click
for the appropriate user and select:- Change password to enter another password for the user.
- Configure to grant or revoke topic access permissions.
-
Click Save.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
Using the CLI, you can change a user's password, grant or revoke topic access permissions.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
In this file, locate the
yandex_mdb_kafka_userresource for the user and make the changes.Using Terraform, you can change a user's password, grant or revoke topic access permissions.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the User.update method and send the following request, e.g., using cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>/users/<user_name>' \ --data '{ "updateMask": "permissions", "permissions": [ { "topicName": "<topic_name>", "role": "<level_of_topic_access_permissions>", "allowHosts": [ <list_of_IP_addresses> ] } ] }'Where:
-
updateMask: List of parameters to update as a single string, separated by commas.Here only one parameter is specified:
permissions. -
permissions: New set of access permissions for topics. Each array element is for a separate topic and has the following structure:topicName: Topic name or name template. To find out the name, get a list of cluster topics. To include all the cluster's topics, use the*mask.role: User’s new role,ACCESS_ROLE_CONSUMER,ACCESS_ROLE_PRODUCER,ACCESS_ROLE_TOPIC_ADMIN, orACCESS_ROLE_ADMIN. TheACCESS_ROLE_ADMINrole is only available if all topics are selected (topicName: "*"). Learn more about the permissions you get with each role.allowHosts: (Optional) New list of IP addresses the user is allowed to access the topic from.
You can request the cluster ID with the list of clusters in the folder and the username, with the list of users in the cluster.
-
-
View the server response to make sure the request was successful.
You can also use the User.update method to change a user's password, and the grantPermission and revokePermission methods to grant or revoke topic access permissions.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the UserService/Update call and send the following request, e.g., using gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/user_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "user_name": "<user_name>", "update_mask": { "paths": [ "permissions" ] }, "permissions": [ { "topic_name": "<topic_name>", "role": "<level_of_topic_access_permissions>", "allow_hosts": [ <list_of_IP_addresses> ] } ] }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.UserService.UpdateWhere:
-
update_mask: List of parameters to update as an array ofpaths[]strings.In this case, the array consists of only one string,
permissions. -
permissions: New set of access permissions for topics. Each array element is for a separate topic and has the following structure:topic_name: Topic name or name template. To find out the name, get a list of cluster topics. To include all the cluster's topics, use the*mask.role: User’s new role,ACCESS_ROLE_CONSUMER,ACCESS_ROLE_PRODUCER,ACCESS_ROLE_TOPIC_ADMIN, orACCESS_ROLE_ADMIN. TheACCESS_ROLE_ADMINrole is only available if all topics are selected (topicName: "*"). Learn more about the permissions you get with each role.allow_hosts: (Optional) New list of IP addresses the user is allowed to access the topic from.
You can request the cluster ID with the list of clusters in the folder and the username, with the list of users in the cluster.
-
-
View the server response to make sure the request was successful.
You can also use the UserService/Update call to change a user’s password, and the grantPermission and revokePermission methods to grant or revoke topic access permissions.
Changing a user password
- In the management console
, go to the relevant folder. - In the list of services, select Managed Service for Kafka.
- Click the cluster name and go to the Users tab.
- Click
for the appropriate user and select Change password. - Set a new password and click Edit.
Note
The password must be between 8 and 128 characters.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To change the user password, run this command:
yc managed-kafka user update <user_name> \
--cluster-name <cluster_name> \
--password <new_password>
Note
The password must be between 8 and 128 characters.
To find out the cluster name, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
Locate the user's
yandex_mdb_kafka_userresource in the file. -
Change the value of the
passwordfield:resource "yandex_mdb_kafka_user" "<user_name>" { ... password = "<password>" ... }Note
The password must be between 8 and 128 characters.
-
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the User.update method and send the following request, e.g., using cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>/users/<user_name>' \ --data '{ "updateMask": "password", "password": "<new_user_password>" }'Where:
-
updateMask: List of parameters to update as a single string, separated by commas.Here only one parameter is specified:
password. -
password: New user password.Note
The password must be between 8 and 128 characters.
You can request the cluster ID with the list of clusters in the folder and the username, with the list of users in the cluster.
-
-
View the server response to make sure the request was successful.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the UserService/Update call and send the following request, e.g., using gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/user_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "user_name": "<user_name>", "update_mask": { "paths": [ "password" ] }, "password": "<new_user_password>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.UserService.UpdateWhere:
-
update_mask: List of parameters to update as an array ofpaths[]strings.In this case, the array consists of only one string,
password. -
password: New user password.Note
The password must be between 8 and 128 characters.
You can request the cluster ID with the list of clusters in the folder and the username, with the list of users in the cluster.
-
-
View the server response to make sure the request was successful.
Granting user permissions
Note
Permissions granted to the user for a topic remain even after the topic is deleted. If, after deleting a topic, you don't revoke the permissions, then, when you create a topic with the same name, the user will have access to it even if you don't explicitly assign them new permissions.
-
In the management console
, go to the relevant folder. -
In the list of services, select Managed Service for Kafka.
-
Select the cluster.
-
Navigate to the Users tab.
-
Click
for the user you need to issue topic permissions to and select Configure. -
Click
Add topic. If there is no such button, it means that the user got permissions for all existing cluster topics.If the user does not need permissions to certain topics, you can revoke them.
-
Select the appropriate topic from the drop-down list or enter its name:
-
Specify the following in the Topic field:
*to allow access to any topics.- Full topic name to allow access to a specific topic.
<prefix>*to grant access to topics whose names start with the prefix. Let's assume you have topics namedtopic_a1,topic_a2, anda3. If you specifytopic*, access will be granted totopic_a1andtopic_a2.
-
Click Add topic.
-
-
Click the
icon in the Roles column for the topic you need and select a role:ACCESS_ROLE_CONSUMER: Access to the topic will be allowed to consumers logged in as this user.ACCESS_ROLE_PRODUCER: Access to the topic will be allowed to producers logged in as this user.ACCESS_ROLE_ADMIN: Only available if access to all topics is selected.ACCESS_ROLE_TOPIC_ADMIN: Role with full permissions to manage topics via the Apache Kafka® Admin API .
You can select the
ACCESS_ROLE_CONSUMERandACCESS_ROLE_PRODUCERroles at the same time to make the user suitable for both producers and consumers.Learn more about the permissions you get with each role.
The user also gains access to data schema subjects. The availability of subjects depends on the specified roles and topics. For more information, see Managed Schema Registry subjects.
-
To grant permissions to other topics, repeat the steps.
-
(Optional) If you granted permissions for a topic accidentally, revoke them.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To grant user permissions:
-
Retrieve a list of cluster topics:
yc managed-kafka topic list --cluster-name <cluster_name> -
Grant access permissions for required topics by providing the
--permissionparameters:yc managed-kafka user update <user_name> \ --cluster-name <cluster_name> \ --permission topic=<topic_name>,role=<user's_role>,allow_host=<allowed_IP_address_1>,allow_host=<allowed_IP_address_2>,...,allow_host=<allowed_IP_address_N>The following
--permissionparameters are available:-
topic: Name of the topic to issue permissions for.If the user does not need permissions to certain topics, you can revoke them.
-
role: User’s role,producer,consumer,admin, ortopic_admin. Learn more about the permissions you get with each role.The
adminrole is only available if all topics are selected (topic=*). -
allow_host: (Optional) IP address the user is allowed to access the topic from. To specify multiple addresses, add the required number ofallow_hostoptions separated by commas.
When you update user permissions, existing permissions get revoked and replaced with new ones. This means the command must always include a complete list of permissions to be assigned to the user.
For example, to grant permissions to a user named
test-userin thekafka-clicluster for thetopic2topic with theproducerrole, while keeping the existingtopic1permissions, run this command:yc managed-kafka user update test-user \ --cluster-name kafka-cli \ --permission topic=topic1,role=consumer \ --permission topic=topic2,role=producerAlong with access to the topic, users also gain access to data schema subjects. The availability of subjects depends on the specified roles and topics. For more information, see Managed Schema Registry subjects.
-
To find out the cluster name, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
Locate the user's
yandex_mdb_kafka_clusterresource in the file. -
Add the
permissionsection:resource "yandex_mdb_kafka_user" "<user_name>" { ... permission { topic_name = "<topic>" role = "<user's_role>" allow_hosts = [ <list_of_allowed_IP_addresses> ] } }Where:
-
topic_name: Topic name. Specify the following:*to allow access to any topics.- Full topic name to allow access to a specific topic.
<prefix>*to grant access to topics whose names start with the prefix. Let's assume you have topics namedtopic_a1,topic_a2, anda3. If you specifytopic*, access will be granted totopic_a1andtopic_a2.
-
role: User’s role,ACCESS_ROLE_CONSUMER,ACCESS_ROLE_PRODUCER,ACCESS_ROLE_TOPIC_ADMIN, orACCESS_ROLE_ADMIN. TheACCESS_ROLE_ADMINrole is only available if all topics are selected (topicName: "*"). Learn more about the permissions you get with each role. -
allow_hosts: List of IP addresses the user is allowed to access the topic from.
Along with access to the topic, users also gain access to data schema subjects. The list of available of subjects depends on the roles and topics you specify. For more information, see Subjects in Managed Schema Registry.
-
If the user does not need permissions to certain topics, you can revoke them.
-
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the User.grantPermission method and send the following request, e.g., via cURL
:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>/users/<user_name>:grantPermission' \ --data '{ "permission": [ { "topicName": "<topic_name>", "role": "<user's_role>", "allowHosts": [ <list_of_IP_addresses> ] } ] }'Where:
permission: New access permission for the topic.topicName: Topic name. To find out the name, get a list of cluster topics.role: User’s role,ACCESS_ROLE_CONSUMER,ACCESS_ROLE_PRODUCER,ACCESS_ROLE_TOPIC_ADMIN, orACCESS_ROLE_ADMIN. TheACCESS_ROLE_ADMINrole is only available if all topics are selected (topicName: "*"). Learn more about the permissions you get with each role.allowHosts: (Optional) List of IP addresses the user is allowed to access the topic from.
You can request the cluster ID with the list of clusters in the folder and the username, with the list of users in the cluster.
-
View the server response to make sure the request was successful.
Along with access to the topic, users also gain access to data schema subjects. The availability of subjects depends on the specified roles and topics. For more information, see Managed Schema Registry subjects.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the UserService/GrantPermission call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/user_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "user_name": "<user_name>", "permission": [ { "topic_name": "<topic_name>", "role": "<user's_role>", "allow_hosts": [ <list_of_IP_addresses> ] } ] }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.UserService.GrantPermissionWhere:
permission: New access permission for the topic.topic_name: Topic name or name template. To find out the name, get a list of cluster topics. To include all the cluster's topics, use the*mask.role: User’s role,ACCESS_ROLE_CONSUMER,ACCESS_ROLE_PRODUCER,ACCESS_ROLE_TOPIC_ADMIN, orACCESS_ROLE_ADMIN. TheACCESS_ROLE_ADMINrole is only available if all topics are selected (topicName: "*"). Learn more about the permissions you get with each role.allow_hosts: (Optional) List of IP addresses the user is allowed to access the topic from.
You can request the cluster ID with the list of clusters in the folder and the username, with the list of users in the cluster.
-
View the server response to make sure the request was successful.
Along with access to the topic, users also gain access to data schema subjects. The availability of subjects depends on the specified roles and topics. For more information, see Managed Schema Registry subjects.
Revoking user permissions
If you revoke the ACCESS_ROLE_ADMIN role from the admin user in a cluster, you will no longer be able to manage topics. Do not revoke this role without first granting it to another user.
- In the management console
, go to the relevant folder. - In the list of services, select Managed Service for Kafka.
- Select the cluster.
- Navigate to the Users tab.
- Click
for the appropriate user and select Configure. - Find the appropriate topic in the list of topics.
- Delete the role you no longer need: click the
icon next to its name. To revoke all permissions for a topic, delete it from the list: hover over the topic name and click at the end of the line.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To revoke access permissions for specific topics, provide an updated list of --permission parameters:
yc managed-kafka user update <user_name> \
--cluster-name <cluster_name> \
--permission topic=<topic_name>,role=<user's_role>,allow_host=<allowed_IP_address_1>,allow_host=<allowed_IP_address_2>,...,allow_host=<allowed_IP_address_N>
When you update user permissions, existing permissions get revoked and replaced with new ones. This means the command must always include a complete list of permissions to be assigned to the user.
The --permission parameter must include at least one topic-role pair, where:
topic: Topic name.role: User’s role,producer,consumer,admin, ortopic_admin. Learn more about the permissions you get with each role.allow_host: (Optional) IP address the user is allowed to access the topic from. To specify multiple addresses, add the required number ofallow_hostoptions separated by commas.
To find out the cluster name, get a list of clusters in the folder.
To revoke all the permissions granted to the user, use the console or delete the user.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
Locate the user's
yandex_mdb_kafka_userresource in the file. -
Edit or delete the
permissionsection. -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the User.revokePermission method and send the following request, e.g., via cURL
:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>/users/<user_name>:revokePermission' \ --data '{ "permission": [ { "topicName": "<topic_name>", "role": "<user's_role>", "allowHosts": [ <list_of_IP_addresses> ] } ] }'Where:
permission: Access permission for the topic to be revoked:topicName: Topic name. To find out the name, get a list of cluster topics.role: User’s role to revoke,ACCESS_ROLE_PRODUCER,ACCESS_ROLE_CONSUMER,ACCESS_ROLE_ADMIN, orACCESS_ROLE_TOPIC_ADMIN.allow_hosts: (Optional) List of IP addresses for which the user’s access permissions to the topic will be revoked.
You can request the cluster ID with the list of clusters in the folder and the username, with the list of users in the cluster.
-
View the server response to make sure the request was successful.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the UserService/RevokePermission call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/user_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "user_name": "<user_name>", "permission": [ { "topic_name": "<topic_name>", "role": "<user's_role>", "allow_hosts": [ <list_of_IP_addresses> ] } ] }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.UserService.RevokePermissionWhere:
permission: Access permission for the topic to be revoked:topic_name: Topic name or name template. To find out the name, get a list of cluster topics.role: User’s role to revoke,ACCESS_ROLE_PRODUCER,ACCESS_ROLE_CONSUMER,ACCESS_ROLE_ADMIN, orACCESS_ROLE_TOPIC_ADMIN.allow_hosts: (Optional) List of IP addresses for which the user’s access to the topic will be revoked.
You can request the cluster ID with the list of clusters in the folder and the username, with the list of users in the cluster.
-
View the server response to make sure the request was successful.
Importing users to Terraform
Using import, you can bring the existing cluster users under Terraform management.
-
In the Terraform configuration file, specify the user you want to import:
resource "yandex_mdb_kafka_user" "<user_name>" {} -
Run the following command to import the user:
terraform import yandex_mdb_kafka_user.<user_name> <cluster_ID>:<user_name>To learn more about importing users, see the Terraform provider documentation.
Deleting a user
If you delete the admin user with the ACCESS_ROLE_ADMIN role in a cluster, you will no longer be able to manage topics. To avoid this, assign this role to another user before deleting the admin user.
- In the management console
, go to the relevant folder. - In the list of services, select Managed Service for Kafka.
- Click the cluster name and go to the Users tab.
- Click
for the appropriate user and select Delete. - In the window that opens, click Delete.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To remove a user, run:
yc managed-kafka user delete <user_name> --cluster-name <cluster_name>
To find out the cluster name, get a list of clusters in the folder.
-
Open the current Terraform configuration file that defines your infrastructure.
For more information about creating this file, see Creating clusters.
-
Delete the user's
yandex_mdb_kafka_userresource. -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation.
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the timeouts block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the User.delete method and send the following request, e.g., via cURL
:curl \ --request DELETE \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/<cluster_ID>/users/<user_name>'You can request the cluster ID with the list of clusters in the folder and the username, with the list of users in the cluster.
-
View the server response to make sure the request was successful.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the UserService/Delete call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/kafka/v1/user_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "user_name": "<user_name>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.kafka.v1.UserService.DeleteYou can request the cluster ID with the list of clusters in the folder and the username, with the list of users in the cluster.
-
View the server response to make sure the request was successful.