Managed Service for Apache Kafka® API, gRPC: ConnectorService
- Calls ConnectorService
- Get
- List
- Create
- CreateConnectorRequest
- ConnectorSpec
- ConnectorConfigMirrorMakerSpec
- ClusterConnectionSpec
- ThisClusterSpec
- ExternalClusterConnectionSpec
- ConnectorConfigS3SinkSpec
- S3ConnectionSpec
- ExternalS3StorageSpec
- Operation
- CreateConnectorMetadata
- Connector
- ConnectorConfigMirrorMaker
- ClusterConnection
- ThisCluster
- ExternalClusterConnection
- ConnectorConfigS3Sink
- S3Connection
- ExternalS3Storage
- Update
- UpdateConnectorRequest
- UpdateConnectorSpec
- ConnectorConfigMirrorMakerSpec
- ClusterConnectionSpec
- ThisClusterSpec
- ExternalClusterConnectionSpec
- UpdateConnectorConfigS3SinkSpec
- S3ConnectionSpec
- ExternalS3StorageSpec
- Operation
- UpdateConnectorMetadata
- Connector
- ConnectorConfigMirrorMaker
- ClusterConnection
- ThisCluster
- ExternalClusterConnection
- ConnectorConfigS3Sink
- S3Connection
- ExternalS3Storage
- Delete
- Resume
- Pause
A set of methods for managing Apache Kafka® connectors.
Call | Description |
---|---|
Get | Returns information about an Apache Kafka® connector. |
List | Retrieves the list of Apache Kafka® connectors in a cluster. |
Create | Creates a new Apache Kafka® connector in a cluster. |
Update | Updates an Apache Kafka® connector. |
Delete | Deletes an Apache Kafka® connector. |
Resume | Resumes an Apache Kafka® connector. |
Pause | Pauses an Apache Kafka® connector. |
Calls ConnectorService
Get
Returns information about an Apache Kafka® connector.
rpc Get (GetConnectorRequest) returns (Connector)
GetConnectorRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the Apache Kafka® cluster the connector belongs to. To get this ID, make a ClusterService.List request. The maximum string length in characters is 50. |
connector_name | string Required. Name of the Apache Kafka® connector to return information about. To get this name, make a ConnectorService.List request. The maximum string length in characters is 256. Value must match the regular expression [a-zA-Z0-9_-]* . |
Connector
Field | Description |
---|---|
name | string Name of the connector. |
tasks_max | google.protobuf.Int64Value Maximum number of connector tasks. Default value is the number of brokers. |
properties | map<string,string> A set of properties passed to Managed Service for Apache Kafka® with the connector configuration. Example: sync.topics.config.enabled: true . |
health | enum Health Connector health.
|
status | enum Status Current status of the connector.
|
cluster_id | string ID of the Apache Kafka® cluster that the connector belongs to. |
connector_config | oneof: connector_config_mirrormaker or connector_config_s3_sink Additional settings for the connector. |
connector_config_mirrormaker | ConnectorConfigMirrorMaker Configuration of the MirrorMaker connector. |
connector_config_s3_sink | ConnectorConfigS3Sink Configuration of S3-Sink connector. |
ConnectorConfigMirrorMaker
Field | Description |
---|---|
source_cluster | ClusterConnection Source cluster connection configuration. |
target_cluster | ClusterConnection Target cluster connection configuration. |
topics | string List of Kafka topics, separated by , . |
replication_factor | google.protobuf.Int64Value Replication factor for automatically created topics. |
ClusterConnection
Field | Description |
---|---|
alias | string Alias of cluster connection configuration. Examples: source , target . |
cluster_connection | oneof: this_cluster or external_cluster Type of connection to Apache Kafka® cluster. |
this_cluster | ThisCluster Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. |
external_cluster | ExternalClusterConnection Configuration of connection to an external cluster with all the necessary credentials. |
ThisCluster
Empty.
ExternalClusterConnection
Field | Description |
---|---|
bootstrap_servers | string List of bootstrap servers of the cluster, separated by , . |
sasl_username | string SASL username to use for connection to the cluster. |
sasl_mechanism | string SASL mechanism to use for connection to the cluster. |
security_protocol | string Security protocol to use for connection to the cluster. |
ConnectorConfigS3Sink
Field | Description |
---|---|
topics | string List of Kafka topics, separated by ','. |
file_compression_type | string The compression type used for files put on GCS. The supported values are: gzip , snappy , zstd , none . Optional, the default is none . |
file_max_records | google.protobuf.Int64Value Max records per file. |
s3_connection | S3Connection Credentials for connecting to S3 storage. |
S3Connection
Field | Description |
---|---|
bucket_name | string |
storage | oneof: external_s3 |
external_s3 | ExternalS3Storage |
ExternalS3Storage
Field | Description |
---|---|
access_key_id | string |
endpoint | string |
region | string Default is 'us-east-1' |
List
Retrieves the list of Apache Kafka® connectors in a cluster.
rpc List (ListConnectorsRequest) returns (ListConnectorsResponse)
ListConnectorsRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the Apache Kafka® cluster to list connectors in. To get this ID, make a ClusterService.List request. The maximum string length in characters is 50. |
page_size | int64 The maximum number of results per page to return. If the number of available results is larger than page_size , the API returns a ListConnectorsResponse.next_page_token that can be used to get the next page of results in the subsequent ConnectorService.List requests. The maximum value is 1000. |
page_token | string Page token that can be used to iterate through multiple pages of results. To get the next page of results, set page_token to the ListConnectorsResponse.next_page_token returned by the previous ConnectorService.List request. The maximum string length in characters is 100. |
ListConnectorsResponse
Field | Description |
---|---|
connectors[] | Connector List of Apache Kafka® Connectors. |
next_page_token | string The token that can be used to get the next page of results. If the number of results is larger than ListConnectorsRequest.page_size, use the next_page_token as the value for the ListConnectorsRequest.page_token in the subsequent ConnectorService.List request to iterate through multiple pages of results. |
Connector
Field | Description |
---|---|
name | string Name of the connector. |
tasks_max | google.protobuf.Int64Value Maximum number of connector tasks. Default value is the number of brokers. |
properties | map<string,string> A set of properties passed to Managed Service for Apache Kafka® with the connector configuration. Example: sync.topics.config.enabled: true . |
health | enum Health Connector health.
|
status | enum Status Current status of the connector.
|
cluster_id | string ID of the Apache Kafka® cluster that the connector belongs to. |
connector_config | oneof: connector_config_mirrormaker or connector_config_s3_sink Additional settings for the connector. |
connector_config_mirrormaker | ConnectorConfigMirrorMaker Configuration of the MirrorMaker connector. |
connector_config_s3_sink | ConnectorConfigS3Sink Configuration of S3-Sink connector. |
ConnectorConfigMirrorMaker
Field | Description |
---|---|
source_cluster | ClusterConnection Source cluster connection configuration. |
target_cluster | ClusterConnection Target cluster connection configuration. |
topics | string List of Kafka topics, separated by , . |
replication_factor | google.protobuf.Int64Value Replication factor for automatically created topics. |
ClusterConnection
Field | Description |
---|---|
alias | string Alias of cluster connection configuration. Examples: source , target . |
cluster_connection | oneof: this_cluster or external_cluster Type of connection to Apache Kafka® cluster. |
this_cluster | ThisCluster Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. |
external_cluster | ExternalClusterConnection Configuration of connection to an external cluster with all the necessary credentials. |
ThisCluster
Empty.
ExternalClusterConnection
Field | Description |
---|---|
bootstrap_servers | string List of bootstrap servers of the cluster, separated by , . |
sasl_username | string SASL username to use for connection to the cluster. |
sasl_mechanism | string SASL mechanism to use for connection to the cluster. |
security_protocol | string Security protocol to use for connection to the cluster. |
ConnectorConfigS3Sink
Field | Description |
---|---|
topics | string List of Kafka topics, separated by ','. |
file_compression_type | string The compression type used for files put on GCS. The supported values are: gzip , snappy , zstd , none . Optional, the default is none . |
file_max_records | google.protobuf.Int64Value Max records per file. |
s3_connection | S3Connection Credentials for connecting to S3 storage. |
S3Connection
Field | Description |
---|---|
bucket_name | string |
storage | oneof: external_s3 |
external_s3 | ExternalS3Storage |
ExternalS3Storage
Field | Description |
---|---|
access_key_id | string |
endpoint | string |
region | string Default is 'us-east-1' |
Create
Creates a new Apache Kafka® connector in a cluster.
rpc Create (CreateConnectorRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:CreateConnectorMetadata
Operation.response:Connector
CreateConnectorRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the Apache Kafka® cluster to create the connector in. To get this ID, make a ClusterService.List request. The maximum string length in characters is 50. |
connector_spec | ConnectorSpec Required. Configuration of the connector to create. |
ConnectorSpec
Field | Description |
---|---|
name | string Name of the connector. |
tasks_max | google.protobuf.Int64Value Maximum number of connector tasks. Default value is the number of brokers. |
properties | map<string,string> A set of properties passed to Managed Service for Apache Kafka® with the connector configuration. Example: sync.topics.config.enabled: true . |
connector_config | oneof: connector_config_mirrormaker or connector_config_s3_sink Additional settings for the connector. |
connector_config_mirrormaker | ConnectorConfigMirrorMakerSpec Configuration of the MirrorMaker connector. |
connector_config_s3_sink | ConnectorConfigS3SinkSpec Configuration of S3-Sink connector. |
ConnectorConfigMirrorMakerSpec
Field | Description |
---|---|
source_cluster | ClusterConnectionSpec Source cluster configuration for the MirrorMaker connector. |
target_cluster | ClusterConnectionSpec Target cluster configuration for the MirrorMaker connector. |
topics | string List of Kafka topics, separated by , . |
replication_factor | google.protobuf.Int64Value Replication factor for automatically created topics. |
ClusterConnectionSpec
Field | Description |
---|---|
alias | string Alias of cluster connection configuration. Examples: source , target . |
cluster_connection | oneof: this_cluster or external_cluster Type of connection to Apache Kafka® cluster. |
this_cluster | ThisClusterSpec Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. |
external_cluster | ExternalClusterConnectionSpec Configuration of connection to an external cluster with all the necessary credentials. |
ThisClusterSpec
Empty.
ExternalClusterConnectionSpec
Field | Description |
---|---|
bootstrap_servers | string List of bootstrap servers of the cluster, separated by , . |
sasl_username | string SASL username to use for connection to the cluster. |
sasl_password | string SASL password to use for connection to the cluster. |
sasl_mechanism | string SASL mechanism to use for connection to the cluster. |
security_protocol | string Security protocol to use for connection to the cluster. |
ssl_truststore_certificates | string CA in PEM format to connect to external cluster. Lines of certificate separated by '\n' symbol. |
ConnectorConfigS3SinkSpec
Field | Description |
---|---|
topics | string List of Kafka topics, separated by ','. |
file_compression_type | string The compression type used for files put on GCS. The supported values are: gzip , snappy , zstd , none . Optional, the default is none . |
file_max_records | google.protobuf.Int64Value Max records per file. |
s3_connection | S3ConnectionSpec Credentials for connecting to S3 storage. |
S3ConnectionSpec
Field | Description |
---|---|
bucket_name | string |
storage | oneof: external_s3 |
external_s3 | ExternalS3StorageSpec |
ExternalS3StorageSpec
Field | Description |
---|---|
access_key_id | string |
secret_access_key | string |
endpoint | string |
region | string Default is 'us-east-1'. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
CreateConnectorMetadata
Field | Description |
---|---|
cluster_id | string ID of the Apache Kafka® cluster the connector is being created in. |
connector_name | string Required. Name of the Apache Kafka® connector that is being created. The maximum string length in characters is 256. Value must match the regular expression [a-zA-Z0-9_-]* . |
Connector
Field | Description |
---|---|
name | string Name of the connector. |
tasks_max | google.protobuf.Int64Value Maximum number of connector tasks. Default value is the number of brokers. |
properties | map<string,string> A set of properties passed to Managed Service for Apache Kafka® with the connector configuration. Example: sync.topics.config.enabled: true . |
health | enum Health Connector health.
|
status | enum Status Current status of the connector.
|
cluster_id | string ID of the Apache Kafka® cluster that the connector belongs to. |
connector_config | oneof: connector_config_mirrormaker or connector_config_s3_sink Additional settings for the connector. |
connector_config_mirrormaker | ConnectorConfigMirrorMaker Configuration of the MirrorMaker connector. |
connector_config_s3_sink | ConnectorConfigS3Sink Configuration of S3-Sink connector. |
ConnectorConfigMirrorMaker
Field | Description |
---|---|
source_cluster | ClusterConnection Source cluster connection configuration. |
target_cluster | ClusterConnection Target cluster connection configuration. |
topics | string List of Kafka topics, separated by , . |
replication_factor | google.protobuf.Int64Value Replication factor for automatically created topics. |
ClusterConnection
Field | Description |
---|---|
alias | string Alias of cluster connection configuration. Examples: source , target . |
cluster_connection | oneof: this_cluster or external_cluster Type of connection to Apache Kafka® cluster. |
this_cluster | ThisCluster Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. |
external_cluster | ExternalClusterConnection Configuration of connection to an external cluster with all the necessary credentials. |
ThisCluster
Empty.
ExternalClusterConnection
Field | Description |
---|---|
bootstrap_servers | string List of bootstrap servers of the cluster, separated by , . |
sasl_username | string SASL username to use for connection to the cluster. |
sasl_mechanism | string SASL mechanism to use for connection to the cluster. |
security_protocol | string Security protocol to use for connection to the cluster. |
ConnectorConfigS3Sink
Field | Description |
---|---|
topics | string List of Kafka topics, separated by ','. |
file_compression_type | string The compression type used for files put on GCS. The supported values are: gzip , snappy , zstd , none . Optional, the default is none . |
file_max_records | google.protobuf.Int64Value Max records per file. |
s3_connection | S3Connection Credentials for connecting to S3 storage. |
S3Connection
Field | Description |
---|---|
bucket_name | string |
storage | oneof: external_s3 |
external_s3 | ExternalS3Storage |
ExternalS3Storage
Field | Description |
---|---|
access_key_id | string |
endpoint | string |
region | string Default is 'us-east-1' |
Update
Updates an Apache Kafka® connector.
rpc Update (UpdateConnectorRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:UpdateConnectorMetadata
Operation.response:Connector
UpdateConnectorRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the Apache Kafka® cluster to update the connector in. To get this ID, make a ClusterService.List request. The maximum string length in characters is 50. |
connector_name | string Required. Name of the connector to update. To get this name, make a ConnectorService.List request. The maximum string length in characters is 256. Value must match the regular expression [a-zA-Z0-9_-]* . |
update_mask | google.protobuf.FieldMask Field mask that specifies which settings of the connector should be updated. |
connector_spec | UpdateConnectorSpec Required. Configuration of the connector to update. |
UpdateConnectorSpec
Field | Description |
---|---|
tasks_max | google.protobuf.Int64Value Maximum number of connector tasks to update. |
properties | map<string,string> A set of new or changed properties to update for the connector. They are passed with the connector configuration to Managed Service for Apache Kafka®. Example: sync.topics.config.enabled: false . |
connector_config | oneof: connector_config_mirrormaker or connector_config_s3_sink Updated configuration for the connector. |
connector_config_mirrormaker | ConnectorConfigMirrorMakerSpec Configuration of the MirrorMaker connector. |
connector_config_s3_sink | UpdateConnectorConfigS3SinkSpec Update specification for S3-Sink Connector. |
ConnectorConfigMirrorMakerSpec
Field | Description |
---|---|
source_cluster | ClusterConnectionSpec Source cluster configuration for the MirrorMaker connector. |
target_cluster | ClusterConnectionSpec Target cluster configuration for the MirrorMaker connector. |
topics | string List of Kafka topics, separated by , . |
replication_factor | google.protobuf.Int64Value Replication factor for automatically created topics. |
ClusterConnectionSpec
Field | Description |
---|---|
alias | string Alias of cluster connection configuration. Examples: source , target . |
cluster_connection | oneof: this_cluster or external_cluster Type of connection to Apache Kafka® cluster. |
this_cluster | ThisClusterSpec Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. |
external_cluster | ExternalClusterConnectionSpec Configuration of connection to an external cluster with all the necessary credentials. |
ThisClusterSpec
Empty.
ExternalClusterConnectionSpec
Field | Description |
---|---|
bootstrap_servers | string List of bootstrap servers of the cluster, separated by , . |
sasl_username | string SASL username to use for connection to the cluster. |
sasl_password | string SASL password to use for connection to the cluster. |
sasl_mechanism | string SASL mechanism to use for connection to the cluster. |
security_protocol | string Security protocol to use for connection to the cluster. |
ssl_truststore_certificates | string CA in PEM format to connect to external cluster. Lines of certificate separated by '\n' symbol. |
UpdateConnectorConfigS3SinkSpec
Field | Description |
---|---|
topics | string List of Kafka topics, separated by ','. |
file_max_records | google.protobuf.Int64Value Max records per file. |
s3_connection | S3ConnectionSpec Credentials for connecting to S3 storage. |
S3ConnectionSpec
Field | Description |
---|---|
bucket_name | string |
storage | oneof: external_s3 |
external_s3 | ExternalS3StorageSpec |
ExternalS3StorageSpec
Field | Description |
---|---|
access_key_id | string |
secret_access_key | string |
endpoint | string |
region | string Default is 'us-east-1'. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
UpdateConnectorMetadata
Field | Description |
---|---|
cluster_id | string Required. ID of the Apache Kafka® cluster the connector is being updated in. The maximum string length in characters is 50. |
connector_name | string Required. Name of the Apache Kafka® connector that is being updated. The maximum string length in characters is 256. Value must match the regular expression [a-zA-Z0-9_-]* . |
Connector
Field | Description |
---|---|
name | string Name of the connector. |
tasks_max | google.protobuf.Int64Value Maximum number of connector tasks. Default value is the number of brokers. |
properties | map<string,string> A set of properties passed to Managed Service for Apache Kafka® with the connector configuration. Example: sync.topics.config.enabled: true . |
health | enum Health Connector health.
|
status | enum Status Current status of the connector.
|
cluster_id | string ID of the Apache Kafka® cluster that the connector belongs to. |
connector_config | oneof: connector_config_mirrormaker or connector_config_s3_sink Additional settings for the connector. |
connector_config_mirrormaker | ConnectorConfigMirrorMaker Configuration of the MirrorMaker connector. |
connector_config_s3_sink | ConnectorConfigS3Sink Configuration of S3-Sink connector. |
ConnectorConfigMirrorMaker
Field | Description |
---|---|
source_cluster | ClusterConnection Source cluster connection configuration. |
target_cluster | ClusterConnection Target cluster connection configuration. |
topics | string List of Kafka topics, separated by , . |
replication_factor | google.protobuf.Int64Value Replication factor for automatically created topics. |
ClusterConnection
Field | Description |
---|---|
alias | string Alias of cluster connection configuration. Examples: source , target . |
cluster_connection | oneof: this_cluster or external_cluster Type of connection to Apache Kafka® cluster. |
this_cluster | ThisCluster Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. |
external_cluster | ExternalClusterConnection Configuration of connection to an external cluster with all the necessary credentials. |
ThisCluster
Empty.
ExternalClusterConnection
Field | Description |
---|---|
bootstrap_servers | string List of bootstrap servers of the cluster, separated by , . |
sasl_username | string SASL username to use for connection to the cluster. |
sasl_mechanism | string SASL mechanism to use for connection to the cluster. |
security_protocol | string Security protocol to use for connection to the cluster. |
ConnectorConfigS3Sink
Field | Description |
---|---|
topics | string List of Kafka topics, separated by ','. |
file_compression_type | string The compression type used for files put on GCS. The supported values are: gzip , snappy , zstd , none . Optional, the default is none . |
file_max_records | google.protobuf.Int64Value Max records per file. |
s3_connection | S3Connection Credentials for connecting to S3 storage. |
S3Connection
Field | Description |
---|---|
bucket_name | string |
storage | oneof: external_s3 |
external_s3 | ExternalS3Storage |
ExternalS3Storage
Field | Description |
---|---|
access_key_id | string |
endpoint | string |
region | string Default is 'us-east-1' |
Delete
Deletes an Apache Kafka® connector.
rpc Delete (DeleteConnectorRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:DeleteConnectorMetadata
Operation.response:google.protobuf.Empty
DeleteConnectorRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the Apache Kafka® cluster to delete the connector from. To get this ID, make a ClusterService.List request. The maximum string length in characters is 50. |
connector_name | string Required. Name of the connector to delete. To get this name, make a ConnectorService.List request. The maximum string length in characters is 256. Value must match the regular expression [a-zA-Z0-9_-]* . |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
DeleteConnectorMetadata
Field | Description |
---|---|
cluster_id | string ID of the Apache Kafka® cluster the connector is being deleted from. |
connector_name | string Name of the Apache Kafka® connector that is being deleted. |
Resume
Resumes an Apache Kafka® connector.
rpc Resume (ResumeConnectorRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:ResumeConnectorMetadata
Operation.response:Connector
ResumeConnectorRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the Apache Kafka® cluster to resume the connector in. To get this ID, make a ClusterService.List request. The maximum string length in characters is 50. |
connector_name | string Required. Name of the Apache Kafka® connector to resume. To get this name, make a ConnectorService.List request. The maximum string length in characters is 256. Value must match the regular expression [a-zA-Z0-9_-]* . |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
ResumeConnectorMetadata
Field | Description |
---|---|
cluster_id | string ID of the Apache Kafka® cluster the connector is being resumed in. |
connector_name | string Required. Name of the Apache Kafka® connector that is beign resumed. The maximum string length in characters is 256. Value must match the regular expression [a-zA-Z0-9_-]* . |
Connector
Field | Description |
---|---|
name | string Name of the connector. |
tasks_max | google.protobuf.Int64Value Maximum number of connector tasks. Default value is the number of brokers. |
properties | map<string,string> A set of properties passed to Managed Service for Apache Kafka® with the connector configuration. Example: sync.topics.config.enabled: true . |
health | enum Health Connector health.
|
status | enum Status Current status of the connector.
|
cluster_id | string ID of the Apache Kafka® cluster that the connector belongs to. |
connector_config | oneof: connector_config_mirrormaker or connector_config_s3_sink Additional settings for the connector. |
connector_config_mirrormaker | ConnectorConfigMirrorMaker Configuration of the MirrorMaker connector. |
connector_config_s3_sink | ConnectorConfigS3Sink Configuration of S3-Sink connector. |
ConnectorConfigMirrorMaker
Field | Description |
---|---|
source_cluster | ClusterConnection Source cluster connection configuration. |
target_cluster | ClusterConnection Target cluster connection configuration. |
topics | string List of Kafka topics, separated by , . |
replication_factor | google.protobuf.Int64Value Replication factor for automatically created topics. |
ClusterConnection
Field | Description |
---|---|
alias | string Alias of cluster connection configuration. Examples: source , target . |
cluster_connection | oneof: this_cluster or external_cluster Type of connection to Apache Kafka® cluster. |
this_cluster | ThisCluster Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. |
external_cluster | ExternalClusterConnection Configuration of connection to an external cluster with all the necessary credentials. |
ThisCluster
Empty.
ExternalClusterConnection
Field | Description |
---|---|
bootstrap_servers | string List of bootstrap servers of the cluster, separated by , . |
sasl_username | string SASL username to use for connection to the cluster. |
sasl_mechanism | string SASL mechanism to use for connection to the cluster. |
security_protocol | string Security protocol to use for connection to the cluster. |
ConnectorConfigS3Sink
Field | Description |
---|---|
topics | string List of Kafka topics, separated by ','. |
file_compression_type | string The compression type used for files put on GCS. The supported values are: gzip , snappy , zstd , none . Optional, the default is none . |
file_max_records | google.protobuf.Int64Value Max records per file. |
s3_connection | S3Connection Credentials for connecting to S3 storage. |
S3Connection
Field | Description |
---|---|
bucket_name | string |
storage | oneof: external_s3 |
external_s3 | ExternalS3Storage |
ExternalS3Storage
Field | Description |
---|---|
access_key_id | string |
endpoint | string |
region | string Default is 'us-east-1' |
Pause
Pauses an Apache Kafka® connector.
rpc Pause (PauseConnectorRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:PauseConnectorMetadata
Operation.response:Connector
PauseConnectorRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the Apache Kafka® cluster to pause the connector in. To get this ID, make a ClusterService.List request. The maximum string length in characters is 50. |
connector_name | string Required. Name of the Apache Kafka® connector to pause. To get this name, make a ConnectorService.List request. The maximum string length in characters is 256. Value must match the regular expression [a-zA-Z0-9_-]* . |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
PauseConnectorMetadata
Field | Description |
---|---|
cluster_id | string ID of the Apache Kafka® cluster the connector is being paused in. |
connector_name | string Required. Name of the Apache Kafka® connector that is being paused. The maximum string length in characters is 256. Value must match the regular expression [a-zA-Z0-9_-]* . |
Connector
Field | Description |
---|---|
name | string Name of the connector. |
tasks_max | google.protobuf.Int64Value Maximum number of connector tasks. Default value is the number of brokers. |
properties | map<string,string> A set of properties passed to Managed Service for Apache Kafka® with the connector configuration. Example: sync.topics.config.enabled: true . |
health | enum Health Connector health.
|
status | enum Status Current status of the connector.
|
cluster_id | string ID of the Apache Kafka® cluster that the connector belongs to. |
connector_config | oneof: connector_config_mirrormaker or connector_config_s3_sink Additional settings for the connector. |
connector_config_mirrormaker | ConnectorConfigMirrorMaker Configuration of the MirrorMaker connector. |
connector_config_s3_sink | ConnectorConfigS3Sink Configuration of S3-Sink connector. |
ConnectorConfigMirrorMaker
Field | Description |
---|---|
source_cluster | ClusterConnection Source cluster connection configuration. |
target_cluster | ClusterConnection Target cluster connection configuration. |
topics | string List of Kafka topics, separated by , . |
replication_factor | google.protobuf.Int64Value Replication factor for automatically created topics. |
ClusterConnection
Field | Description |
---|---|
alias | string Alias of cluster connection configuration. Examples: source , target . |
cluster_connection | oneof: this_cluster or external_cluster Type of connection to Apache Kafka® cluster. |
this_cluster | ThisCluster Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. |
external_cluster | ExternalClusterConnection Configuration of connection to an external cluster with all the necessary credentials. |
ThisCluster
Empty.
ExternalClusterConnection
Field | Description |
---|---|
bootstrap_servers | string List of bootstrap servers of the cluster, separated by , . |
sasl_username | string SASL username to use for connection to the cluster. |
sasl_mechanism | string SASL mechanism to use for connection to the cluster. |
security_protocol | string Security protocol to use for connection to the cluster. |
ConnectorConfigS3Sink
Field | Description |
---|---|
topics | string List of Kafka topics, separated by ','. |
file_compression_type | string The compression type used for files put on GCS. The supported values are: gzip , snappy , zstd , none . Optional, the default is none . |
file_max_records | google.protobuf.Int64Value Max records per file. |
s3_connection | S3Connection Credentials for connecting to S3 storage. |
S3Connection
Field | Description |
---|---|
bucket_name | string |
storage | oneof: external_s3 |
external_s3 | ExternalS3Storage |
ExternalS3Storage
Field | Description |
---|---|
access_key_id | string |
endpoint | string |
region | string Default is 'us-east-1' |