Managed Service for Apache Kafka® API, gRPC: ConnectorService.Update

Статья создана

Yandex Cloud

Обновлена 17 октября 2024 г.

gRPC request
UpdateConnectorRequest
UpdateConnectorSpec
ConnectorConfigMirrorMakerSpec
ClusterConnectionSpec
ThisClusterSpec
ExternalClusterConnectionSpec
UpdateConnectorConfigS3SinkSpec
S3ConnectionSpec
ExternalS3StorageSpec
operation.Operation
UpdateConnectorMetadata
Connector
ConnectorConfigMirrorMaker
ClusterConnection
ThisCluster
ExternalClusterConnection
ConnectorConfigS3Sink
S3Connection
ExternalS3Storage

Updates an Apache Kafka® connector.

gRPC request

rpc Update (UpdateConnectorRequest) returns (operation.Operation)

UpdateConnectorRequest

{
  "clusterId": "string",
  "connectorName": "string",
  "updateMask": "google.protobuf.FieldMask",
  "connectorSpec": {
    "tasksMax": "google.protobuf.Int64Value",
    "properties": "string",
    // Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`
    "connectorConfigMirrormaker": {
      "sourceCluster": {
        "alias": "string",
        // Includes only one of the fields `thisCluster`, `externalCluster`
        "thisCluster": "ThisClusterSpec",
        "externalCluster": {
          "bootstrapServers": "string",
          "saslUsername": "string",
          "saslPassword": "string",
          "saslMechanism": "string",
          "securityProtocol": "string",
          "sslTruststoreCertificates": "string"
        }
        // end of the list of possible fields
      },
      "targetCluster": {
        "alias": "string",
        // Includes only one of the fields `thisCluster`, `externalCluster`
        "thisCluster": "ThisClusterSpec",
        "externalCluster": {
          "bootstrapServers": "string",
          "saslUsername": "string",
          "saslPassword": "string",
          "saslMechanism": "string",
          "securityProtocol": "string",
          "sslTruststoreCertificates": "string"
        }
        // end of the list of possible fields
      },
      "topics": "string",
      "replicationFactor": "google.protobuf.Int64Value"
    },
    "connectorConfigS3Sink": {
      "topics": "string",
      "fileMaxRecords": "google.protobuf.Int64Value",
      "s3Connection": {
        "bucketName": "string",
        // Includes only one of the fields `externalS3`
        "externalS3": {
          "accessKeyId": "string",
          "secretAccessKey": "string",
          "endpoint": "string",
          "region": "string"
        }
        // end of the list of possible fields
      }
    }
    // end of the list of possible fields
  }
}

Field	Description
clusterId	string Required field. ID of the Apache Kafka® cluster to update the connector in. To get this ID, make a ClusterService.List request.
connectorName	string Required field. Name of the connector to update. To get this name, make a ConnectorService.List request.
updateMask	google.protobuf.FieldMask Field mask that specifies which settings of the connector should be updated.
connectorSpec	UpdateConnectorSpec Required field. Configuration of the connector to update.

UpdateConnectorSpec

Field	Description
tasksMax	google.protobuf.Int64Value Maximum number of connector tasks to update.
properties	string A set of new or changed properties to update for the connector. They are passed with the connector configuration to Managed Service for Apache Kafka®. Example: `sync.topics.config.enabled: false`.
connectorConfigMirrormaker	ConnectorConfigMirrorMakerSpec Configuration of the MirrorMaker connector. Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`. Updated configuration for the connector.
connectorConfigS3Sink	UpdateConnectorConfigS3SinkSpec Update specification for S3-Sink Connector. Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`. Updated configuration for the connector.

ConnectorConfigMirrorMakerSpec

Field	Description
sourceCluster	ClusterConnectionSpec Source cluster configuration for the MirrorMaker connector.
targetCluster	ClusterConnectionSpec Target cluster configuration for the MirrorMaker connector.
topics	string List of Kafka topics, separated by `,`.
replicationFactor	google.protobuf.Int64Value Replication factor for automatically created topics.

ClusterConnectionSpec

Field	Description
alias	string Alias of cluster connection configuration. Examples: `source`, `target`.
thisCluster	ThisClusterSpec Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. Includes only one of the fields `thisCluster`, `externalCluster`. Type of connection to Apache Kafka® cluster.
externalCluster	ExternalClusterConnectionSpec Configuration of connection to an external cluster with all the necessary credentials. Includes only one of the fields `thisCluster`, `externalCluster`. Type of connection to Apache Kafka® cluster.

ThisClusterSpec

Field	Description
Empty

ExternalClusterConnectionSpec

Field	Description
bootstrapServers	string List of bootstrap servers of the cluster, separated by `,`.
saslUsername	string SASL username to use for connection to the cluster.
saslPassword	string SASL password to use for connection to the cluster.
saslMechanism	string SASL mechanism to use for connection to the cluster.
securityProtocol	string Security protocol to use for connection to the cluster.
sslTruststoreCertificates	string CA in PEM format to connect to external cluster. Lines of certificate separated by '\n' symbol.

UpdateConnectorConfigS3SinkSpec

Specification for update Kafka S3-Sink Connector.

Field	Description
topics	string List of Kafka topics, separated by ','.
fileMaxRecords	google.protobuf.Int64Value Max records per file.
s3Connection	S3ConnectionSpec Credentials for connecting to S3 storage.

S3ConnectionSpec

Specification for S3Connection -
settings of connection to AWS-compatible S3 storage, that
are source or target of Kafka S3-connectors.
YC Object Storage is AWS-compatible.

Field

Description

bucketName

string

externalS3

ExternalS3StorageSpec

Includes only one of the fields externalS3.

ExternalS3StorageSpec

Field	Description
accessKeyId	string
secretAccessKey	string
endpoint	string
region	string Default is 'us-east-1'.

operation.Operation

{
  "id": "string",
  "description": "string",
  "createdAt": "google.protobuf.Timestamp",
  "createdBy": "string",
  "modifiedAt": "google.protobuf.Timestamp",
  "done": "bool",
  "metadata": {
    "clusterId": "string",
    "connectorName": "string"
  },
  // Includes only one of the fields `error`, `response`
  "error": "google.rpc.Status",
  "response": {
    "name": "string",
    "tasksMax": "google.protobuf.Int64Value",
    "properties": "string",
    "health": "Health",
    "status": "Status",
    "clusterId": "string",
    // Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`
    "connectorConfigMirrormaker": {
      "sourceCluster": {
        "alias": "string",
        // Includes only one of the fields `thisCluster`, `externalCluster`
        "thisCluster": "ThisCluster",
        "externalCluster": {
          "bootstrapServers": "string",
          "saslUsername": "string",
          "saslMechanism": "string",
          "securityProtocol": "string"
        }
        // end of the list of possible fields
      },
      "targetCluster": {
        "alias": "string",
        // Includes only one of the fields `thisCluster`, `externalCluster`
        "thisCluster": "ThisCluster",
        "externalCluster": {
          "bootstrapServers": "string",
          "saslUsername": "string",
          "saslMechanism": "string",
          "securityProtocol": "string"
        }
        // end of the list of possible fields
      },
      "topics": "string",
      "replicationFactor": "google.protobuf.Int64Value"
    },
    "connectorConfigS3Sink": {
      "topics": "string",
      "fileCompressionType": "string",
      "fileMaxRecords": "google.protobuf.Int64Value",
      "s3Connection": {
        "bucketName": "string",
        // Includes only one of the fields `externalS3`
        "externalS3": {
          "accessKeyId": "string",
          "endpoint": "string",
          "region": "string"
        }
        // end of the list of possible fields
      }
    }
    // end of the list of possible fields
  }
  // end of the list of possible fields
}

An Operation resource. For more information, see Operation.

Field	Description
id	string ID of the operation.
description	string Description of the operation. 0-256 characters long.
createdAt	google.protobuf.Timestamp Creation timestamp.
createdBy	string ID of the user or service account who initiated the operation.
modifiedAt	google.protobuf.Timestamp The time when the Operation resource was last modified.
done	bool If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available.
metadata	UpdateConnectorMetadata Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any.
error	google.rpc.Status The error result of the operation in case of failure or cancellation. Includes only one of the fields `error`, `response`. The operation result. If `done == false` and there was no failure detected, neither `error` nor `response` is set. If `done == false` and there was a failure detected, `error` is set. If `done == true`, exactly one of `error` or `response` is set.
response	Connector The normal response of the operation in case of success. If the original method returns no data on success, such as Delete, the response is google.protobuf.Empty. If the original method is the standard Create/Update, the response should be the target resource of the operation. Any method that returns a long-running operation should document the response type, if any. Includes only one of the fields `error`, `response`. The operation result. If `done == false` and there was no failure detected, neither `error` nor `response` is set. If `done == false` and there was a failure detected, `error` is set. If `done == true`, exactly one of `error` or `response` is set.

UpdateConnectorMetadata

Field

Description

clusterId

string

Required field. ID of the Apache Kafka® cluster the connector is being updated in.

connectorName

string

Required field. Name of the Apache Kafka® connector that is being updated.

Connector

Field	Description
name	string Name of the connector.
tasksMax	google.protobuf.Int64Value Maximum number of connector tasks. Default value is the number of brokers.
properties	string A set of properties passed to Managed Service for Apache Kafka® with the connector configuration. Example: `sync.topics.config.enabled: true`.
health	enum Health Connector health. `HEALTH_UNKNOWN`: Health of the connector is unknown. `ALIVE`: Connector is running. `DEAD`: Connector has failed to start.
status	enum Status Current status of the connector. `STATUS_UNKNOWN`: Connector state is unknown. `RUNNING`: Connector is running normally. `ERROR`: Connector has encountered a problem and cannot operate. `PAUSED`: Connector is paused.
clusterId	string ID of the Apache Kafka® cluster that the connector belongs to.
connectorConfigMirrormaker	ConnectorConfigMirrorMaker Configuration of the MirrorMaker connector. Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`. Additional settings for the connector.
connectorConfigS3Sink	ConnectorConfigS3Sink Configuration of S3-Sink connector. Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`. Additional settings for the connector.

ConnectorConfigMirrorMaker

Field	Description
sourceCluster	ClusterConnection Source cluster connection configuration.
targetCluster	ClusterConnection Target cluster connection configuration.
topics	string List of Kafka topics, separated by `,`.
replicationFactor	google.protobuf.Int64Value Replication factor for automatically created topics.

ClusterConnection

Field	Description
alias	string Alias of cluster connection configuration. Examples: `source`, `target`.
thisCluster	ThisCluster Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. Includes only one of the fields `thisCluster`, `externalCluster`. Type of connection to Apache Kafka® cluster.
externalCluster	ExternalClusterConnection Configuration of connection to an external cluster with all the necessary credentials. Includes only one of the fields `thisCluster`, `externalCluster`. Type of connection to Apache Kafka® cluster.

ThisCluster

Field	Description
Empty

ExternalClusterConnection

Field	Description
bootstrapServers	string List of bootstrap servers of the cluster, separated by `,`.
saslUsername	string SASL username to use for connection to the cluster.
saslMechanism	string SASL mechanism to use for connection to the cluster.
securityProtocol	string Security protocol to use for connection to the cluster.

ConnectorConfigS3Sink

An Apache Kafka® S3-Sink
connector resource.

Field	Description
topics	string List of Kafka topics, separated by ','.
fileCompressionType	string The compression type used for files put on GCS. The supported values are: `gzip`, `snappy`, `zstd`, `none`. Optional, the default is `none`.
fileMaxRecords	google.protobuf.Int64Value Max records per file.
s3Connection	S3Connection Credentials for connecting to S3 storage.

S3Connection

Resource for S3Connection -
settings of connection to AWS-compatible S3 storage, that
are source or target of Kafka S3-connectors.
YC Object Storage is AWS-compatible.

Field

Description

bucketName

string

externalS3

ExternalS3Storage

Includes only one of the fields externalS3.

ExternalS3Storage

Field	Description
accessKeyId	string
endpoint	string
region	string Default is 'us-east-1'

Managed Service for Apache Kafka® API, gRPC: ConnectorService.Update

gRPC requestgRPC request

UpdateConnectorRequestUpdateConnectorRequest

UpdateConnectorSpecUpdateConnectorSpec

ConnectorConfigMirrorMakerSpecConnectorConfigMirrorMakerSpec

ClusterConnectionSpecClusterConnectionSpec

ThisClusterSpecThisClusterSpec

ExternalClusterConnectionSpecExternalClusterConnectionSpec

UpdateConnectorConfigS3SinkSpecUpdateConnectorConfigS3SinkSpec

S3ConnectionSpecS3ConnectionSpec

ExternalS3StorageSpecExternalS3StorageSpec

operation.Operationoperation.Operation

UpdateConnectorMetadataUpdateConnectorMetadata

ConnectorConnector

ConnectorConfigMirrorMakerConnectorConfigMirrorMaker

ClusterConnectionClusterConnection

ThisClusterThisCluster

ExternalClusterConnectionExternalClusterConnection

ConnectorConfigS3SinkConnectorConfigS3Sink

S3ConnectionS3Connection

ExternalS3StorageExternalS3Storage

Была ли статья полезна?

gRPC request

UpdateConnectorRequest

UpdateConnectorSpec

ConnectorConfigMirrorMakerSpec

ClusterConnectionSpec

ThisClusterSpec

ExternalClusterConnectionSpec

UpdateConnectorConfigS3SinkSpec

S3ConnectionSpec

ExternalS3StorageSpec

operation.Operation

UpdateConnectorMetadata

Connector

ConnectorConfigMirrorMaker

ClusterConnection

ThisCluster

ExternalClusterConnection

ConnectorConfigS3Sink

S3Connection

ExternalS3Storage