Managed Service for Apache Kafka® API, REST: Connector.Create

Written by

Updated at October 17, 2024

HTTP request
Path parameters
Body parameters
ConnectorSpec
ConnectorConfigMirrorMakerSpec
ClusterConnectionSpec
ExternalClusterConnectionSpec
ConnectorConfigS3SinkSpec
S3ConnectionSpec
ExternalS3StorageSpec
Response
CreateConnectorMetadata
Status
Connector
ConnectorConfigMirrorMaker
ClusterConnection
ExternalClusterConnection
ConnectorConfigS3Sink
S3Connection
ExternalS3Storage

Creates a new Apache Kafka® connector in a cluster.

HTTP request

POST https://mdb.api.cloud.yandex.net/managed-kafka/v1/clusters/{clusterId}/connectors

Path parameters

Field

Description

clusterId

string

Required field. ID of the Apache Kafka® cluster to create the connector in.

To get this ID, make a ClusterService.List request.

Body parameters

{
  "connectorSpec": {
    "name": "string",
    "tasksMax": "string",
    "properties": "string",
    // Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`
    "connectorConfigMirrormaker": {
      "sourceCluster": {
        "alias": "string",
        // Includes only one of the fields `thisCluster`, `externalCluster`
        "thisCluster": "object",
        "externalCluster": {
          "bootstrapServers": "string",
          "saslUsername": "string",
          "saslPassword": "string",
          "saslMechanism": "string",
          "securityProtocol": "string",
          "sslTruststoreCertificates": "string"
        }
        // end of the list of possible fields
      },
      "targetCluster": {
        "alias": "string",
        // Includes only one of the fields `thisCluster`, `externalCluster`
        "thisCluster": "object",
        "externalCluster": {
          "bootstrapServers": "string",
          "saslUsername": "string",
          "saslPassword": "string",
          "saslMechanism": "string",
          "securityProtocol": "string",
          "sslTruststoreCertificates": "string"
        }
        // end of the list of possible fields
      },
      "topics": "string",
      "replicationFactor": "string"
    },
    "connectorConfigS3Sink": {
      "topics": "string",
      "fileCompressionType": "string",
      "fileMaxRecords": "string",
      "s3Connection": {
        "bucketName": "string",
        // Includes only one of the fields `externalS3`
        "externalS3": {
          "accessKeyId": "string",
          "secretAccessKey": "string",
          "endpoint": "string",
          "region": "string"
        }
        // end of the list of possible fields
      }
    }
    // end of the list of possible fields
  }
}

Field

Description

connectorSpec

ConnectorSpec

Required field. Configuration of the connector to create.

ConnectorSpec

An object that represents an Apache Kafka® connector.

See the documentation for details.

Field	Description
name	string Name of the connector.
tasksMax	string (int64) Maximum number of connector tasks. Default value is the number of brokers.
properties	string A set of properties passed to Managed Service for Apache Kafka® with the connector configuration. Example: `sync.topics.config.enabled: true`.
connectorConfigMirrormaker	ConnectorConfigMirrorMakerSpec Configuration of the MirrorMaker connector. Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`. Additional settings for the connector.
connectorConfigS3Sink	ConnectorConfigS3SinkSpec Configuration of S3-Sink connector. Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`. Additional settings for the connector.

ConnectorConfigMirrorMakerSpec

Field	Description
sourceCluster	ClusterConnectionSpec Source cluster configuration for the MirrorMaker connector.
targetCluster	ClusterConnectionSpec Target cluster configuration for the MirrorMaker connector.
topics	string List of Kafka topics, separated by `,`.
replicationFactor	string (int64) Replication factor for automatically created topics.

ClusterConnectionSpec

Field	Description
alias	string Alias of cluster connection configuration. Examples: `source`, `target`.
thisCluster	object Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. Includes only one of the fields `thisCluster`, `externalCluster`. Type of connection to Apache Kafka® cluster.
externalCluster	ExternalClusterConnectionSpec Configuration of connection to an external cluster with all the necessary credentials. Includes only one of the fields `thisCluster`, `externalCluster`. Type of connection to Apache Kafka® cluster.

ExternalClusterConnectionSpec

Field	Description
bootstrapServers	string List of bootstrap servers of the cluster, separated by `,`.
saslUsername	string SASL username to use for connection to the cluster.
saslPassword	string SASL password to use for connection to the cluster.
saslMechanism	string SASL mechanism to use for connection to the cluster.
securityProtocol	string Security protocol to use for connection to the cluster.
sslTruststoreCertificates	string CA in PEM format to connect to external cluster. Lines of certificate separated by '\n' symbol.

ConnectorConfigS3SinkSpec

Specification for Kafka S3-Sink Connector.

Field	Description
topics	string List of Kafka topics, separated by ','.
fileCompressionType	string The compression type used for files put on GCS. The supported values are: `gzip`, `snappy`, `zstd`, `none`. Optional, the default is `none`.
fileMaxRecords	string (int64) Max records per file.
s3Connection	S3ConnectionSpec Credentials for connecting to S3 storage.

S3ConnectionSpec

Specification for S3Connection -
settings of connection to AWS-compatible S3 storage, that
are source or target of Kafka S3-connectors.
YC Object Storage is AWS-compatible.

Field

Description

bucketName

string

externalS3

ExternalS3StorageSpec

Includes only one of the fields externalS3.

ExternalS3StorageSpec

Field	Description
accessKeyId	string
secretAccessKey	string
endpoint	string
region	string Default is 'us-east-1'.

Response

HTTP Code: 200 - OK

{
  "id": "string",
  "description": "string",
  "createdAt": "string",
  "createdBy": "string",
  "modifiedAt": "string",
  "done": "boolean",
  "metadata": {
    "clusterId": "string",
    "connectorName": "string"
  },
  // Includes only one of the fields `error`, `response`
  "error": {
    "code": "integer",
    "message": "string",
    "details": [
      "object"
    ]
  },
  "response": {
    "name": "string",
    "tasksMax": "string",
    "properties": "string",
    "health": "string",
    "status": "string",
    "clusterId": "string",
    // Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`
    "connectorConfigMirrormaker": {
      "sourceCluster": {
        "alias": "string",
        // Includes only one of the fields `thisCluster`, `externalCluster`
        "thisCluster": "object",
        "externalCluster": {
          "bootstrapServers": "string",
          "saslUsername": "string",
          "saslMechanism": "string",
          "securityProtocol": "string"
        }
        // end of the list of possible fields
      },
      "targetCluster": {
        "alias": "string",
        // Includes only one of the fields `thisCluster`, `externalCluster`
        "thisCluster": "object",
        "externalCluster": {
          "bootstrapServers": "string",
          "saslUsername": "string",
          "saslMechanism": "string",
          "securityProtocol": "string"
        }
        // end of the list of possible fields
      },
      "topics": "string",
      "replicationFactor": "string"
    },
    "connectorConfigS3Sink": {
      "topics": "string",
      "fileCompressionType": "string",
      "fileMaxRecords": "string",
      "s3Connection": {
        "bucketName": "string",
        // Includes only one of the fields `externalS3`
        "externalS3": {
          "accessKeyId": "string",
          "endpoint": "string",
          "region": "string"
        }
        // end of the list of possible fields
      }
    }
    // end of the list of possible fields
  }
  // end of the list of possible fields
}

An Operation resource. For more information, see Operation.

Field	Description
id	string ID of the operation.
description	string Description of the operation. 0-256 characters long.
createdAt	string (date-time) Creation timestamp. String in RFC3339 text format. The range of possible values is from `0001-01-01T00:00:00Z` to `9999-12-31T23:59:59.999999999Z`, i.e. from 0 to 9 digits for fractions of a second. To work with values in this field, use the APIs described in the Protocol Buffers reference. In some languages, built-in datetime utilities do not support nanosecond precision (9 digits).
createdBy	string ID of the user or service account who initiated the operation.
modifiedAt	string (date-time) The time when the Operation resource was last modified. String in RFC3339 text format. The range of possible values is from `0001-01-01T00:00:00Z` to `9999-12-31T23:59:59.999999999Z`, i.e. from 0 to 9 digits for fractions of a second. To work with values in this field, use the APIs described in the Protocol Buffers reference. In some languages, built-in datetime utilities do not support nanosecond precision (9 digits).
done	boolean If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available.
metadata	CreateConnectorMetadata Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any.
error	Status The error result of the operation in case of failure or cancellation. Includes only one of the fields `error`, `response`. The operation result. If `done == false` and there was no failure detected, neither `error` nor `response` is set. If `done == false` and there was a failure detected, `error` is set. If `done == true`, exactly one of `error` or `response` is set.
response	Connector The normal response of the operation in case of success. If the original method returns no data on success, such as Delete, the response is google.protobuf.Empty. If the original method is the standard Create/Update, the response should be the target resource of the operation. Any method that returns a long-running operation should document the response type, if any. Includes only one of the fields `error`, `response`. The operation result. If `done == false` and there was no failure detected, neither `error` nor `response` is set. If `done == false` and there was a failure detected, `error` is set. If `done == true`, exactly one of `error` or `response` is set.

CreateConnectorMetadata

Field

Description

clusterId

string

ID of the Apache Kafka® cluster the connector is being created in.

connectorName

string

Required field. Name of the Apache Kafka® connector that is being created.

Status

The error result of the operation in case of failure or cancellation.

Field	Description
code	integer (int32) Error code. An enum value of google.rpc.Code.
message	string An error message.
details[]	object A list of messages that carry the error details.

Connector

Field	Description
name	string Name of the connector.
tasksMax	string (int64) Maximum number of connector tasks. Default value is the number of brokers.
properties	string A set of properties passed to Managed Service for Apache Kafka® with the connector configuration. Example: `sync.topics.config.enabled: true`.
health	enum (Health) Connector health. `HEALTH_UNKNOWN`: Health of the connector is unknown. `ALIVE`: Connector is running. `DEAD`: Connector has failed to start.
status	enum (Status) Current status of the connector. `STATUS_UNKNOWN`: Connector state is unknown. `RUNNING`: Connector is running normally. `ERROR`: Connector has encountered a problem and cannot operate. `PAUSED`: Connector is paused.
clusterId	string ID of the Apache Kafka® cluster that the connector belongs to.
connectorConfigMirrormaker	ConnectorConfigMirrorMaker Configuration of the MirrorMaker connector. Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`. Additional settings for the connector.
connectorConfigS3Sink	ConnectorConfigS3Sink Configuration of S3-Sink connector. Includes only one of the fields `connectorConfigMirrormaker`, `connectorConfigS3Sink`. Additional settings for the connector.

ConnectorConfigMirrorMaker

Field	Description
sourceCluster	ClusterConnection Source cluster connection configuration.
targetCluster	ClusterConnection Target cluster connection configuration.
topics	string List of Kafka topics, separated by `,`.
replicationFactor	string (int64) Replication factor for automatically created topics.

ClusterConnection

Field	Description
alias	string Alias of cluster connection configuration. Examples: `source`, `target`.
thisCluster	object Connection configuration of the cluster the connector belongs to. As all credentials are already known, leave this parameter empty. Includes only one of the fields `thisCluster`, `externalCluster`. Type of connection to Apache Kafka® cluster.
externalCluster	ExternalClusterConnection Configuration of connection to an external cluster with all the necessary credentials. Includes only one of the fields `thisCluster`, `externalCluster`. Type of connection to Apache Kafka® cluster.

ExternalClusterConnection

Field	Description
bootstrapServers	string List of bootstrap servers of the cluster, separated by `,`.
saslUsername	string SASL username to use for connection to the cluster.
saslMechanism	string SASL mechanism to use for connection to the cluster.
securityProtocol	string Security protocol to use for connection to the cluster.

ConnectorConfigS3Sink

An Apache Kafka® S3-Sink
connector resource.

Field	Description
topics	string List of Kafka topics, separated by ','.
fileCompressionType	string The compression type used for files put on GCS. The supported values are: `gzip`, `snappy`, `zstd`, `none`. Optional, the default is `none`.
fileMaxRecords	string (int64) Max records per file.
s3Connection	S3Connection Credentials for connecting to S3 storage.

S3Connection

Resource for S3Connection -
settings of connection to AWS-compatible S3 storage, that
are source or target of Kafka S3-connectors.
YC Object Storage is AWS-compatible.

Field

Description

bucketName

string

externalS3

ExternalS3Storage

Includes only one of the fields externalS3.

ExternalS3Storage

Field	Description
accessKeyId	string
endpoint	string
region	string Default is 'us-east-1'

Managed Service for Apache Kafka® API, REST: Connector.Create

HTTP requestHTTP request

Path parametersPath parameters

Body parametersBody parameters

ConnectorSpecConnectorSpec

ConnectorConfigMirrorMakerSpecConnectorConfigMirrorMakerSpec

ClusterConnectionSpecClusterConnectionSpec

ExternalClusterConnectionSpecExternalClusterConnectionSpec

ConnectorConfigS3SinkSpecConnectorConfigS3SinkSpec

S3ConnectionSpecS3ConnectionSpec

ExternalS3StorageSpecExternalS3StorageSpec

ResponseResponse

CreateConnectorMetadataCreateConnectorMetadata

StatusStatus

ConnectorConnector

ConnectorConfigMirrorMakerConnectorConfigMirrorMaker

ClusterConnectionClusterConnection

ExternalClusterConnectionExternalClusterConnection

ConnectorConfigS3SinkConnectorConfigS3Sink

S3ConnectionS3Connection

ExternalS3StorageExternalS3Storage

Was the article helpful?

HTTP request

Path parameters

Body parameters

ConnectorSpec

ConnectorConfigMirrorMakerSpec

ClusterConnectionSpec

ExternalClusterConnectionSpec

ConnectorConfigS3SinkSpec

S3ConnectionSpec

ExternalS3StorageSpec

Response

CreateConnectorMetadata

Status

Connector

ConnectorConfigMirrorMaker

ClusterConnection

ExternalClusterConnection

ConnectorConfigS3Sink

S3Connection

ExternalS3Storage