yandex_datatransfer_endpoint (Data Source)

Data Transfer endpoint. For more information, see the official
documentation

SchemaSchema

OptionalOptional

• endpoint_id (String) Identifier of the endpoint to return.

To get the endpoint ID, make an [EndpointService.List] request.

• folder_id (String) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto
• id (String) Identifier of the endpoint to return.

To get the endpoint ID, make an [EndpointService.List] request.

• timeouts (Attributes) (see below for nested schema)

Read-OnlyRead-Only

• description (String) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto
• labels (Map of String) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto
• name (String) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto
• settings (Attributes List) DataTransfer Endpoint Settings block (see below for nested schema)

Nested Schema for Nested Schema for timeouts

Optional:

• create (String) A string that can be parsed as a duration consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours).
• delete (String) A string that can be parsed as a duration consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours). Setting a timeout for a Delete operation is only applicable if changes are saved into state before the destroy operation occurs.
• read (String) A string that can be parsed as a duration consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours). Read operations occur during any refresh or planning operation when refresh is enabled.
• update (String) A string that can be parsed as a duration consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours).

Nested Schema for Nested Schema for settings

Read-Only:

• clickhouse_source (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• clickhouse_target (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• kafka_source (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• kafka_target (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• metrika_source (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• mongo_source (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• mongo_target (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• mysql_source (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• mysql_target (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• postgres_source (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• postgres_target (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• ydb_source (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• ydb_target (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• yds_source (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)
• yds_target (Attributes List) package: yandex.cloud.datatransfer.v1
  filename: yandex/cloud/datatransfer/v1/endpoint.proto (see below for nested schema)

Nested Schema for Nested Schema for settings.clickhouse_source

Read-Only:

• clickhouse_cluster_name (String) Name of the ClickHouse cluster. For Managed ClickHouse that is name of
  ShardGroup or managed cluster ID by default
• connection (Attributes List) Connection settings (see below for nested schema)
• exclude_tables (List of String) Exclude list of tables for replication. If none or empty list is presented -
  will replicate all tables. Can contain * patterns.
• include_tables (List of String) White list of tables for replication. If none or empty list is presented - will
  replicate all tables. Can contain * patterns.
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.clickhouse_source.connection

Read-Only:

• connection_options (Attributes List) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto (see below for nested schema)

Nested Schema for Nested Schema for settings.clickhouse_source.connection.connection_options

Read-Only:

• connection_manager_connection (Attributes List) Get ClickHouse installation params and credentials from Connection Manager (see below for nested schema)
• database (String) Database name
• mdb_cluster_id (String) Identifier of the Managed ClickHouse cluster
• on_premise (Attributes List) Connection settings of the on-premise ClickHouse server (see below for nested schema)
• password (Attributes List) Password for the database access (see below for nested schema)
• user (String) User for database access. Required unless connection_manager_connection is used

Nested Schema for Nested Schema for settings.clickhouse_source.connection.connection_options.connection_manager_connection

Read-Only:

• connection_id (String) ID of connection in Connection Manager with installation params and credetials
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.clickhouse_source.connection.connection_options.on_premise

Read-Only:

• http_port (Number) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto
• native_port (Number) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto
• shards (Attributes List) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto (see below for nested schema)
• tls_mode (Attributes List) TLS settings for server connection. Disabled by default (see below for nested schema)

Nested Schema for Nested Schema for settings.clickhouse_source.connection.connection_options.on_premise.shards

Read-Only:

• hosts (List of String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto
• name (String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto

Nested Schema for Nested Schema for settings.clickhouse_source.connection.connection_options.on_premise.tls_mode

Read-Only:

• disabled (Attributes List) Empty block designating that the connection is not secured, i.e. plaintext
  connection (see below for nested schema)
• enabled (Attributes List) TLS is used for the server connection (see below for nested schema)

Nested Schema for Nested Schema for settings.clickhouse_source.connection.connection_options.on_premise.tls_mode.disabled

Nested Schema for Nested Schema for settings.clickhouse_source.connection.connection_options.on_premise.tls_mode.enabled

Read-Only:

• ca_certificate (String) CA certificate

X.509 certificate of the certificate authority which issued the server's
certificate, in PEM format. When CA certificate is specified, TLS is used to
connect to the server. If CA certificate is empty, the server's certificate must
be signed by a well-known CA

Nested Schema for Nested Schema for settings.clickhouse_source.connection.connection_options.password

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.clickhouse_target

Read-Only:

• alt_names (Attributes List) Table renaming rules in target (see below for nested schema)
• cleanup_policy (String) How to clean collections when activating the transfer. One of
  CLICKHOUSE_CLEANUP_POLICY_DISABLED or CLICKHOUSE_CLEANUP_POLICY_DROP
• clickhouse_cluster_name (String) Name of the ClickHouse cluster. For Managed ClickHouse that is name of
  ShardGroup or managed cluster ID by default.
• connection (Attributes List) Connection settings (see below for nested schema)
• is_schema_migration_disabled (Boolean) Whether can change table schema if schema changed on source
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• sharding (Attributes List) Shard selection rules for the data being transferred (see below for nested schema)
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.clickhouse_target.alt_names

Read-Only:

• from_name (String) Source table name
• to_name (String) Target table name

Nested Schema for Nested Schema for settings.clickhouse_target.connection

Read-Only:

• connection_options (Attributes List) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto (see below for nested schema)

Nested Schema for Nested Schema for settings.clickhouse_target.connection.connection_options

Read-Only:

• connection_manager_connection (Attributes List) Get ClickHouse installation params and credentials from Connection Manager (see below for nested schema)
• database (String) Database name
• mdb_cluster_id (String) Identifier of the Managed ClickHouse cluster
• on_premise (Attributes List) Connection settings of the on-premise ClickHouse server (see below for nested schema)
• password (Attributes List) Password for the database access (see below for nested schema)
• user (String) User for database access. Required unless connection_manager_connection is used

Nested Schema for Nested Schema for settings.clickhouse_target.connection.connection_options.connection_manager_connection

Read-Only:

• connection_id (String) ID of connection in Connection Manager with installation params and credetials
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.clickhouse_target.connection.connection_options.on_premise

Read-Only:

• http_port (Number) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto
• native_port (Number) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto
• shards (Attributes List) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto (see below for nested schema)
• tls_mode (Attributes List) TLS settings for server connection. Disabled by default (see below for nested schema)

Nested Schema for Nested Schema for settings.clickhouse_target.connection.connection_options.on_premise.shards

Read-Only:

• hosts (List of String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto
• name (String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto

Nested Schema for Nested Schema for settings.clickhouse_target.connection.connection_options.on_premise.tls_mode

Read-Only:

• disabled (Attributes List) Empty block designating that the connection is not secured, i.e. plaintext
  connection (see below for nested schema)
• enabled (Attributes List) TLS is used for the server connection (see below for nested schema)

Nested Schema for Nested Schema for settings.clickhouse_target.connection.connection_options.on_premise.tls_mode.disabled

Nested Schema for Nested Schema for settings.clickhouse_target.connection.connection_options.on_premise.tls_mode.enabled

Read-Only:

• ca_certificate (String) CA certificate

X.509 certificate of the certificate authority which issued the server's
certificate, in PEM format. When CA certificate is specified, TLS is used to
connect to the server. If CA certificate is empty, the server's certificate must
be signed by a well-known CA

Nested Schema for Nested Schema for settings.clickhouse_target.connection.connection_options.password

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.clickhouse_target.sharding

Read-Only:

• column_value_hash (Attributes List) Shard data by the hash value of the specified column (see below for nested schema)
• custom_mapping (Attributes List) A custom shard mapping by the value of the specified column (see below for nested schema)
• round_robin (Attributes List) Distribute incoming rows between ClickHouse shards in a round-robin manner.
  Specify as an empty block to enable (see below for nested schema)
• transfer_id (Attributes List) Shard data by ID of the transfer (see below for nested schema)

Nested Schema for Nested Schema for settings.clickhouse_target.sharding.column_value_hash

Read-Only:

• column_name (String) The name of the column to calculate hash from

Nested Schema for Nested Schema for settings.clickhouse_target.sharding.custom_mapping

Read-Only:

• column_name (String) The name of the column to inspect when deciding the shard to chose for an
  incoming row
• mapping (Attributes List) The mapping of the specified column values to the shard names (see below for nested schema)

Nested Schema for Nested Schema for settings.clickhouse_target.sharding.transfer_id.mapping

Read-Only:

• column_value (Attributes List) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto (see below for nested schema)
• shard_name (String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/clickhouse.proto

Nested Schema for Nested Schema for settings.clickhouse_target.sharding.transfer_id.mapping.column_value

Read-Only:

• string_value (String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/common.proto

Nested Schema for Nested Schema for settings.clickhouse_target.sharding.round_robin

Nested Schema for Nested Schema for settings.clickhouse_target.sharding.transfer_id

Nested Schema for Nested Schema for settings.kafka_source

Read-Only:

• auth (Attributes List) Authentication settings (see below for nested schema)
• connection (Attributes List) Connection settings (see below for nested schema)
• parser (Attributes List) Data parsing parameters. If not set, the source messages are read in raw (see below for nested schema)
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• topic_name (String, Deprecated) Deprecated. Please use topic_names instead
  Full source topic name
• topic_names (List of String) List of full source topic names to read
• transformer (Attributes List) Transform data with a custom Cloud Function (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_source.auth

Read-Only:

• no_auth (Attributes List) No authentication (see below for nested schema)
• sasl (Attributes List) Authentication with SASL (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_source.auth.no_auth

Nested Schema for Nested Schema for settings.kafka_source.auth.sasl

Read-Only:

• mechanism (String) SASL mechanism for authentication, use one of: KAFKA_MECHANISM_SHA256,
  KAFKA_MECHANISM_SHA512
• password (Attributes List) Password for user (see below for nested schema)
• user (String) User name

Nested Schema for Nested Schema for settings.kafka_source.auth.sasl.password

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.kafka_source.connection

Read-Only:

• cluster_id (String) Managed Service for Kafka cluster ID.
  Set only one of: cluster_id/on_premise/connection_manager_connection
• connection_manager_connection (Attributes List) Get Kafka installation params and credentials from Connection Manager
  Set only one of: cluster_id/on_premise/connection_manager_connection (see below for nested schema)
• on_premise (Attributes List) Connection options for on-premise Kafka
  Set only one of: cluster_id/on_premise/connection_manager_connection (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_source.connection.connection_manager_connection

Read-Only:

• connection_id (String) ID of connection in Connection Manager with installation params and credetials
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.kafka_source.connection.on_premise

Read-Only:

• broker_urls (List of String) Kafka broker URLs
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet
• tls_mode (Attributes List) TLS settings for broker connection. Disabled by default. (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_source.connection.on_premise.tls_mode

Read-Only:

• disabled (Attributes List) Empty block designating that the connection is not secured, i.e. plaintext
  connection (see below for nested schema)
• enabled (Attributes List) TLS is used for the server connection (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_source.connection.on_premise.tls_mode.disabled

Nested Schema for Nested Schema for settings.kafka_source.connection.on_premise.tls_mode.enabled

Read-Only:

• ca_certificate (String) CA certificate

X.509 certificate of the certificate authority which issued the server's
certificate, in PEM format. When CA certificate is specified, TLS is used to
connect to the server. If CA certificate is empty, the server's certificate must
be signed by a well-known CA

Nested Schema for Nested Schema for settings.kafka_source.parser

Read-Only:

• audit_trails_v1_parser (Attributes List) Parse Audit Trails data. Empty struct (see below for nested schema)
• cloud_logging_parser (Attributes List) Parse Cloud Logging data. Empty struct (see below for nested schema)
• json_parser (Attributes List) Parse data in json format (see below for nested schema)
• tskv_parser (Attributes List) Parse data in tskv format (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_source.parser.audit_trails_v1_parser

Nested Schema for Nested Schema for settings.kafka_source.parser.cloud_logging_parser

Nested Schema for Nested Schema for settings.kafka_source.parser.json_parser

Read-Only:

• add_rest_column (Boolean) Will add _rest column for all unknown fields
• data_schema (Attributes List) Data parsing scheme (see below for nested schema)
• null_keys_allowed (Boolean) Allow null keys, if no - null keys will be putted to unparsed data
• unescape_string_values (Boolean) Unescape string values

Nested Schema for Nested Schema for settings.kafka_source.parser.tskv_parser.data_schema

Read-Only:

• fields (Attributes List) Description of the data schema in the array of fields structure (see below for nested schema)
• json_fields (String, Deprecated) Description of the data schema as JSON specification

Nested Schema for Nested Schema for settings.kafka_source.parser.tskv_parser.data_schema.fields

Read-Only:

• fields (Attributes List) Description of the column schema in the array of fields structure (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_source.parser.tskv_parser.data_schema.json_fields.fields

Read-Only:

• key (Boolean) Mark field as Primary Key
• name (String) Field name
• path (String) Path to the field
• required (Boolean) Mark field as required
• type (String) Field type, one of: INT64, INT32, INT16, INT8, UINT64, UINT32,
  UINT16, UINT8, DOUBLE, BOOLEAN, STRING, UTF8, ANY, DATETIME.

Nested Schema for Nested Schema for settings.kafka_source.parser.tskv_parser

Read-Only:

• add_rest_column (Boolean) Will add _rest column for all unknown fields
• data_schema (Attributes List) Data parsing scheme (see below for nested schema)
• null_keys_allowed (Boolean) Allow null keys, if no - null keys will be putted to unparsed data
• unescape_string_values (Boolean) Unescape string values

Nested Schema for Nested Schema for settings.kafka_source.parser.tskv_parser.data_schema

Read-Only:

• fields (Attributes List) Description of the data schema in the array of fields structure (see below for nested schema)
• json_fields (String, Deprecated) Description of the data schema as JSON specification

Nested Schema for Nested Schema for settings.kafka_source.parser.tskv_parser.data_schema.fields

Read-Only:

• fields (Attributes List) Description of the column schema in the array of fields structure (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_source.parser.tskv_parser.data_schema.json_fields.fields

Read-Only:

• key (Boolean) Mark field as Primary Key
• name (String) Field name
• path (String) Path to the field
• required (Boolean) Mark field as required
• type (String) Field type, one of: INT64, INT32, INT16, INT8, UINT64, UINT32,
  UINT16, UINT8, DOUBLE, BOOLEAN, STRING, UTF8, ANY, DATETIME.

Nested Schema for Nested Schema for settings.kafka_source.transformer

Optional:

• service_account_id (String) Service account

Read-Only:

• buffer_flush_interval (String) Flush interval
• buffer_size (String) Buffer size for function. Maximum 4 GB. Use value with units, i.e. 10 B, 20 kB,
  2.0 MB, 30 MB, 1.0 GB
• cloud_function (String) Cloud function
• invocation_timeout (String) Invocation timeout
• number_of_retries (Number) Number of retries

Nested Schema for Nested Schema for settings.kafka_target

Read-Only:

• auth (Attributes List) Authentication settings (see below for nested schema)
• connection (Attributes List) Connection settings (see below for nested schema)
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• serializer (Attributes List) Data serialization format settings (see below for nested schema)
• topic_settings (Attributes List) Target topic settings (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_target.auth

Read-Only:

• no_auth (Attributes List) No authentication (see below for nested schema)
• sasl (Attributes List) Authentication with SASL (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_target.auth.no_auth

Nested Schema for Nested Schema for settings.kafka_target.auth.sasl

Read-Only:

• mechanism (String) SASL mechanism for authentication, use one of: KAFKA_MECHANISM_SHA256,
  KAFKA_MECHANISM_SHA512
• password (Attributes List) Password for user (see below for nested schema)
• user (String) User name

Nested Schema for Nested Schema for settings.kafka_target.auth.sasl.password

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.kafka_target.connection

Read-Only:

• cluster_id (String) Managed Service for Kafka cluster ID.
  Set only one of: cluster_id/on_premise/connection_manager_connection
• connection_manager_connection (Attributes List) Get Kafka installation params and credentials from Connection Manager
  Set only one of: cluster_id/on_premise/connection_manager_connection (see below for nested schema)
• on_premise (Attributes List) Connection options for on-premise Kafka
  Set only one of: cluster_id/on_premise/connection_manager_connection (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_target.connection.connection_manager_connection

Read-Only:

• connection_id (String) ID of connection in Connection Manager with installation params and credetials
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.kafka_target.connection.on_premise

Read-Only:

• broker_urls (List of String) Kafka broker URLs
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet
• tls_mode (Attributes List) TLS settings for broker connection. Disabled by default. (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_target.connection.on_premise.tls_mode

Read-Only:

• disabled (Attributes List) Empty block designating that the connection is not secured, i.e. plaintext
  connection (see below for nested schema)
• enabled (Attributes List) TLS is used for the server connection (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_target.connection.on_premise.tls_mode.disabled

Nested Schema for Nested Schema for settings.kafka_target.connection.on_premise.tls_mode.enabled

Read-Only:

• ca_certificate (String) CA certificate

X.509 certificate of the certificate authority which issued the server's
certificate, in PEM format. When CA certificate is specified, TLS is used to
connect to the server. If CA certificate is empty, the server's certificate must
be signed by a well-known CA

Nested Schema for Nested Schema for settings.kafka_target.serializer

Read-Only:

• serializer_auto (Attributes List) Empty block. Select the serialization format automatically (see below for nested schema)
• serializer_debezium (Attributes List) Serialize data in debezium json format (see below for nested schema)
• serializer_json (Attributes List) Empty block. Serialize data in json format (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_target.serializer.serializer_auto

Nested Schema for Nested Schema for settings.kafka_target.serializer.serializer_debezium

Read-Only:

• serializer_parameters (Attributes List) A list of Debezium parameters set by the structure of the key and value
  string fields (see below for nested schema)

Nested Schema for Nested Schema for settings.kafka_target.serializer.serializer_json.serializer_parameters

Read-Only:

• key (String) Name of the serializer parameter
• value (String) Value of the serializer parameter

Nested Schema for Nested Schema for settings.kafka_target.serializer.serializer_json

Nested Schema for Nested Schema for settings.kafka_target.topic_settings

Read-Only:

• topic (Attributes List) All messages will be sent to one topic (see below for nested schema)
• topic_prefix (String) Topic prefix
  Messages will be sent to topic with name <topic_prefix>..<table_name>.
  Analogue of the Debezium setting database.server.name.

Nested Schema for Nested Schema for settings.kafka_target.topic_settings.topic

Read-Only:

• save_tx_order (Boolean) Save transactions order
  Not to split events queue into separate per-table queues.
• topic_name (String) Full topic name

Nested Schema for Nested Schema for settings.metrika_source

Read-Only:

• counter_ids (List of Number) Counter IDs
• streams (Attributes List) Streams (see below for nested schema)
• token (Attributes List) Authentication token (see below for nested schema)

Nested Schema for Nested Schema for settings.metrika_source.streams

Read-Only:

• columns (List of String) Column names
• type (String) Stream type, one of: METRIKA_STREAM_TYPE_HITS, METRIKA_STREAM_TYPE_VISITS,
  METRIKA_STREAM_TYPE_HITS_V2

Nested Schema for Nested Schema for settings.metrika_source.token

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.mongo_source

Read-Only:

• collections (Attributes List) List of collections for replication. Empty list implies replication of all
  tables on the deployment. Allowed to use * as collection name. (see below for nested schema)
• connection (Attributes List) Connection settings (see below for nested schema)
• excluded_collections (Attributes List) List of forbidden collections for replication. Allowed to use * as collection
  name for forbid all collections of concrete schema. (see below for nested schema)
• secondary_preferred_mode (Boolean) Read mode for mongo client: whether the secondary server should be preferred to
  the primary when copying data
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.mongo_source.collections

Read-Only:

• collection_name (String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/mongo.proto
• database_name (String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/mongo.proto

Nested Schema for Nested Schema for settings.mongo_source.connection

Read-Only:

• connection_options (Attributes List) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/mongo.proto (see below for nested schema)

Nested Schema for Nested Schema for settings.mongo_source.connection.connection_options

Read-Only:

• auth_source (String) Database name associated with the credentials
• connection_manager_connection (Attributes List) Get StoreDoc/MongoDB installation params and credentials from Connection Manager
  Use one of: mdb_cluster_id/on_premise/connection_manager_connection (see below for nested schema)
• mdb_cluster_id (String) Identifier of the Yandex StoreDoc cluster
  Use one of: mdb_cluster_id/on_premise/connection_manager_connection
• on_premise (Attributes List) Connection settings of the on-premise MongoDB server
  Use one of: mdb_cluster_id/on_premise/connection_manager_connection (see below for nested schema)
• password (Attributes List) Password for user (see below for nested schema)
• user (String) User name, required unless connection_manager_connection is used

Nested Schema for Nested Schema for settings.mongo_source.connection.connection_options.connection_manager_connection

Read-Only:

• connection_id (String) ID of connectionmanager connection with mongodb/Yandex Storedoc installation
  parameters and credentials
• replica_set (String) Replica set name, used only for on-premise mongodb installations

Nested Schema for Nested Schema for settings.mongo_source.connection.connection_options.on_premise

Read-Only:

• hosts (List of String) Host names of the replica set
• port (Number) TCP Port number
• replica_set (String) Replica set name
• tls_mode (Attributes List) TLS settings for the server connection. Empty implies plaintext connection (see below for nested schema)

Nested Schema for Nested Schema for settings.mongo_source.connection.connection_options.on_premise.tls_mode

Read-Only:

• disabled (Attributes List) Empty block designating that the connection is not secured, i.e. plaintext
  connection (see below for nested schema)
• enabled (Attributes List) TLS is used for the server connection (see below for nested schema)

Nested Schema for Nested Schema for settings.mongo_source.connection.connection_options.on_premise.tls_mode.disabled

Nested Schema for Nested Schema for settings.mongo_source.connection.connection_options.on_premise.tls_mode.enabled

Read-Only:

• ca_certificate (String) CA certificate

X.509 certificate of the certificate authority which issued the server's
certificate, in PEM format. When CA certificate is specified, TLS is used to
connect to the server. If CA certificate is empty, the server's certificate must
be signed by a well-known CA

Nested Schema for Nested Schema for settings.mongo_source.connection.connection_options.password

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.mongo_source.excluded_collections

Read-Only:

• collection_name (String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/mongo.proto
• database_name (String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/mongo.proto

Nested Schema for Nested Schema for settings.mongo_target

Read-Only:

• cleanup_policy (String) How to clean collections when activating the transfer. One of DISABLED, DROP
  or TRUNCATE
• connection (Attributes List) Connection settings (see below for nested schema)
• database (String) Database name. If not empty, then all the data will be written to the database
  with the specified name; otherwise the database name is the same as in the
  source endpoint
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.mongo_target.connection

Read-Only:

• connection_options (Attributes List) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/mongo.proto (see below for nested schema)

Nested Schema for Nested Schema for settings.mongo_target.connection.connection_options

Read-Only:

• auth_source (String) Database name associated with the credentials
• connection_manager_connection (Attributes List) Get StoreDoc/MongoDB installation params and credentials from Connection Manager
  Use one of: mdb_cluster_id/on_premise/connection_manager_connection (see below for nested schema)
• mdb_cluster_id (String) Identifier of the Yandex StoreDoc cluster
  Use one of: mdb_cluster_id/on_premise/connection_manager_connection
• on_premise (Attributes List) Connection settings of the on-premise MongoDB server
  Use one of: mdb_cluster_id/on_premise/connection_manager_connection (see below for nested schema)
• password (Attributes List) Password for user (see below for nested schema)
• user (String) User name, required unless connection_manager_connection is used

Nested Schema for Nested Schema for settings.mongo_target.connection.connection_options.connection_manager_connection

Read-Only:

• connection_id (String) ID of connectionmanager connection with mongodb/Yandex Storedoc installation
  parameters and credentials
• replica_set (String) Replica set name, used only for on-premise mongodb installations

Nested Schema for Nested Schema for settings.mongo_target.connection.connection_options.on_premise

Read-Only:

• hosts (List of String) Host names of the replica set
• port (Number) TCP Port number
• replica_set (String) Replica set name
• tls_mode (Attributes List) TLS settings for the server connection. Empty implies plaintext connection (see below for nested schema)

Nested Schema for Nested Schema for settings.mongo_target.connection.connection_options.on_premise.tls_mode

Read-Only:

• disabled (Attributes List) Empty block designating that the connection is not secured, i.e. plaintext
  connection (see below for nested schema)
• enabled (Attributes List) TLS is used for the server connection (see below for nested schema)

Nested Schema for Nested Schema for settings.mongo_target.connection.connection_options.on_premise.tls_mode.disabled

Nested Schema for Nested Schema for settings.mongo_target.connection.connection_options.on_premise.tls_mode.enabled

Read-Only:

• ca_certificate (String) CA certificate

X.509 certificate of the certificate authority which issued the server's
certificate, in PEM format. When CA certificate is specified, TLS is used to
connect to the server. If CA certificate is empty, the server's certificate must
be signed by a well-known CA

Nested Schema for Nested Schema for settings.mongo_target.connection.connection_options.password

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.mysql_source

Read-Only:

• connection (Attributes List) Database connection settings (see below for nested schema)
• database (String) Name of the database to transfer

You can leave it empty, then it will be possible to transfer tables from several
databases at the same time from this source.

• exclude_tables_regex (List of String) Opposite of include_table_regex. The tables matching the specified regular
  expressions will not be transferred
• include_tables_regex (List of String) List of regular expressions of table names which should be transferred. A table
  name is formatted as schemaname.tablename. For example, a single regular
  expression may look like ^mydb.employees$
• object_transfer_settings (Attributes List) Schema migration
  Defines which database schema objects should be transferred, e.g. views,
  routines, etc.
  All of the attrubutes in the block are optional and should be either
  BEFORE_DATA, AFTER_DATA or NEVER." (see below for nested schema)
• password (Attributes List) Password for database access. (see below for nested schema)
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• service_database (String) Database for service tables
  Default: data source database. Here created technical tables (__tm_keeper,
  __tm_gtid_keeper).
• timezone (String) Database timezone

Is used for parsing timestamps for saving source timezones. Accepts values from
IANA timezone database. Default: local timezone.

• user (String) User for database access. Required unless connection manager connection is used

Nested Schema for Nested Schema for settings.mysql_source.connection

Read-Only:

• connection_manager_connection (Attributes List) Get Mysql installation params and credentials from Connection Manager (see below for nested schema)
• mdb_cluster_id (String) Managed Service for MySQL cluster ID
• on_premise (Attributes List) Connection options for on-premise MySQL (see below for nested schema)

Nested Schema for Nested Schema for settings.mysql_source.connection.connection_manager_connection

Read-Only:

• connection_id (String) ID of connection in Connection Manager with installation params and credetials
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.mysql_source.connection.on_premise

Read-Only:

• hosts (List of String) List of host names of the MySQL server. Exactly one host is expected
• port (Number) Port for the database connection
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet
• tls_mode (Attributes List) TLS settings for server connection. Disabled by default. (see below for nested schema)

Nested Schema for Nested Schema for settings.mysql_source.connection.on_premise.tls_mode

Read-Only:

• disabled (Attributes List) Empty block designating that the connection is not secured, i.e. plaintext
  connection (see below for nested schema)
• enabled (Attributes List) TLS is used for the server connection (see below for nested schema)

Nested Schema for Nested Schema for settings.mysql_source.connection.on_premise.tls_mode.disabled

Nested Schema for Nested Schema for settings.mysql_source.connection.on_premise.tls_mode.enabled

Read-Only:

• ca_certificate (String) CA certificate

X.509 certificate of the certificate authority which issued the server's
certificate, in PEM format. When CA certificate is specified, TLS is used to
connect to the server. If CA certificate is empty, the server's certificate must
be signed by a well-known CA

Nested Schema for Nested Schema for settings.mysql_source.object_transfer_settings

Read-Only:

• routine (String) Routines

CREATE PROCEDURE ... ; CREATE FUNCTION ... ;

• tables (String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/mysql.proto
• trigger (String) Triggers

CREATE TRIGGER ...

• view (String) Views

CREATE VIEW ...

Nested Schema for Nested Schema for settings.mysql_source.password

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.mysql_target

Read-Only:

• cleanup_policy (String) Cleanup policy for activate, reactivate and reupload processes.
  One of DISABLED, DROP or TRUNCATE Default is DISABLED.
• connection (Attributes List) Database connection settings (see below for nested schema)
• database (String) Database name

Allowed to leave it empty, then the tables will be created in databases with the
same names as on the source. If this field is empty, then you must fill below db
schema for service table.

• is_schema_migration_disabled (Boolean) Whether can change table schema if schema changed on source
• password (Attributes List) Password for database access. (see below for nested schema)
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• service_database (String) Database schema for service table
  Default: db name. Here created technical tables (__tm_keeper, __tm_gtid_keeper).
• skip_constraint_checks (Boolean) Disable constraints checks
  When true, disables foreign key checks and unique checks. False by default.
  See
  foreign_key_checks.
  Recommend to disable for increase replication speed unless schema contains
  cascading operations
• sql_mode (String) sql_mode to use when
  interacting with the server.
  Defaults to NO_AUTO_VALUE_ON_ZERO,NO_DIR_IN_CREATE,NO_ENGINE_SUBSTITUTION
• timezone (String) Database timezone

Is used for parsing timestamps for saving source timezones. Accepts values from
IANA timezone database. Default: local timezone.

• user (String) User for database access. Required unless connection manager connection is used

Nested Schema for Nested Schema for settings.mysql_target.connection

Read-Only:

• connection_manager_connection (Attributes List) Get Mysql installation params and credentials from Connection Manager (see below for nested schema)
• mdb_cluster_id (String) Managed Service for MySQL cluster ID
• on_premise (Attributes List) Connection options for on-premise MySQL (see below for nested schema)

Nested Schema for Nested Schema for settings.mysql_target.connection.connection_manager_connection

Read-Only:

• connection_id (String) ID of connection in Connection Manager with installation params and credetials
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.mysql_target.connection.on_premise

Read-Only:

• hosts (List of String) List of host names of the MySQL server. Exactly one host is expected
• port (Number) Port for the database connection
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet
• tls_mode (Attributes List) TLS settings for server connection. Disabled by default. (see below for nested schema)

Nested Schema for Nested Schema for settings.mysql_target.connection.on_premise.tls_mode

Read-Only:

• disabled (Attributes List) Empty block designating that the connection is not secured, i.e. plaintext
  connection (see below for nested schema)
• enabled (Attributes List) TLS is used for the server connection (see below for nested schema)

Nested Schema for Nested Schema for settings.mysql_target.connection.on_premise.tls_mode.disabled

Nested Schema for Nested Schema for settings.mysql_target.connection.on_premise.tls_mode.enabled

Read-Only:

• ca_certificate (String) CA certificate

X.509 certificate of the certificate authority which issued the server's
certificate, in PEM format. When CA certificate is specified, TLS is used to
connect to the server. If CA certificate is empty, the server's certificate must
be signed by a well-known CA

Nested Schema for Nested Schema for settings.mysql_target.password

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.postgres_source

Read-Only:

• connection (Attributes List) Database connection settings (see below for nested schema)
• database (String) Name of the database to transfer
• exclude_tables (List of String) List of tables which will not be transfered, formatted as schemaname.tablename
  If omitted or empty list is specified, all tables are replicated.
  Can contain schema_name.* patterns.
• include_tables (List of String) List of tables to transfer, formatted as schemaname.tablename.
  If omitted or an empty list is specified, all tables will be transferred.
  Can contain schema_name.* patterns.
• object_transfer_settings (Attributes List) Defines which database schema objects should be transferred, e.g. views,
  functions, etc.
  All of the attributes in this block are optional and should be either
  BEFORE_DATA, AFTER_DATA or NEVER (see below for nested schema)
• password (Attributes List) Password for database access. (see below for nested schema)
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• service_schema (String) Name of the database schema in which auxiliary tables needed for the transfer
  will be created (__consumer_keeper, __data_transfer_mole_finder).
  Empty service_schema implies schema public
• slot_gigabyte_lag_limit (Number) Maximum WAL size held by the replication slot (API - in bytes, terraform - in
  gigabytes);
  Exceeding this limit will result in a replication failure and deletion of the
  replication slot.
  Default is 50 gigabytes
• user (String) User for database access. Required unless Connection Manager connection is used.

Nested Schema for Nested Schema for settings.postgres_source.connection

Read-Only:

• connection_manager_connection (Attributes List) Get Postgres installation params and credentials from Connection Manager (see below for nested schema)
• mdb_cluster_id (String) Managed Service for PostgreSQL cluster ID
• on_premise (Attributes List) Connection options for on-premise PostgreSQL (see below for nested schema)

Nested Schema for Nested Schema for settings.postgres_source.connection.connection_manager_connection

Read-Only:

• connection_id (String) ID of connection in Connection Manager with installation params and credetials
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.postgres_source.connection.on_premise

Read-Only:

• hosts (List of String) PG installation hosts
• port (Number) PG port. Will be used if the cluster ID is not specified.
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet
• tls_mode (Attributes List) TLS settings for server connection. Disabled by default. (see below for nested schema)

Nested Schema for Nested Schema for settings.postgres_source.connection.on_premise.tls_mode

Read-Only:

• disabled (Attributes List) Empty block designating that the connection is not secured, i.e. plaintext
  connection (see below for nested schema)
• enabled (Attributes List) TLS is used for the server connection (see below for nested schema)

Nested Schema for Nested Schema for settings.postgres_source.connection.on_premise.tls_mode.disabled

Nested Schema for Nested Schema for settings.postgres_source.connection.on_premise.tls_mode.enabled

Read-Only:

• ca_certificate (String) CA certificate

X.509 certificate of the certificate authority which issued the server's
certificate, in PEM format. When CA certificate is specified, TLS is used to
connect to the server. If CA certificate is empty, the server's certificate must
be signed by a well-known CA

Nested Schema for Nested Schema for settings.postgres_source.object_transfer_settings

Read-Only:

• cast (String) Casts

CREATE CAST ...

• collation (String) Collations

CREATE COLLATION ...

• constraint (String) Constraints

ALTER TABLE ... ADD CONSTRAINT ...

• default_values (String) Default values

ALTER TABLE ... ALTER COLUMN ... SET DEFAULT ...

• fk_constraint (String) Foreign keys

ALTER TABLE ... ADD FOREIGN KEY ...

• function (String) Functions

CREATE FUNCTION ...

• index (String) Indexes

CREATE INDEX ...

• materialized_view (String) Materialized views

CREATE MATERIALIZED VIEW ...

• policy (String) Policies

CREATE POLICY ...

• primary_key (String) Primary keys

ALTER TABLE ... ADD PRIMARY KEY ...

• rule (String) Rules

CREATE RULE ...

• sequence (String) Sequences

CREATE SEQUENCE ...

• sequence_owned_by (String) Owned sequences

CREATE SEQUENCE ... OWNED BY ...

• sequence_set (String) package: yandex.cloud.datatransfer.v1.endpoint
  filename: yandex/cloud/datatransfer/v1/endpoint/postgres.proto
• table (String) Tables

CREATE TABLE ...

• trigger (String) Triggers

CREATE TRIGGER ...

• type (String) Types

CREATE TYPE ...

• view (String) Views

CREATE VIEW ...

Nested Schema for Nested Schema for settings.postgres_source.password

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.postgres_target

Read-Only:

• cleanup_policy (String) Cleanup policy for activate, reactivate and reupload processes.
  One of: DISABLED, DROP, TRUNCATE. Default is TRUNCATE
• connection (Attributes List) Database connection settings (see below for nested schema)
• database (String) Target database name
• is_schema_migration_disabled (Boolean) Whether can change table schema if schema changed on source
• password (Attributes List) Password for database access. (see below for nested schema)
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• user (String) User for database access. Required unless Connection Manager connection is used

Nested Schema for Nested Schema for settings.postgres_target.connection

Read-Only:

• connection_manager_connection (Attributes List) Get Postgres installation params and credentials from Connection Manager (see below for nested schema)
• mdb_cluster_id (String) Managed Service for PostgreSQL cluster ID
• on_premise (Attributes List) Connection options for on-premise PostgreSQL (see below for nested schema)

Nested Schema for Nested Schema for settings.postgres_target.connection.connection_manager_connection

Read-Only:

• connection_id (String) ID of connection in Connection Manager with installation params and credetials
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.postgres_target.connection.on_premise

Read-Only:

• hosts (List of String) PG installation hosts
• port (Number) PG port. Will be used if the cluster ID is not specified.
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet
• tls_mode (Attributes List) TLS settings for server connection. Disabled by default. (see below for nested schema)

Nested Schema for Nested Schema for settings.postgres_target.connection.on_premise.tls_mode

Read-Only:

• disabled (Attributes List) Empty block designating that the connection is not secured, i.e. plaintext
  connection (see below for nested schema)
• enabled (Attributes List) TLS is used for the server connection (see below for nested schema)

Nested Schema for Nested Schema for settings.postgres_target.connection.on_premise.tls_mode.disabled

Nested Schema for Nested Schema for settings.postgres_target.connection.on_premise.tls_mode.enabled

Read-Only:

• ca_certificate (String) CA certificate

X.509 certificate of the certificate authority which issued the server's
certificate, in PEM format. When CA certificate is specified, TLS is used to
connect to the server. If CA certificate is empty, the server's certificate must
be signed by a well-known CA

Nested Schema for Nested Schema for settings.postgres_target.password

Read-Only:

• raw (String, Sensitive) Raw secret value

Nested Schema for Nested Schema for settings.ydb_source

Optional:

• service_account_id (String) Service account ID for interaction with database

Read-Only:

• changefeed_custom_consumer_name (String) Consumer for pre-created change feed if any
• changefeed_custom_name (String) Pre-created change feed if any
• database (String) Database path in YDB where tables are stored.
  Example: /ru/transfer_manager/prod/data-transfer-yt
• instance (String) Instance of YDB. example: ydb-ru-prestable.yandex.net:2135.
  If not specified, will be determined by database
• paths (List of String) A list of paths which should be uploaded. When not specified, all available
  tables are uploaded
• sa_key_content (String, Sensitive) Authorization Key
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database. If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.ydb_target

Optional:

• service_account_id (String) Service account ID for interaction with database

Read-Only:

• cleanup_policy (String) Cleanup policy determine how to clean collections when activating the transfer.
  One of YDB_CLEANUP_POLICY_DISABLED or YDB_CLEANUP_POLICY_DROP
• database (String) Database path in YDB where tables are stored.
  Example: /ru/transfer_manager/prod/data-transfer
• default_compression (String) Compression that will be used for default columns family on YDB table creation.
  One of YDB_DEFAULT_COMPRESSION_UNSPECIFIED,
  YDB_DEFAULT_COMPRESSION_DISABLED, YDB_DEFAULT_COMPRESSION_LZ4
• instance (String) Instance of YDB. example: ydb-ru-prestable.yandex.net:2135.
  If not specified, will be determined by database
• is_schema_migration_disabled (Boolean) Whether can change table schema if schema changed on source
• is_table_column_oriented (Boolean) Whether a column-oriented (i.e. OLAP) tables should be created.
  Default is false (create row-oriented OLTP tables)
• path (String) Path extension for database, each table will be layouted into this path
• sa_key_content (String, Sensitive) Authentication key
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database.
  If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.yds_source

Optional:

• service_account_id (String) Service account ID which has read access to the stream.

Read-Only:

• allow_ttl_rewind (Boolean) Should continue working, if consumer read lag exceed TTL of topic
  False: stop the transfer in error state, if detected lost data. True: continue
  working with losing part of data
• consumer (String) Custom consumer - for important streams
• database (String) Database path in YDB for streams
  Example: /ru/transfer_manager/prod/data-transfer
• endpoint (String) YDS Endpoint for dedicated db
• parser (Attributes List) Data parsing rules (see below for nested schema)
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• stream (String) Stream to read
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database. If omitted, the server has to be accessible via Internet
• supported_codecs (List of String) List of supported compression codecs
  Options: YDS_COMPRESSION_CODEC_RAW, YDS_COMPRESSION_CODEC_ZSTD,
  YDS_COMPRESSION_CODEC_GZIP

Nested Schema for Nested Schema for settings.yds_source.parser

Read-Only:

• audit_trails_v1_parser (Attributes List) Parse Audit Trails data. Empty struct (see below for nested schema)
• cloud_logging_parser (Attributes List) Parse Cloud Logging data. Empty struct (see below for nested schema)
• json_parser (Attributes List) Parse data in json format (see below for nested schema)
• tskv_parser (Attributes List) Parse data in tskv format (see below for nested schema)

Nested Schema for Nested Schema for settings.yds_source.parser.audit_trails_v1_parser

Nested Schema for Nested Schema for settings.yds_source.parser.cloud_logging_parser

Nested Schema for Nested Schema for settings.yds_source.parser.json_parser

Read-Only:

• add_rest_column (Boolean) Will add _rest column for all unknown fields
• data_schema (Attributes List) Data parsing scheme (see below for nested schema)
• null_keys_allowed (Boolean) Allow null keys, if no - null keys will be putted to unparsed data
• unescape_string_values (Boolean) Unescape string values

Nested Schema for Nested Schema for settings.yds_source.parser.tskv_parser.data_schema

Read-Only:

• fields (Attributes List) Description of the data schema in the array of fields structure (see below for nested schema)
• json_fields (String, Deprecated) Description of the data schema as JSON specification

Nested Schema for Nested Schema for settings.yds_source.parser.tskv_parser.data_schema.fields

Read-Only:

• fields (Attributes List) Description of the column schema in the array of fields structure (see below for nested schema)

Nested Schema for Nested Schema for settings.yds_source.parser.tskv_parser.data_schema.json_fields.fields

Read-Only:

• key (Boolean) Mark field as Primary Key
• name (String) Field name
• path (String) Path to the field
• required (Boolean) Mark field as required
• type (String) Field type, one of: INT64, INT32, INT16, INT8, UINT64, UINT32,
  UINT16, UINT8, DOUBLE, BOOLEAN, STRING, UTF8, ANY, DATETIME.

Nested Schema for Nested Schema for settings.yds_source.parser.tskv_parser

Read-Only:

• add_rest_column (Boolean) Will add _rest column for all unknown fields
• data_schema (Attributes List) Data parsing scheme (see below for nested schema)
• null_keys_allowed (Boolean) Allow null keys, if no - null keys will be putted to unparsed data
• unescape_string_values (Boolean) Unescape string values

Nested Schema for Nested Schema for settings.yds_source.parser.tskv_parser.data_schema

Read-Only:

• fields (Attributes List) Description of the data schema in the array of fields structure (see below for nested schema)
• json_fields (String, Deprecated) Description of the data schema as JSON specification

Nested Schema for Nested Schema for settings.yds_source.parser.tskv_parser.data_schema.fields

Read-Only:

• fields (Attributes List) Description of the column schema in the array of fields structure (see below for nested schema)

Nested Schema for Nested Schema for settings.yds_source.parser.tskv_parser.data_schema.json_fields.fields

Read-Only:

• key (Boolean) Mark field as Primary Key
• name (String) Field name
• path (String) Path to the field
• required (Boolean) Mark field as required
• type (String) Field type, one of: INT64, INT32, INT16, INT8, UINT64, UINT32,
  UINT16, UINT8, DOUBLE, BOOLEAN, STRING, UTF8, ANY, DATETIME.

Nested Schema for Nested Schema for settings.yds_target

Optional:

• service_account_id (String) Service account ID which has read access to the stream

Read-Only:

• compression_codec (String) Codec to use for output data compression. If not specified, no compression will
  be done
  Options: YDS_COMPRESSION_CODEC_RAW, YDS_COMPRESSION_CODEC_ZSTD,
  YDS_COMPRESSION_CODEC_GZIP
• database (String) Database path in YDB for streams
  Example: /ru/transfer_manager/prod/data-transfer
• endpoint (String) YDS Endpoint for dedicated db
• save_tx_order (Boolean) Save transaction order
  Not to split events queue into separate per-table queues.
  Incompatible with setting Topic prefix, only with Topic full name.
• security_groups (List of String) List of security groups that the transfer associated with this endpoint should
  use
• serializer (Attributes List) Data serialization format (see below for nested schema)
• stream (String) Stream to write to
• subnet_id (String) Identifier of the Yandex Cloud VPC subnetwork to user for accessing the
  database. If omitted, the server has to be accessible via Internet

Nested Schema for Nested Schema for settings.yds_target.serializer

Read-Only:

• serializer_auto (Attributes List) Empty block. Select the serialization format automatically (see below for nested schema)
• serializer_debezium (Attributes List) Serialize data in debezium json format (see below for nested schema)
• serializer_json (Attributes List) Empty block. Serialize data in json format (see below for nested schema)

Nested Schema for Nested Schema for settings.yds_target.serializer.serializer_auto

Nested Schema for Nested Schema for settings.yds_target.serializer.serializer_debezium

Read-Only:

• serializer_parameters (Attributes List) A list of Debezium parameters set by the structure of the key and value
  string fields (see below for nested schema)

Nested Schema for Nested Schema for settings.yds_target.serializer.serializer_json.serializer_parameters

Read-Only:

• key (String) Name of the serializer parameter
• value (String) Value of the serializer parameter

Nested Schema for Nested Schema for settings.yds_target.serializer.serializer_json