Written by

🚧 [Experimental] List data transfers

Returns transfers in the specified collection or the current DataLens tenant.

Request

POST

https://api.datalens.tech/rpc/listDataTransfers

Headers

Name

Description

x-dl-api-version

Type: string

API version header.

Const: 3

Example: 3

Body

application/json

{
  "collectionId": "example",
  "pageSize": "example",
  "pageToken": "example",
  "filter": [
    "example"
  ]
}

Name	Description
collectionId	Type: string Collection to list. When omitted, lists transfers in the current DataLens tenant. Min length: `1` Example: `example`
filter	Type: string[] Filtering is not supported; only an empty array is accepted. Max items: `0` Example `[ "example" ]`
pageSize	Type: string Maximum number of transfers, encoded as a string. The default is 100. Pattern: `^\d+$` Example: `example`
pageToken	Type: string Next-page token from the preceding response. Example: `example`

Responses

200 OK

Response

Body

application/json

{
  "transfers": [
    {
      "id": "example",
      "collectionId": "example",
      "cloudEnvironmentId": "example",
      "name": "example",
      "description": "example",
      "createdAt": {
        "seconds": "example",
        "nanos": 0.5
      },
      "createdById": "example",
      "updatedAt": {
        "seconds": "example",
        "nanos": 0.5
      },
      "updatedById": "example",
      "status": "TRANSFER_STATUS_UNSPECIFIED",
      "labels": {},
      "config": {
        "source": {},
        "destination": {
          "endpoint": "icebergNamespace",
          "icebergNamespace": {}
        },
        "type": {},
        "tables": [
          "example"
        ]
      }
    }
  ],
  "nextPageToken": "example"
}

Name

Description

nextPageToken

Type: string

Token for the next page of transfers.

Example: example

transfers

Type: DataTransfer[]

Transfers matching the request.

Example

[
  {
    "id": "example",
    "collectionId": "example",
    "cloudEnvironmentId": "example",
    "name": "example",
    "description": "example",
    "createdAt": {
      "seconds": "example",
      "nanos": 0.5
    },
    "createdById": "example",
    "updatedAt": {
      "seconds": "example",
      "nanos": 0.5
    },
    "updatedById": "example",
    "status": "TRANSFER_STATUS_UNSPECIFIED",
    "labels": {},
    "config": {
      "source": {
        "endpoint": "postgresConnection",
        "postgresConnection": {}
      },
      "destination": {
        "endpoint": "icebergNamespace",
        "icebergNamespace": {
          "namespaceId": "example"
        }
      },
      "type": {
        "type": "regular",
        "regular": {}
      },
      "tables": [
        "example"
      ]
    }
  }
]

DataTransferIncrementalTable

Name	Description
cursorColumn	Type: string Column used to detect new and changed rows. Example: `example`
initialState	Type: string Initial cursor value for the first transfer run. Example: `example`
tableName	Type: string Name of the source table. Example: `example`
tableNamespace	Type: string Namespace or schema of the source table. Example: `example`

Example

{
  "tableNamespace": "example",
  "tableName": "example",
  "cursorColumn": "example",
  "initialState": "example"
}

DataTransfer

Name

Description

cloudEnvironmentId

Type: string

ID of the associated cloud environment.

Example: example

collectionId

Type: string

ID of the DataLens collection that contains the transfer.

Example: example

config

Type: object

destination

Type: object

endpoint

Type: string

Destination endpoint type.

Const: icebergNamespace

Example: example

icebergNamespace

Type: object

namespaceId

Type: string

ID of the destination Iceberg namespace.

Example: example

Iceberg destination namespace settings.

Example

{
  "namespaceId": "example"
}

Destination settings.

Example

{
  "endpoint": "icebergNamespace",
  "icebergNamespace": {
    "namespaceId": "example"
  }
}

source

One of 3 types

Type: object

endpoint

Type: string

Source endpoint type.

Const: postgresConnection

Example: example

postgresConnection

Type: object

connectionId

Type: string

ID of the source DataLens connection.

Example: example

database

Type: string

Source database name.

Example: example

PostgreSQL source connection settings.

Example

{
  "connectionId": "example",
  "database": "example"
}

Example

{
  "endpoint": "postgresConnection",
  "postgresConnection": {
    "connectionId": "example",
    "database": "example"
  }
}

Type: object

clickhouseConnection

Type: object

connectionId

Type: string

ID of the source DataLens connection.

Example: example

database

Type: string

Source database name.

Example: example

ClickHouse source connection settings.

Example

{
  "connectionId": "example",
  "database": "example"
}

endpoint

Type: string

Source endpoint type.

Const: clickhouseConnection

Example: example

Example

{
  "endpoint": "clickhouseConnection",
  "clickhouseConnection": {
    "connectionId": "example",
    "database": "example"
  }
}

Type: object

endpoint

Type: string

Source endpoint type.

Const: mysqlConnection

Example: example

mysqlConnection

Type: object

connectionId

Type: string

ID of the source DataLens connection.

Example: example

database

Type: string

Source database name.

Example: example

MySQL source connection settings.

Example

{
  "connectionId": "example",
  "database": "example"
}

Example

{
  "endpoint": "mysqlConnection",
  "mysqlConnection": {
    "connectionId": "example",
    "database": "example"
  }
}

Source connection settings.

Example

{
  "endpoint": "postgresConnection",
  "postgresConnection": {
    "connectionId": "example",
    "database": "example"
  }
}

tables

Type: string[]

Source tables to transfer.

Example

[
  "example"
]

type

One of 2 types

Type: object

regular

Type: object

cronExpression

Type: string

Five-field cron expression in UTC.

Example: example

tables

Type: DataTransferIncrementalTable[]

Incremental snapshot settings.

Example

[
  {
    "tableNamespace": "example",
    "tableName": "example",
    "cursorColumn": "example",
    "initialState": "example"
  }
]

Regular transfer settings.

Example

{
  "cronExpression": "example",
  "tables": [
    {
      "tableNamespace": "example",
      "tableName": "example",
      "cursorColumn": "example",
      "initialState": "example"
    }
  ]
}

type

Type: string

Transfer mode.

Const: regular

Example: example

Example

{
  "type": "regular",
  "regular": {
    "cronExpression": "example",
    "tables": [
      {
        "tableNamespace": "example",
        "tableName": "example",
        "cursorColumn": "example",
        "initialState": "example"
      }
    ]
  }
}

Type: object

oneTime

Type: object

One-time transfer settings.

Example

{}

type

Type: string

Transfer mode.

Const: oneTime

Example: example

Example

{
  "type": "oneTime",
  "oneTime": {}
}

Transfer mode and its settings.

Example

{
  "type": "regular",
  "regular": {
    "cronExpression": "example",
    "tables": [
      {
        "tableNamespace": "example",
        "tableName": "example",
        "cursorColumn": "example",
        "initialState": "example"
      }
    ]
  }
}

Transfer configuration.

Example

{
  "source": {
    "endpoint": "postgresConnection",
    "postgresConnection": {
      "connectionId": "example",
      "database": "example"
    }
  },
  "destination": {
    "endpoint": "icebergNamespace",
    "icebergNamespace": {
      "namespaceId": "example"
    }
  },
  "type": {
    "type": "regular",
    "regular": {
      "cronExpression": "example",
      "tables": [
        {}
      ]
    }
  },
  "tables": [
    "example"
  ]
}

createdAt

Type: object

seconds

Type: string

Number of seconds since the Unix epoch.

Example: example

nanos

Type: number

Fractional seconds in nanoseconds.

Time when the transfer was created.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

createdById

Type: string

ID of the user who created the transfer.

Example: example

description

Type: string

Description of the transfer.

Example: example

Type: string

ID of the transfer.

Example: example

labels

Type: object

[additional]

Type: string

Example: example

Transfer labels.

Example

{}

name

Type: string

Name of the transfer.

Example: example

status

Type: string

Current status of the transfer.

Enum: TRANSFER_STATUS_UNSPECIFIED, CREATING, CREATED, RUNNING, STOPPING, STOPPED, ERROR, SNAPSHOTTING, DONE, PAUSED, PREPARING, UNKNOWN

updatedAt

Type: object

seconds

Type: string

Number of seconds since the Unix epoch.

Example: example

nanos

Type: number

Fractional seconds in nanoseconds.

Time when the transfer was last updated.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

updatedById

Type: string

ID of the user who last updated the transfer.

Example: example

Example

{
  "id": "example",
  "collectionId": "example",
  "cloudEnvironmentId": "example",
  "name": "example",
  "description": "example",
  "createdAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "createdById": "example",
  "updatedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "updatedById": "example",
  "status": "TRANSFER_STATUS_UNSPECIFIED",
  "labels": {},
  "config": {
    "source": {
      "endpoint": "postgresConnection",
      "postgresConnection": {
        "connectionId": "example",
        "database": "example"
      }
    },
    "destination": {
      "endpoint": "icebergNamespace",
      "icebergNamespace": {
        "namespaceId": "example"
      }
    },
    "type": {
      "type": "regular",
      "regular": {
        "cronExpression": "example",
        "tables": [
          null
        ]
      }
    },
    "tables": [
      "example"
    ]
  }
}

🚧 [Experimental] List data transfers

RequestRequest

HeadersHeaders

BodyBody

ResponsesResponses

200 OK200 OK

BodyBody

DataTransferIncrementalTableDataTransferIncrementalTable

DataTransferDataTransfer

Was the article helpful?

Request

Headers

Body

Responses

200 OK

Body

DataTransferIncrementalTable

DataTransfer