Written by

🚧 [Experimental] Get Spark cluster

Returns the specified Spark cluster.

Request

POST

https://api.datalens.tech/rpc/getSparkCluster

Headers

Name

Description

x-dl-api-version

Type: string

API version header.

Const: 3

Example: 3

Body

application/json

{
  "id": "example"
}

Name

Description

Type: string

ID of the Spark cluster to return.

Min length: 1

Max length: 50

Example: example

Responses

200 OK

Response

Body

application/json

{
  "id": "example",
  "clusterId": "example",
  "collectionId": "example",
  "cloudEnvironmentId": "example",
  "name": "example",
  "description": "example",
  "labels": {},
  "config": {
    "sparkVersion": "example",
    "resourcePools": {
      "driver": {
        "resourcePresetId": "example",
        "scalePolicy": {}
      },
      "executor": {
        "resourcePresetId": "example",
        "scalePolicy": {}
      }
    },
    "dependencies": {
      "pipPackages": [
        "example"
      ],
      "debPackages": [
        "example"
      ]
    },
    "logging": {
      "enabled": true
    }
  },
  "health": "HEALTH_UNKNOWN",
  "status": "STATUS_UNSPECIFIED",
  "entryId": "example"
}

Name

Description

cloudEnvironmentId

Type: string

ID of the associated cloud environment.

Min length: 1

Example: example

clusterId

Type: string

ID of the managed Spark cluster.

Example: example

collectionId

Type: string

ID of the DataLens collection that contains the cluster.

Min length: 1

Example: example

config

Type: object

dependencies

Type: object | null

Packages installed in the cluster.

Example

{
  "pipPackages": [
    "example"
  ],
  "debPackages": [
    "example"
  ]
}

logging

Type: object | null

Cluster logging configuration.

Example

{
  "enabled": true
}

resourcePools

Type: object

driver

Type: object

resourcePresetId

Type: string

ID of the Spark resource preset.

Min length: 1

Max length: 50

Example: example

scalePolicy

Any of 2 types

Type: object

fixedScale

Type: object

size

Type: string

Number of instances in the resource pool.

Pattern: ^(?:[1-9]|[1-9]\d|100)$

Example: example

Example

{
  "size": "example"
}

scaleType

Type: string

Const: fixedScale

Example: example

Example

{
  "fixedScale": {
    "size": "example"
  },
  "scaleType": "fixedScale"
}

Type: object

autoScale

Type: object

initialSize

Type: string

Initial number of instances in the resource pool.

Pattern: ^(?:0|[1-9]\d?|100)$

Example: example

maxSize

Type: string

Maximum number of instances in the resource pool.

Pattern: ^(?:[1-9]|[1-9]\d|100)$

Example: example

minSize

Type: string

Minimum number of instances in the resource pool.

Pattern: ^(?:0|[1-9]\d?|100)$

Example: example

Example

{
  "minSize": "example",
  "maxSize": "example",
  "initialSize": "example"
}

scaleType

Type: string

Const: autoScale

Example: example

Example

{
  "autoScale": {
    "minSize": "example",
    "maxSize": "example",
    "initialSize": "example"
  },
  "scaleType": "autoScale"
}

Resource pool scaling policy.

Example

{
  "fixedScale": {
    "size": "example"
  },
  "scaleType": "fixedScale"
}

Driver resource pool.

Example

{
  "resourcePresetId": "example",
  "scalePolicy": {
    "fixedScale": {
      "size": "example"
    },
    "scaleType": "fixedScale"
  }
}

executor

Type: object

resourcePresetId

Type: string

ID of the Spark resource preset.

Min length: 1

Max length: 50

Example: example

scalePolicy

Any of 2 types

Type: object

fixedScale

Type: object

size

Type: string

Number of instances in the resource pool.

Pattern: ^(?:[1-9]|[1-9]\d|100)$

Example: example

Example

{
  "size": "example"
}

scaleType

Type: string

Const: fixedScale

Example: example

Example

{
  "fixedScale": {
    "size": "example"
  },
  "scaleType": "fixedScale"
}

Type: object

autoScale

Type: object

initialSize

Type: string

Initial number of instances in the resource pool.

Pattern: ^(?:0|[1-9]\d?|100)$

Example: example

maxSize

Type: string

Maximum number of instances in the resource pool.

Pattern: ^(?:[1-9]|[1-9]\d|100)$

Example: example

minSize

Type: string

Minimum number of instances in the resource pool.

Pattern: ^(?:0|[1-9]\d?|100)$

Example: example

Example

{
  "minSize": "example",
  "maxSize": "example",
  "initialSize": "example"
}

scaleType

Type: string

Const: autoScale

Example: example

Example

{
  "autoScale": {
    "minSize": "example",
    "maxSize": "example",
    "initialSize": "example"
  },
  "scaleType": "autoScale"
}

Resource pool scaling policy.

Example

{
  "fixedScale": {
    "size": "example"
  },
  "scaleType": "fixedScale"
}

Executor resource pool.

Example

{
  "resourcePresetId": "example",
  "scalePolicy": {
    "fixedScale": {
      "size": "example"
    },
    "scaleType": "fixedScale"
  }
}

Example

{
  "driver": {
    "resourcePresetId": "example",
    "scalePolicy": {
      "fixedScale": {
        "size": "example"
      },
      "scaleType": "fixedScale"
    }
  },
  "executor": {
    "resourcePresetId": "example",
    "scalePolicy": {
      "fixedScale": {
        "size": "example"
      },
      "scaleType": "fixedScale"
    }
  }
}

sparkVersion

Type: string

Spark version used by the cluster.

Example: example

Example

{
  "sparkVersion": "example",
  "resourcePools": {
    "driver": {
      "resourcePresetId": "example",
      "scalePolicy": {
        "fixedScale": {},
        "scaleType": "fixedScale"
      }
    },
    "executor": {
      "resourcePresetId": "example",
      "scalePolicy": {
        "fixedScale": {},
        "scaleType": "fixedScale"
      }
    }
  },
  "dependencies": {
    "pipPackages": [
      "example"
    ],
    "debPackages": [
      "example"
    ]
  },
  "logging": {
    "enabled": true
  }
}

description

Type: string

Description of the Spark cluster.

Example: example

entryId

Type: string

ID of the DataLens entry of the Spark cluster. Empty when the cluster has no entry.

Example: example

health

Type: string

Aggregated health of the Spark cluster.

Enum: HEALTH_UNKNOWN, ALIVE, DEAD, DEGRADED

Type: string

ID of the Spark cluster.

Min length: 1

Example: example

labels

Type: object

[additional]

Type: string

Example: example

Spark cluster labels.

Example

{}

name

Type: string

Name of the Spark cluster.

Min length: 1

Example: example

status

Type: string

Current status of the Spark cluster.

Enum: STATUS_UNSPECIFIED, CREATING, RUNNING, UPDATING, ERROR, STOPPING, STOPPED, STARTING

🚧 [Experimental] Get Spark cluster

RequestRequest

HeadersHeaders

BodyBody

ResponsesResponses

200 OK200 OK

BodyBody

Was the article helpful?

Request

Headers

Body

Responses

200 OK

Body