Written by

🚧 [Experimental] Get Spark application

Returns the specified Spark application.

Request

POST

https://api.datalens.tech/rpc/getSparkApplication

Headers

Name

Description

x-dl-api-version

Type: string

API version header.

Const: 3

Example: 3

Body

application/json

{
  "clusterId": "example",
  "applicationId": "example"
}

Name

Description

applicationId

Type: string

ID of the Spark application to return.

Min length: 1

Max length: 50

Example: example

clusterId

Type: string

ID of the Spark cluster.

Min length: 1

Max length: 50

Example: example

Responses

200 OK

Response

Body

application/json

{
  "id": "example",
  "clusterId": "example",
  "createdAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "startedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "finishedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "name": "example",
  "createdBy": "example",
  "status": "STATUS_UNSPECIFIED",
  "connectUrl": "example",
  "catalogs": [
    {
      "catalogId": "example"
    }
  ],
  "sparkApplication": {
    "args": [
      "example"
    ],
    "jarFileUris": [
      "example"
    ],
    "fileUris": [
      "example"
    ],
    "archiveUris": [
      "example"
    ],
    "properties": {},
    "mainJarFileUri": "example",
    "mainClass": "example",
    "packages": [
      "example"
    ],
    "repositories": [
      "example"
    ],
    "excludePackages": [
      "example"
    ]
  },
  "applicationSpec": "sparkApplication"
}

Any of 4 types

Type: object

applicationSpec

Type: string

Const: sparkApplication

Example: example

catalogs

Type: object[]

catalogId

Type: string

ID of the REST catalog.

Min length: 1

Max length: 50

Example: example

REST catalogs attached to the Spark application.

Example

[
  {
    "catalogId": "example"
  }
]

clusterId

Type: string

ID of the Spark cluster.

Min length: 1

Max length: 50

Example: example

connectUrl

Type: string

Spark Connect URL.

Example: example

createdAt

Type: object

seconds

Type: string

Number of seconds since the Unix epoch.

Example: example

nanos

Type: number

Fractional seconds in nanoseconds.

Time when the Spark application was created.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

createdBy

Type: string

ID of the user who created the Spark application.

Min length: 1

Example: example

finishedAt

Type: object | null

Time when the Spark application was finished.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

Type: string

ID of the Spark application.

Min length: 1

Max length: 50

Example: example

name

Type: string

Name of the Spark application.

Example: example

sparkApplication

Type: object

archiveUris

Type: string[]

Archives extracted in the Spark working directory.

Max items: 100

Example

[
  "example"
]

args

Type: string[]

Arguments passed to the Spark driver.

Max items: 100

Example

[
  "example"
]

excludePackages

Type: string[]

Maven packages excluded during dependency resolution.

Max items: 100

Example

[
  "example"
]

fileUris

Type: string[]

Files copied to the Spark working directory.

Max items: 100

Example

[
  "example"
]

jarFileUris

Type: string[]

JAR files added to the Spark classpaths.

Max items: 100

Example

[
  "example"
]

mainClass

Type: string

Name of the driver main class.

Max length: 255

Example: example

mainJarFileUri

Type: string

URI of the JAR file that contains the main class.

Min length: 1

Max length: 2047

Example: example

packages

Type: string[]

Maven packages added to the Spark classpaths.

Max items: 100

Example

[
  "example"
]

properties

Type: object

[additional]

Type: string

Max length: 256

Example: example

Spark configuration properties.

Example

{}

repositories

Type: string[]

Additional Maven repositories.

Max items: 10

Example

[
  "example"
]

Example

{
  "args": [
    "example"
  ],
  "jarFileUris": [
    "example"
  ],
  "fileUris": [
    "example"
  ],
  "archiveUris": [
    "example"
  ],
  "properties": {},
  "mainJarFileUri": "example",
  "mainClass": "example",
  "packages": [
    "example"
  ],
  "repositories": [
    "example"
  ],
  "excludePackages": [
    "example"
  ]
}

startedAt

Type: object | null

Time when the Spark application was started.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

status

Type: string

Current status of the Spark application.

Enum: STATUS_UNSPECIFIED, PROVISIONING, PENDING, RUNNING, ERROR, DONE, CANCELLED, CANCELLING

Example

{
  "id": "example",
  "clusterId": "example",
  "createdAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "startedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "finishedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "name": "example",
  "createdBy": "example",
  "status": "STATUS_UNSPECIFIED",
  "connectUrl": "example",
  "catalogs": [
    {
      "catalogId": "example"
    }
  ],
  "sparkApplication": {
    "args": [
      "example"
    ],
    "jarFileUris": [
      "example"
    ],
    "fileUris": [
      "example"
    ],
    "archiveUris": [
      "example"
    ],
    "properties": {},
    "mainJarFileUri": "example",
    "mainClass": "example",
    "packages": [
      "example"
    ],
    "repositories": [
      "example"
    ],
    "excludePackages": [
      "example"
    ]
  },
  "applicationSpec": "sparkApplication"
}

Type: object

applicationSpec

Type: string

Const: pysparkApplication

Example: example

catalogs

Type: object[]

catalogId

Type: string

ID of the REST catalog.

Min length: 1

Max length: 50

Example: example

REST catalogs attached to the Spark application.

Example

[
  {
    "catalogId": "example"
  }
]

clusterId

Type: string

ID of the Spark cluster.

Min length: 1

Max length: 50

Example: example

connectUrl

Type: string

Spark Connect URL.

Example: example

createdAt

Type: object

seconds

Type: string

Number of seconds since the Unix epoch.

Example: example

nanos

Type: number

Fractional seconds in nanoseconds.

Time when the Spark application was created.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

createdBy

Type: string

ID of the user who created the Spark application.

Min length: 1

Example: example

finishedAt

Type: object | null

Time when the Spark application was finished.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

Type: string

ID of the Spark application.

Min length: 1

Max length: 50

Example: example

name

Type: string

Name of the Spark application.

Example: example

pysparkApplication

Type: object

archiveUris

Type: string[]

Archives extracted in the Spark working directory.

Max items: 100

Example

[
  "example"
]

args

Type: string[]

Arguments passed to the PySpark driver.

Max items: 100

Example

[
  "example"
]

excludePackages

Type: string[]

Maven packages excluded during dependency resolution.

Max items: 100

Example

[
  "example"
]

fileUris

Type: string[]

Files copied to the Spark working directory.

Max items: 100

Example

[
  "example"
]

jarFileUris

Type: string[]

JAR files added to the Spark classpaths.

Max items: 100

Example

[
  "example"
]

mainPythonFileUri

Type: string

URI of the main Python file.

Min length: 1

Max length: 2047

Example: example

packages

Type: string[]

Maven packages added to the Spark classpaths.

Max items: 100

Example

[
  "example"
]

properties

Type: object

[additional]

Type: string

Max length: 256

Example: example

Spark configuration properties.

Example

{}

pythonFileUris

Type: string[]

Additional Python files passed to PySpark.

Max items: 100

Example

[
  "example"
]

repositories

Type: string[]

Additional Maven repositories.

Max items: 10

Example

[
  "example"
]

Example

{
  "args": [
    "example"
  ],
  "jarFileUris": [
    "example"
  ],
  "fileUris": [
    "example"
  ],
  "archiveUris": [
    "example"
  ],
  "properties": {},
  "mainPythonFileUri": "example",
  "pythonFileUris": [
    "example"
  ],
  "packages": [
    "example"
  ],
  "repositories": [
    "example"
  ],
  "excludePackages": [
    "example"
  ]
}

startedAt

Type: object | null

Time when the Spark application was started.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

status

Type: string

Current status of the Spark application.

Enum: STATUS_UNSPECIFIED, PROVISIONING, PENDING, RUNNING, ERROR, DONE, CANCELLED, CANCELLING

Example

{
  "id": "example",
  "clusterId": "example",
  "createdAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "startedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "finishedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "name": "example",
  "createdBy": "example",
  "status": "STATUS_UNSPECIFIED",
  "connectUrl": "example",
  "catalogs": [
    {
      "catalogId": "example"
    }
  ],
  "pysparkApplication": {
    "args": [
      "example"
    ],
    "jarFileUris": [
      "example"
    ],
    "fileUris": [
      "example"
    ],
    "archiveUris": [
      "example"
    ],
    "properties": {},
    "mainPythonFileUri": "example",
    "pythonFileUris": [
      "example"
    ],
    "packages": [
      "example"
    ],
    "repositories": [
      "example"
    ],
    "excludePackages": [
      "example"
    ]
  },
  "applicationSpec": "pysparkApplication"
}

Type: object

applicationSpec

Type: string

Const: sparkConnectApplication

Example: example

catalogs

Type: object[]

catalogId

Type: string

ID of the REST catalog.

Min length: 1

Max length: 50

Example: example

REST catalogs attached to the Spark application.

Example

[
  {
    "catalogId": "example"
  }
]

clusterId

Type: string

ID of the Spark cluster.

Min length: 1

Max length: 50

Example: example

connectUrl

Type: string

Spark Connect URL.

Example: example

createdAt

Type: object

seconds

Type: string

Number of seconds since the Unix epoch.

Example: example

nanos

Type: number

Fractional seconds in nanoseconds.

Time when the Spark application was created.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

createdBy

Type: string

ID of the user who created the Spark application.

Min length: 1

Example: example

finishedAt

Type: object | null

Time when the Spark application was finished.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

Type: string

ID of the Spark application.

Min length: 1

Max length: 50

Example: example

name

Type: string

Name of the Spark application.

Example: example

sparkConnectApplication

Type: object

archiveUris

Type: string[]

Archives extracted in the Spark working directory.

Max items: 100

Example

[
  "example"
]

excludePackages

Type: string[]

Maven packages excluded during dependency resolution.

Max items: 100

Example

[
  "example"
]

fileUris

Type: string[]

Files copied to the Spark working directory.

Max items: 100

Example

[
  "example"
]

jarFileUris

Type: string[]

JAR files added to the Spark classpaths.

Max items: 100

Example

[
  "example"
]

packages

Type: string[]

Maven packages added to the Spark classpaths.

Max items: 100

Example

[
  "example"
]

properties

Type: object

[additional]

Type: string

Max length: 256

Example: example

Spark configuration properties.

Example

{}

repositories

Type: string[]

Additional Maven repositories.

Max items: 10

Example

[
  "example"
]

Example

{
  "jarFileUris": [
    "example"
  ],
  "fileUris": [
    "example"
  ],
  "archiveUris": [
    "example"
  ],
  "properties": {},
  "packages": [
    "example"
  ],
  "repositories": [
    "example"
  ],
  "excludePackages": [
    "example"
  ]
}

startedAt

Type: object | null

Time when the Spark application was started.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

status

Type: string

Current status of the Spark application.

Enum: STATUS_UNSPECIFIED, PROVISIONING, PENDING, RUNNING, ERROR, DONE, CANCELLED, CANCELLING

Example

{
  "id": "example",
  "clusterId": "example",
  "createdAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "startedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "finishedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "name": "example",
  "createdBy": "example",
  "status": "STATUS_UNSPECIFIED",
  "connectUrl": "example",
  "catalogs": [
    {
      "catalogId": "example"
    }
  ],
  "sparkConnectApplication": {
    "jarFileUris": [
      "example"
    ],
    "fileUris": [
      "example"
    ],
    "archiveUris": [
      "example"
    ],
    "properties": {},
    "packages": [
      "example"
    ],
    "repositories": [
      "example"
    ],
    "excludePackages": [
      "example"
    ]
  },
  "applicationSpec": "sparkConnectApplication"
}

Type: object

catalogs

Type: object[]

catalogId

Type: string

ID of the REST catalog.

Min length: 1

Max length: 50

Example: example

REST catalogs attached to the Spark application.

Example

[
  {
    "catalogId": "example"
  }
]

clusterId

Type: string

ID of the Spark cluster.

Min length: 1

Max length: 50

Example: example

connectUrl

Type: string

Spark Connect URL.

Example: example

createdAt

Type: object

seconds

Type: string

Number of seconds since the Unix epoch.

Example: example

nanos

Type: number

Fractional seconds in nanoseconds.

Time when the Spark application was created.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

createdBy

Type: string

ID of the user who created the Spark application.

Min length: 1

Example: example

finishedAt

Type: object | null

Time when the Spark application was finished.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

Type: string

ID of the Spark application.

Min length: 1

Max length: 50

Example: example

name

Type: string

Name of the Spark application.

Example: example

startedAt

Type: object | null

Time when the Spark application was started.

Example

{
  "seconds": "example",
  "nanos": 0.5
}

status

Type: string

Current status of the Spark application.

Enum: STATUS_UNSPECIFIED, PROVISIONING, PENDING, RUNNING, ERROR, DONE, CANCELLED, CANCELLING

Example

{
  "id": "example",
  "clusterId": "example",
  "createdAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "startedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "finishedAt": {
    "seconds": "example",
    "nanos": 0.5
  },
  "name": "example",
  "createdBy": "example",
  "status": "STATUS_UNSPECIFIED",
  "connectUrl": "example",
  "catalogs": [
    {
      "catalogId": "example"
    }
  ]
}

🚧 [Experimental] Get Spark application

RequestRequest

HeadersHeaders

BodyBody

ResponsesResponses

200 OK200 OK

BodyBody

Was the article helpful?

Request

Headers

Body

Responses

200 OK

Body