Managed Spark API, gRPC: ClusterService.Start

Written by

Yandex Cloud

Updated at April 14, 2025

gRPC request
StartClusterRequest
operation.Operation
StartClusterMetadata
Cluster
ClusterConfig
ResourcePools
ResourcePool
ScalePolicy
FixedScale
AutoScale
HistoryServerConfig
Dependencies
Metastore
NetworkConfig
LoggingConfig
UILink
MaintenanceWindow
AnytimeMaintenanceWindow
WeeklyMaintenanceWindow
MaintenanceOperation

Start the specified Spark cluster.

gRPC request

rpc Start (StartClusterRequest) returns (operation.Operation)

StartClusterRequest

{
  "cluster_id": "string"
}

Field

Description

cluster_id

string

Required field. ID of the Spark cluster that is being started.

operation.Operation

{
  "id": "string",
  "description": "string",
  "created_at": "google.protobuf.Timestamp",
  "created_by": "string",
  "modified_at": "google.protobuf.Timestamp",
  "done": "bool",
  "metadata": {
    "cluster_id": "string"
  },
  // Includes only one of the fields `error`, `response`
  "error": "google.rpc.Status",
  "response": {
    "id": "string",
    "folder_id": "string",
    "created_at": "google.protobuf.Timestamp",
    "name": "string",
    "description": "string",
    "labels": "map<string, string>",
    "config": {
      "resource_pools": {
        "driver": {
          "resource_preset_id": "string",
          "scale_policy": {
            // Includes only one of the fields `fixed_scale`, `auto_scale`
            "fixed_scale": {
              "size": "int64"
            },
            "auto_scale": {
              "min_size": "int64",
              "max_size": "int64"
            }
            // end of the list of possible fields
          }
        },
        "executor": {
          "resource_preset_id": "string",
          "scale_policy": {
            // Includes only one of the fields `fixed_scale`, `auto_scale`
            "fixed_scale": {
              "size": "int64"
            },
            "auto_scale": {
              "min_size": "int64",
              "max_size": "int64"
            }
            // end of the list of possible fields
          }
        }
      },
      "history_server": {
        "enabled": "bool"
      },
      "dependencies": {
        "pip_packages": [
          "string"
        ],
        "deb_packages": [
          "string"
        ]
      },
      "metastore": {
        "cluster_id": "string"
      }
    },
    "status": "Status",
    "network": {
      "subnet_ids": [
        "string"
      ],
      "security_group_ids": [
        "string"
      ]
    },
    "deletion_protection": "bool",
    "service_account_id": "string",
    "logging": {
      "enabled": "bool",
      // Includes only one of the fields `folder_id`, `log_group_id`
      "folder_id": "string",
      "log_group_id": "string"
      // end of the list of possible fields
    },
    "health": "Health",
    "links": [
      {
        "name": "string",
        "url": "string"
      }
    ],
    "maintenance_window": {
      // Includes only one of the fields `anytime`, `weekly_maintenance_window`
      "anytime": "AnytimeMaintenanceWindow",
      "weekly_maintenance_window": {
        "day": "WeekDay",
        "hour": "int64"
      }
      // end of the list of possible fields
    },
    "planned_operation": {
      "info": "string",
      "delayed_until": "google.protobuf.Timestamp",
      "latest_maintenance_time": "google.protobuf.Timestamp",
      "next_maintenance_window_time": "google.protobuf.Timestamp"
    }
  }
  // end of the list of possible fields
}

An Operation resource. For more information, see Operation.

Field	Description
id	string ID of the operation.
description	string Description of the operation. 0-256 characters long.
created_at	google.protobuf.Timestamp Creation timestamp.
created_by	string ID of the user or service account who initiated the operation.
modified_at	google.protobuf.Timestamp The time when the Operation resource was last modified.
done	bool If the value is `false`, it means the operation is still in progress. If `true`, the operation is completed, and either `error` or `response` is available.
metadata	StartClusterMetadata Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any.
error	google.rpc.Status The error result of the operation in case of failure or cancellation. Includes only one of the fields `error`, `response`. The operation result. If `done == false` and there was no failure detected, neither `error` nor `response` is set. If `done == false` and there was a failure detected, `error` is set. If `done == true`, exactly one of `error` or `response` is set.
response	Cluster The normal response of the operation in case of success. If the original method returns no data on success, such as Delete, the response is google.protobuf.Empty. If the original method is the standard Create/Update, the response should be the target resource of the operation. Any method that returns a long-running operation should document the response type, if any. Includes only one of the fields `error`, `response`. The operation result. If `done == false` and there was no failure detected, neither `error` nor `response` is set. If `done == false` and there was a failure detected, `error` is set. If `done == true`, exactly one of `error` or `response` is set.

StartClusterMetadata

Field

Description

cluster_id

string

ID of the Spark cluster.

Cluster

Spark cluster.

Field	Description
id	string Required field. Unique ID of the Spark cluster. This ID is assigned by Cloud in the process of creating a Spark cluster.
folder_id	string ID of the folder that the Spark cluster belongs to.
created_at	google.protobuf.Timestamp The time when the Spark cluster was created.
name	string Name of the Spark cluster. The name is unique within the folder. 1-64 characters long.
description	string Description of the Spark cluster. 0-256 characters long.
labels	object (map<string, string>)
config	ClusterConfig Configuration of the Spark cluster.
status	enum Status Cluster status. `STATUS_UNKNOWN`: Cluster status is unknown `CREATING`: Cluster is being created `RUNNING`: Cluster is running normally. `UPDATING`: Cluster is being updated. `ERROR`: Cluster encountered a problem and cannot operate. `STOPPING`: Cluster is stopping. `STOPPED`: Cluster is stopped. `STARTING`: Cluster is starting.
network	NetworkConfig
deletion_protection	bool Deletion Protection inhibits deletion of the cluster
service_account_id	string Service account that will be used to access a YC resources
logging	LoggingConfig Cloud logging configuration.
health	enum Health Aggregated cluster health. `HEALTH_UNKNOWN`: Cluster is in unknown state (we have no data). `ALIVE`: Cluster is alive and well. `DEAD`: Cluster is inoperable (it cannot perform any of its essential functions). `DEGRADED`: Cluster is partially alive (it can perform some of its essential functions).
links[]	UILink UI URLs
maintenance_window	MaintenanceWindow Window of maintenance operations.
planned_operation	MaintenanceOperation Maintenance operation planned at nearest maintenance_window.

ClusterConfig

Field	Description
resource_pools	ResourcePools Required field.
history_server	HistoryServerConfig Configuration for HistoryServer
dependencies	Dependencies Container custom environment dependencies
metastore	Metastore Metastore Cluster

ResourcePools

Field

Description

driver

ResourcePool

Required field.

executor

ResourcePool

Required field.

ResourcePool

Field

Description

resource_preset_id

string

Required field. ID of the preset for computational resources allocated to a instance (e.g., CPU, memory, etc.).

scale_policy

ScalePolicy

Required field.

ScalePolicy

Field

Description

fixed_scale

FixedScale

Includes only one of the fields fixed_scale, auto_scale.

auto_scale

AutoScale

Includes only one of the fields fixed_scale, auto_scale.

FixedScale

Field	Description
size	int64

AutoScale

Field	Description
min_size	int64
max_size	int64

HistoryServerConfig

Field	Description
enabled	bool

Dependencies

Field	Description
pip_packages[]	string
deb_packages[]	string

Metastore

Field	Description
cluster_id	string

NetworkConfig

Field

Description

subnet_ids[]

string

IDs of VPC network subnets where instances of the cluster are attached.

security_group_ids[]

string

User security groups

LoggingConfig

Field	Description
enabled	bool
folder_id	string Includes only one of the fields `folder_id`, `log_group_id`.
log_group_id	string Includes only one of the fields `folder_id`, `log_group_id`.

UILink

Field	Description
name	string
url	string

MaintenanceWindow

Field

Description

anytime

AnytimeMaintenanceWindow

Includes only one of the fields anytime, weekly_maintenance_window.

weekly_maintenance_window

WeeklyMaintenanceWindow

Includes only one of the fields anytime, weekly_maintenance_window.

AnytimeMaintenanceWindow

Field	Description
Empty

WeeklyMaintenanceWindow

Field

Description

day

enum WeekDay

WEEK_DAY_UNSPECIFIED
MON
TUE
WED
THU
FRI
SAT
SUN

hour

int64

Hour of the day in UTC.

MaintenanceOperation

Field	Description
info	string
delayed_until	google.protobuf.Timestamp
latest_maintenance_time	google.protobuf.Timestamp
next_maintenance_window_time	google.protobuf.Timestamp

Managed Spark API, gRPC: ClusterService.Start

gRPC requestgRPC request

StartClusterRequestStartClusterRequest

operation.Operationoperation.Operation

StartClusterMetadataStartClusterMetadata

ClusterCluster

ClusterConfigClusterConfig

ResourcePoolsResourcePools

ResourcePoolResourcePool

ScalePolicyScalePolicy

FixedScaleFixedScale

AutoScaleAutoScale

HistoryServerConfigHistoryServerConfig

DependenciesDependencies

MetastoreMetastore

NetworkConfigNetworkConfig

LoggingConfigLoggingConfig

UILinkUILink

MaintenanceWindowMaintenanceWindow

AnytimeMaintenanceWindowAnytimeMaintenanceWindow

WeeklyMaintenanceWindowWeeklyMaintenanceWindow

MaintenanceOperationMaintenanceOperation

Was the article helpful?

gRPC request

StartClusterRequest

operation.Operation

StartClusterMetadata

Cluster

ClusterConfig

ResourcePools

ResourcePool

ScalePolicy

FixedScale

AutoScale

HistoryServerConfig

Dependencies

Metastore

NetworkConfig

LoggingConfig

UILink

MaintenanceWindow

AnytimeMaintenanceWindow

WeeklyMaintenanceWindow

MaintenanceOperation