yandex_spark_cluster (Data Source)

Managed Spark cluster.

Example usageExample usage

//
// Get information about Apache Spark cluster by name
//
data "yandex_spark_cluster" "spark_cluster_by_name" {
  name = "spark-created-with-terraform"
}

//
// Get information about Apache Spark cluster by id
//
data "yandex_spark_cluster" "spark_cluster_by_id" {
  id = "<spark-cluster-id>"
}

SchemaSchema

OptionalOptional

• folder_id (String) ID of the cloud folder that the cluster belongs to.
• id (String) Unique ID of the cluster.
• name (String) Name of the cluster. The name is unique within the folder.
• timeouts (Block, Optional) (see below for nested schema)

Read-OnlyRead-Only

• config (Attributes) Configuration of the Spark cluster. (see below for nested schema)
• created_at (String) The timestamp when the cluster was created.
• deletion_protection (Boolean) The true value means that resource is protected from accidental deletion.
• description (String) Description of the cluster. 0-256 characters long.
• labels (Map of String) Cluster labels as key/value pairs.
• logging (Attributes) Cloud Logging configuration. (see below for nested schema)
• maintenance_window (Attributes) Configuration of the window for maintenance operations. (see below for nested schema)
• network (Attributes) Network configuration. (see below for nested schema)
• service_account_id (String) The service account used by the cluster to access cloud resources.
• status (String) Status of the cluster.

Nested Schema for Nested Schema for timeouts

Optional:

• read (String) A string that can be parsed as a duration consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours). Read operations occur during any refresh or planning operation when refresh is enabled.

Nested Schema for Nested Schema for config

Read-Only:

• dependencies (Attributes) Environment dependencies. (see below for nested schema)
• history_server (Attributes) History Server configuration. (see below for nested schema)
• metastore (Attributes) Metastore configuration. (see below for nested schema)
• resource_pools (Attributes) Computational resources. (see below for nested schema)

Nested Schema for Nested Schema for config.dependencies

Read-Only:

• deb_packages (Set of String) Deb-packages that need to be installed using system package manager.
• pip_packages (Set of String) Python packages that need to be installed using pip (in pip requirement format).

Nested Schema for Nested Schema for config.history_server

Read-Only:

• enabled (Boolean) Enable Spark History Server.

Nested Schema for Nested Schema for config.metastore

Read-Only:

• cluster_id (String) Metastore cluster ID for default spark configuration.

Nested Schema for Nested Schema for config.resource_pools

Read-Only:

• driver (Attributes) Computational resources for the driver pool. (see below for nested schema)
• executor (Attributes) Computational resources for the executor pool. (see below for nested schema)

Nested Schema for Nested Schema for config.resource_pools.driver

Read-Only:

• max_size (Number) Maximum node count for the driver pool with autoscaling.
• min_size (Number) Minimum node count for the driver pool with autoscaling.
• resource_preset_id (String) Resource preset ID for the driver pool.
• size (Number) Node count for the driver pool with fixed size.

Nested Schema for Nested Schema for config.resource_pools.executor

Read-Only:

• max_size (Number) Maximum node count for the executor pool with autoscaling.
• min_size (Number) Minimum node count for the executor pool with autoscaling.
• resource_preset_id (String) Resource preset ID for the executor pool.
• size (Number) Node count for the executor pool with fixed size.

Nested Schema for Nested Schema for logging

Read-Only:

• enabled (Boolean) Enable log delivery to Cloud Logging.
• folder_id (String) Logs will be written to default log group of specified folder. Exactly one of the attributes folder_id or log_group_id should be specified.
• log_group_id (String) Logs will be written to the specified log group. Exactly one of the attributes folder_id or log_group_id should be specified.

Nested Schema for Nested Schema for maintenance_window

Read-Only:

• day (String) Day of week for maintenance window. One of MON, TUE, WED, THU, FRI, SAT, SUN.
• hour (Number) Hour of day in UTC time zone (1-24) for maintenance window.
• type (String) Type of maintenance window. Can be either ANYTIME or WEEKLY. If WEEKLY, day and hour must be specified.

Nested Schema for Nested Schema for network

Read-Only:

• security_group_ids (Set of String) Network security groups.
• subnet_ids (Set of String) Network subnets.