Yandex Cloud
Search
Contact UsGet started
  • Blog
  • Pricing
  • Documentation
  • All Services
  • System Status
    • Featured
    • Infrastructure & Network
    • Data Platform
    • Containers
    • Developer tools
    • Serverless
    • Security
    • Monitoring & Resources
    • ML & AI
    • Business tools
  • All Solutions
    • By industry
    • By use case
    • Economics and Pricing
    • Security
    • Technical Support
    • Customer Stories
    • Cloud credits to scale your IT product
    • Gateway to Russia
    • Cloud for Startups
    • Education and Science
    • Yandex Cloud Partner program
  • Blog
  • Pricing
  • Documentation
© 2025 Direct Cursus Technology L.L.C.
Yandex Data Processing
  • Getting started
    • Resource relationships
    • Runtime environment
    • Yandex Data Processing component interfaces and ports
    • Jobs in Yandex Data Processing
    • Spark jobs
    • Automatic scaling
    • Decommissioning subclusters and hosts
    • Networking in Yandex Data Processing
    • Maintenance
    • Quotas and limits
    • Storage in Yandex Data Processing
    • Component properties
    • Apache Iceberg™ in Yandex Data Processing
    • Delta Lake in Yandex Data Processing
    • Logs in Yandex Data Processing
    • Initialization scripts
  • Access management
  • Pricing policy
  • Terraform reference
  • Monitoring metrics
  • Audit Trails events
  • Public materials
  • FAQ
  1. Concepts
  2. Jobs in Yandex Data Processing

Jobs in Yandex Data Processing

Written by
Yandex Cloud
Updated at February 18, 2025

In a Yandex Data Processing cluster, you can create and run jobs. This allows you to regularly upload datasets from Object Storage buckets, use them in calculations, and generate analytics.

The following job types are supported:

  • Hive
  • MapReduce
  • PySpark
  • Spark

When creating a job, specify:

  • Arguments: Values used by the job's main executable file.
  • Properties: The key:value pairs that configure image components.

To create and start jobs, you can:

  • Use the Yandex Cloud interfaces. For more information, see basic examples for working with jobs.

  • Connect directly to the cluster node. For more information, see the example in the Running jobs from remote hosts that are not part of the cluster section.

To successfully run a job:

  • Grant access to the required Object Storage buckets for the cluster service account.

    We recommend using at least two buckets:

    • One with read-only permissions for storing the source data and files required to run the job.
    • Another one with read and write permissions for storing job run results. Specify it when creating a cluster.
  • When creating a job, provide all files required for it.

If there are enough computing resources in the cluster, the jobs you created will be running concurrently; otherwise, a job queue will be formed.

Job logsJob logs

Job logs are saved in Yandex Cloud Logging. For more information, see Working with logs.

Was the article helpful?

Previous
Yandex Data Processing component interfaces and ports
Next
Spark jobs
© 2025 Direct Cursus Technology L.L.C.