Connecting to a Yandex Data Processing host over SSH

You can connect to a Yandex Data Processing cluster over SSH.

Before connecting:

1. Configure security groups.
2. Get the host FQDN.

Connect using a standard SSH clientConnect using a standard SSH client

To connect to a Yandex Data Processing host, make sure the SSH key you specified when creating the Yandex Data Processing cluster is accessible on the local machine or interim VM. You can copy the key to the machine that will connect to the cluster, or connect to it with an SSH agent.

1. If you do not have public access configured for your subcluster, connect to the interim VM over SSH.

2. Start the SSH agent:

   eval `ssh-agent -s`
   

3. Add the required key to the list of those available to the agent:

   ssh-add ~/.ssh/example-key
   

4. Open a SSH connection to the Yandex Data Processing host by specifying its FQDN or IP address if public access is enabled for the host. For image version 2.0, specify the ubuntu user; for image version 1.4, root, e.g.:

   ssh ubuntu@rc1b-dataproc-m-fh4y4nur********.mdb.yandexcloud.net
   

   Result:

   ubuntu@rc1b-dataproc-m-fh4y4nur********:~#
   

5. Make sure Hadoop commands are executed, e.g.:

   hadoop version
   

   Result:

   Hadoop 2.8.5
   Subversion https://github.yandex-team.ru/mdb/bigtop.git -r 78508f2a4b4f3dc8b3d295ccb50a45a4********
   Compiled by robot-pgaas-ci on 2019-04-16T10:35Z
   Compiled with protoc 2.5.0
   From source with checksum 9942ca5c745417c14e31883********
   This command was run using /usr/lib/hadoop/hadoop-common-2.8.5.jar