Connecting to component interfaces
You can connect to Yandex Data Processing component interfaces either using UI Proxy or a jump host VM. For more information, see Component interfaces and ports.
UI Proxy
Warning
You may need to additionally set up security groups to use UI Proxy.
Enabling component interfaces
- Navigate to the folder dashboard
and select Yandex Data Processing. - Select the cluster and click
Edit in the top panel. - Under Configuration, enable UI Proxy.
- Click Save changes.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To enable access to cluster component web interfaces, set the --ui-proxy parameter to true:
yc dataproc cluster update <cluster_name_or_ID> \
--ui-proxy=true
You can get the cluster ID and name with the list of clusters in the folder.
Use the update API method and include the following in the request:
- Cluster ID in the
clusterIdparameter. To find out the cluster ID, get a list of clusters in the folder. truein theuiProxyparameter.- List of cluster configuration fields to update, in the
updateMaskparameter (in this case,uiProxy).
Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.
Retrieving a list of connection URLs
- Navigate to the folder dashboard
and select Yandex Data Processing. - Click the cluster name.
- You can find links for connecting to component web interfaces under UI Proxy.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To get a list of URLs for connecting to the web interfacess of the Yandex Data Processing cluster components, run the following command:
yc dataproc cluster list-ui-links <cluster_name_or_ID>
You can get the cluster ID and name with the list of clusters in the folder.
Use the listUILinks API method and include the cluster ID in the clusterId request parameter.
You can get the cluster ID with the list of clusters in the folder.
Port forwarding
To access a component's network interface from the internet, create an intermediate virtual machine in Yandex Compute Cloud.
Requirements for a jump host VM:
- It must have an assigned public IP address.
- It must be hosted in the same network as the required Yandex Data Processing cluster.
- Security group settings must allow traffic exchange with the cluster via the ports of the appropriate components.
Learn more about configuring security groups for connections with port forwarding here.
To connect to the Yandex Data Processing host port you need, run the following command:
ssh -A \
-J <VM_user_name>@<VM_public_IP_address> \
-L <port_number>:<Yandex_Data_Processing_host_FQDN>:<port_number> <username>@<Yandex_Data_Processing_host_FQDN>
Where:
-
-A: Activates connection redirects from the authentication agent with the jump host VM to the target host of the Yandex Data Processing cluster. -
-J: Connection to the target host via a jump host VM. Establishes a connection over SSH to the jump host VM that will redirect packages to the target host in the Yandex Data Processing cluster. -
-L: Local port redirection to the Yandex Data Processing cluster host.To connect to cluster hosts with image version 1.x, use
rootas the username; for version 2.x, useubuntu.
You can find the Yandex Data Processing host’s FQDN on the Yandex Data Processing cluster page, in the Hosts column of the Host FQDN tab.
You can find port numbers for Yandex Data Processing components under Component interfaces and ports.