Connecting to component interfaces
You can connect to the Yandex Data Processing component interfaces either using UI Proxy or a jump host VM. For more information, see Component interfaces and ports.
UI Proxy
Warning
For UI Proxy to work correctly, you need to establish network connectivity between your device, the cluster master node, and the UI Proxy servers.
Make sure your device and the cluster master node are allowed outbound access to these service IP addresses:
158.160.167.1702a0d:d6c1:0:1c::23e
Set up security groups if required.
Enabling component interfaces
- Open the folder dashboard
. - Navigate to Yandex Data Processing.
- Select the cluster and click
Edit in the top panel. - Under Configuration, enable UI Proxy.
- Click Save changes.
If you do not have the Yandex Cloud CLI yet, install and initialize it.
The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.
To enable access to cluster component web interfaces, set the --ui-proxy parameter to true:
yc dataproc cluster update <cluster_name_or_ID> \
--ui-proxy=true
You can get the cluster ID and name with the list of clusters in the folder.
Use the update API method and provide the following in the request:
- Cluster ID in the
clusterIdparameter. To find out the cluster ID, get the list of clusters in the folder. truein theuiProxyparameter.- List of cluster configuration fields to update, in the
updateMaskparameter (in this case,uiProxy).
Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.
Getting a list of connection URLs
- Open the folder dashboard
. - Navigate to Yandex Data Processing.
- Click the cluster name.
- You can find links for connecting to the component web interfaces under UI Proxy.
If you do not have the Yandex Cloud CLI yet, install and initialize it.
The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.
To get a list of URLs for connecting to the web interfaces of the Yandex Data Processing cluster components, run the following command:
yc dataproc cluster list-ui-links <cluster_name_or_ID>
You can get the cluster ID and name with the list of clusters in the folder.
Call the listUILinks API method, providing the cluster ID in the clusterId request parameter.
You can get the cluster ID with the list of clusters in the folder.
Port forwarding
To access a component's network interface from the internet, create a jump host VM in Yandex Compute Cloud.
Requirements for a jump host VM:
- It must have a public IP address assigned.
- It must be hosted in the same network as your Yandex Data Processing cluster.
- Security group settings must allow traffic exchange with the cluster via the ports of the components in question.
Learn more about configuring security groups for connections with port forwarding here.
To connect to the Yandex Data Processing host port you need, run the following command:
ssh -A \
-J <VM_user_name>@<VM_public_IP_address> \
-L <port_number>:<Yandex_Data_Processing_host_FQDN>:<port_number> <username>@<Yandex_Data_Processing_host_FQDN>
Where:
-
-A: Enables forwarding the authentication agent connection from the jump host VM to the target host in the Yandex Data Processing cluster. -
-J: Connects to the target host through the jump host: establishes an SSH connection to the jump host VM, which forwards traffic to the target host in the Yandex Data Processing cluster. -
-L: Forwards a local port to the target host in the Yandex Data Processing cluster.To connect to cluster hosts with image 1.x, use
rootas the username; for version 2.x, useubuntu.
You can find the Yandex Data Processing host’s FQDN in the Hosts tab of the Yandex Data Processing cluster page, in the Host FQDN column.
For Yandex Data Processing component port numbers, see Component interfaces and ports.