Connecting to component interfaces
You can connect to Yandex Data Processing component interfaces either using UI Proxy or a jump host VM. For more information, see Component interfaces and ports.
UI Proxy
Warning
You may need to additionally set up security groups to use UI Proxy.
Enabling component interfaces
- Go to the folder page and select Yandex Data Processing.
- Select a cluster and click Edit in the top panel.
- Under Configuration, enable the UI Proxy option.
- Click Save changes.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.
To enable access to cluster component web interfaces, set true for the --ui-proxy parameter:
yc dataproc cluster update <cluster_name_or_ID> \
--ui-proxy=true
You can get the cluster ID and name with a list of clusters in the folder.
Use the update API method and include the following in the request:
- Cluster ID, in the
clusterIdparameter. To find out the cluster ID, get a list of clusters in the folder. truein theuiProxyparameter.- List of updatable cluster configuration fields in the
updateMaskparameter (in this case,uiProxy).
Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.
Retrieving a list of connection URLs
- Go to the folder page and select Yandex Data Processing.
- Click the cluster name.
- Connection links for the component web interfaces can be found under UI Proxy.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.
To retrieve a list of URLs to connect to Yandex Data Processing cluster component web interfaces, run the following command:
yc dataproc cluster list-ui-links <cluster_name_or_ID>
You can get the cluster ID and name with a list of clusters in the folder.
Use the listUILinks API method and include the cluster ID in the clusterId request parameter.
You can get the cluster ID with a list of clusters in the folder.
Port forwarding
To access a component's network interface from the internet, create an intermediate virtual machine in Yandex Compute Cloud.
Requirements for a jump host VM:
- An assigned public IP address.
- Hosted in the same network as the required Yandex Data Processing cluster.
- Security group settings that allow traffic exchange with the cluster via the corresponding components' ports.
For step-by-step instructions on how to configure security groups for port forwarding, see Configuring security groups.
To connect to the Yandex Data Processing host port you need, run the following command:
ssh -A \
-J <VM_user_name>@<VM_public_IP_address> \
-L <port_number>:<Yandex_Data_Processing_host_FQDN>:<port_number> <username>@<Yandex_Data_Processing_host_FQDN>
Where:
-
-A: Activates connection redirects from the authentication agent with the jump host VM to the target host of the Yandex Data Processing cluster. -
-J: Connection to the target host via a jump host VM. Establishes a connection over SSH to the jump host VM that will redirect packages to the target host in the Yandex Data Processing cluster. -
-L: Local port redirection to the Yandex Data Processing cluster host.To connect to cluster hosts with image version 1.x, use
rootfor username; for version 2.x, useubuntu.
You can find the FQDN of the Yandex Data Processing host on the Yandex Data Processing cluster page, in the Hosts tab, in the Host FQDN column.
Port numbers for Yandex Data Processing components are shown under Component interfaces and ports.