Connecting to component interfaces
You can connect to Yandex Data Processing component interfaces either using UI Proxy or a jump host VM. For more information, see Component interfaces and ports.
UI Proxy
Warning
You may need to additionally set up security groups to use UI Proxy.
Enabling component interfaces
- Go to the folder page
and select Yandex Data Processing. - Select a cluster and click
Edit in the top panel. - Under Configuration, enable the UI Proxy option.
- Click Save changes.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To enable access to cluster component web interfaces, set true
for the --ui-proxy
parameter:
yc dataproc cluster update <cluster_name_or_ID> \
--ui-proxy=true
You can get the cluster ID and name with a list of clusters in the folder.
Use the update API method and include the following in the request:
- Cluster ID in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. true
in theuiProxy
parameter.- List of updatable cluster configuration fields in the
updateMask
parameter (in this case,uiProxy
).
Warning
This API method overrides all parameters of the object being modified that were not explicitly passed in the request to the default values. To avoid this, list the settings you want to change in the updateMask
parameter (one line separated by commas).
Retrieving a list of connection URLs
- Go to the folder page
and select Yandex Data Processing. - Click the cluster name.
- Connection links for the component web interfaces can be found under UI Proxy.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To retrieve a list of URLs to connect to Yandex Data Processing cluster component web interfaces, run the following command:
yc dataproc cluster list-ui-links <cluster_name_or_ID>
You can get the cluster ID and name with a list of clusters in the folder.
Use the listUILinks API method and include the cluster ID in the clusterId
request parameter.
You can get the cluster ID with a list of clusters in the folder.
Port forwarding
To access a component's network interface from the internet, create a jump host VM in Yandex Compute Cloud.
Requirements for a jump host VM:
- An assigned public IP address.
- Hosted in the same network as the required Yandex Data Processing cluster.
- Security group settings that allow traffic exchange with the cluster via the corresponding components' ports.
For step-by-step instructions on how to configure security groups for port forwarding, see Configuring security groups.
To connect to the Yandex Data Processing host port you need, run the following command:
ssh -A \
-J <VM_user_name>@<VM_public_IP_address> \
-L <port_number>:<Yandex_Data_Processing_host_FQDN>:<port_number> <username>@<Yandex_Data_Processing_host_FQDN>
Where:
-
-A
: Activates connection redirects from the authentication agent with the jump host VM to the target host of the Yandex Data Processing cluster. -
-J
: Connection to the target host via a jump host VM. Establishes a connection over SSH to the jump host VM that will redirect packages to the target host in the Yandex Data Processing cluster. -
-L
: Local port redirection to the Yandex Data Processing cluster host.To connect to cluster hosts with image version 1.x, use
root
for username; for version 2.x, useubuntu
.
You can find the FQDN of the Yandex Data Processing host on the Yandex Data Processing cluster page, in the Hosts tab, in the Host FQDN column.
Port numbers for Yandex Data Processing components are shown under Component interfaces and ports.