Connecting to component interfaces
You can connect to Yandex Data Proc component interfaces either using UI Proxy or an intermediate virtual machine. For more information, see Component interfaces and ports.
UI Proxy
Warning
You may need to additionally set up security groups to use UI Proxy.
Enabling component interfaces
- Go to the folder page
and select Data Proc. - Select a cluster and click
Edit cluster on the top panel. - Under Configuration, enable the UI Proxy option.
- Click Save changes.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To enable access to cluster component web interfaces, select true
for the --ui-proxy
parameter:
yc dataproc cluster update <cluster_name_or_ID> \
--ui-proxy=true
You can get the cluster ID and name with a list of clusters in the folder.
Use the update API method and provide the following in the request:
- Cluster ID in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. true
value in theuiProxy
parameter.- List of cluster configuration fields to update in the
updateMask
parameter (in this case,uiProxy
).
Warning
This API method overrides all parameters of the object being modified that were not explicitly passed in the request to the default values. To avoid this, list the settings you want to change in the updateMask
parameter (one line separated by commas).
Retrieving a list of connection URLs
- Go to the folder page
and select Data Proc. - Click the cluster name.
- Connection links for the component web interfaces can be found under UI Proxy.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To retrieve a list of URLs to connect to Yandex Data Proc cluster component web interfaces, run the following command:
yc dataproc cluster list-ui-links <cluster_name_or_ID>
You can get the cluster ID and name with a list of clusters in the folder.
Use the listUILinks API method and include the cluster ID in the clusterId
request parameter.
You can get the cluster ID with a list of clusters in the folder.
Port forwarding
To access a component's network interface from the internet, create an intermediate virtual machine in Yandex Compute Cloud.
Requirements for an intermediate VM:
- An assigned public IP address.
- Hosted in the same network as the required Yandex Data Proc cluster.
- Security group settings that allow traffic exchange with the cluster via the corresponding components' ports.
For step-by-step guides on how to configure security groups for port forwarding, see Configuring security groups.
To connect to the Yandex Data Proc host port you need, run the following command:
ssh -A \
-J <VM_public_IP_address> \
-L <port_number>:<Data_Proc_host_FQDN>:<port_number> <username>@<Data_Proc_host_FQDN>
Where:
-
-A
: Activates connection redirects from the authentication agent on the jump host VM to the Yandex Data Proc cluster host. -
-J
: Use jump host VM to connect to target host. Connect over SSH to the jump host VM that will redirect packages to the target host in the Yandex Data Proc cluster. -
-L
: Redirect the local port to the Yandex Data Proc cluster host.To connect to cluster hosts with 1.x as the image version, use
root
as your username, for version 2.x, useubuntu
.
You can find the FQDN of the Yandex Data Proc host on the Yandex Data Proc cluster page, in the Hosts tab, in the Host FQDN column.
Port numbers for Yandex Data Proc components are shown under Component interfaces and ports.