Connecting resources from different folders
In Yandex Cloud, you create network resources, e.g., cloud networks and subnets, in separate folders disconnected from each other and you often need to connect them. You can do it using the Multi-folder VPC
method extending the scope of a VPC network to multiple folders.
Depending on the selected Yandex Cloud tools, you can extend your network scope to other folders by:
- Moving subnets to other folders using the
management console (UI)
andYC CLI
. - Creating subnets in target folders with
YC CLI
. - Creating subnets in target folders with
Terraform
.
Once you placed a subnet into the target folder, you can connect other folder resources to it, including VMs, Managed Service for Kubernetes clusters, database hosts, load balancers, load testing agents, etc. As a result, you will have a network connecting resources from different folders.
In this tutorial, we will set up three VMs residing in different folders and connected by a shared network. We will create a cloud network in one of these folders and then extend its scope to other folders, thus connecting the resources located there.
Warning
You can only move subnets between folders within a single cloud.
In our example, we have a dev environment, including the CI/CD module with its components located in the net-folder
. These components should be able to connect to other components located in dev, stage, and prod folders.
You can see this configuration in the picture below.
We are going to connect VMs residing in different subnets into one network. The VMs will be able to address each other by IP or FQDN addresses.
Steps to follow
Depending on the selected tools, steps to create Multi-folder VPC
may differ.
To create the test infrastructure and connect its resources:
- Get your cloud ready.
- Create folders.
- Create a VPC cloud network with subnets.
- Move the subnets.
- Create VM instances.
- Check the connectivity.
If you no longer need the resources you created, delete them.
Get your cloud ready
Sign up for Yandex Cloud and create a billing account:
- Go to the management console
and log in to Yandex Cloud or create an account if you do not have one yet. - On the Yandex Cloud Billing
page, make sure you have a billing account linked and it has theACTIVE
orTRIAL_ACTIVE
status. If you do not have a billing account, create one.
If you have an active billing account, you can go to the cloud page
Learn more about clouds and folders.
Required paid resources
The infrastructure support costs include:
- Fee for continuously running VMs (see Yandex Compute Cloud pricing).
- Fee for using public IP addresses and outgoing traffic (see Yandex Virtual Private Cloud pricing).
Configure access permissions
Configure access rights for the folder:
- To create networks and manage subnets, assign the
vpc.admin
or thevpc.privateAdmin
,vpc.publicAdmin
, andvpc.securityGroups.admin
roles to the service account or user. - To create and manage VMs, assign the
vpc.user
andcompute.admin
roles for the folder to the service account or user.
For granular network access management, use security groups.
Create folders
-
Create the
net-folder
,dev-folder
, andprod-folder
folders:Management consoleCLITerraformAPI- In the management console
, select a cloud and click Create folder. - Specify the folder name:
net-folder
. - Disable Create a default network to create your network and subnets manually.
- Click Create.
Similarly, create
dev-folder
andprod-folder
.If you do not have the Yandex Cloud CLI yet, install and initialize it.
Note
To create resources with CLI, get authenticated under a service account with the
admin
role for the cloud.-
Read the
create folder
command description:yc resource-manager folder create --help
-
Create the
net-folder
,dev-folder
, andprod-folder
cloud folders:yc resource-manager folder create --name net-folder yc resource-manager folder create --name dev-folder yc resource-manager folder create --name prod-folder
-
If you don't have Terraform, install it and configure the Yandex Cloud provider.
-
Specify the Yandex Cloud Terraform provider configuration:
# ================================== # Terraform & Provider Configuration # ================================== terraform { required_providers { yandex = { source = "yandex-cloud/yandex" version = "~> 0.92.0" } } }
-
Specify input variables:
variable "cloud_id" { description = "YC cloud-id. Taken from environment variable." }
-
Specify your target cloud folders:
# ======== # Folders # ======== resource "yandex_resourcemanager_folder" "net_folder" { cloud_id = var.cloud_id name = "net-folder" } resource "yandex_resourcemanager_folder" "dev_folder" { cloud_id = var.cloud_id name = "dev-folder" } resource "yandex_resourcemanager_folder" "prod_folder" { cloud_id = var.cloud_id name = "prod-folder" }
-
Create the required infrastructure:
-
Run the following commands:
export TF_VAR_cloud_id=$(yc config get cloud-id) export YC_TOKEN=$(yc iam create-token) terraform apply
-
When asked to confirm the changes, enter
yes
and wait for the operation to complete.
-
Use the create REST API method for the Folder resource or the FolderService/Create gRPC API call.
- In the management console
Create a VPC cloud network with subnets
In net-folder
, create the shared-net
network and three subnets with the following settings:
Subnet name | Prefix | Availability zone | Target folder |
---|---|---|---|
subnet-a |
10.1.11.0/24 |
ru-central1-a |
net-folder |
subnet-b |
10.1.12.0/24 |
ru-central1-b |
dev-folder |
subnet-d |
10.1.13.0/24 |
ru-central1-d |
prod-folder |
-
Create a cloud network:
Management consoleCLITerraformAPI- In the management console
, navigate tonet-folder
. - In the list of services, select Virtual Private Cloud.
- Click Create network.
- Specify the network name:
shared-net
. - Disable Create subnets to create subnets manually.
- Click Create network.
-
Read the description of the
network create
command:yc vpc network create --help
-
Create the
shared-net
cloud network innet-folder
.yc vpc network create --folder-name net-folder --name shared-net
-
Specify your target network:
# ============= # VPC Resources # ============= resource "yandex_vpc_network" "shared_net" { folder_id = yandex_resourcemanager_folder.net_folder.id name = "shared-net" }
-
Run the following commands:
export TF_VAR_cloud_id=$(yc config get cloud-id) export YC_TOKEN=$(yc iam create-token) terraform apply
-
Enter
yes
to confirm changes. -
Wait for the operation to complete.
Use the create REST API method for the Network resource or the NetworkService/Create gRPC API call.
- In the management console
-
Create subnets named
subnet-a
,subnet-b
, andsubnet-d
in theru-central1-a
,ru-central1-b
, andru-central1-d
availability zones, respectively:Management consoleCLITerraformAPI- In the management console
, navigate tonet-folder
. - In the list of services, select Virtual Private Cloud.
- Click
shared-net
. - Click Create subnet.
- Specify the subnet name:
subnet-a
,subnet-b
, orsubnet-d
. - Select the availability zone from the drop-down list:
ru-central1-a
,ru-central1-b
, orru-central1-d
, respectively. - Specify the subnet CIDR:
10.1.11.0
as IP address and24
as subnet mask. For more information about IP address ranges, see Cloud networks and subnets. - Click Create subnet.
-
Read the
subnet create
command description:yc vpc subnet create --help
-
Create subnets in the target folders:
yc vpc subnet create --folder-name net-folder --name subnet-a \ --network-name shared-net --zone ru-central1-a --range 10.1.11.0/24 yc vpc subnet create --folder-name dev-folder --name subnet-b \ --network-name shared-net --zone ru-central1-b --range 10.1.12.0/24 yc vpc subnet create --folder-name prod-folder --name subnet-d \ --network-name shared-net --zone ru-central1-d --range 10.1.13.0/24
-
Check the new subnet status:
yc vpc subnet list --folder-name net-folder yc vpc subnet list --folder-name dev-folder yc vpc subnet list --folder-name prod-folder
-
Specify your target subnets:
resource "yandex_vpc_subnet" "subnet_a" { folder_id = yandex_resourcemanager_folder.net_folder.id name = "subnet-a" description = "NET folder subnet" v4_cidr_blocks = ["10.1.11.0/24"] zone = "ru-central1-a" network_id = yandex_vpc_network.shared_net.id } resource "yandex_vpc_subnet" "subnet_b" { folder_id = yandex_resourcemanager_folder.dev_folder.id name = "subnet-b" description = "DEV folder subnet" v4_cidr_blocks = ["10.1.12.0/24"] zone = "ru-central1-b" network_id = yandex_vpc_network.shared_net.id } resource "yandex_vpc_subnet" "subnet_d" { folder_id = yandex_resourcemanager_folder.prod_folder.id name = "subnet-d" description = "PROD folder subnet" v4_cidr_blocks = ["10.1.13.0/24"] zone = "ru-central1-d" network_id = yandex_vpc_network.shared_net.id }
-
Run the following commands:
export TF_VAR_cloud_id=$(yc config get cloud-id) export YC_TOKEN=$(yc iam create-token) terraform apply
-
When asked to confirm the changes, enter
yes
and wait for the operation to complete.
Use the create REST API method for the Subnet resource or the SubnetService/Create gRPC API call.
- In the management console
Move the subnets
Move subnet-b
to dev-folder
:
- In the management console
, navigate tonet-folder
. - In the list of services, select Virtual Private Cloud.
- Click
shared-net
. - Click
next tosubnet-b
and select Move. - Select
dev-folder
from the drop-sown list. - Click Move.
-
Read the
subnet move
command description:yc vpc subnet move --help
-
Move the subnet:
yc vpc subnet move subnet-b \ --destination-folder-name dev-folder
Use the move REST API method for the Subnet resource or the SubnetService/Move gRPC API call.
Similarly, move subnet-d
to prod-folder
:
Create VMs
Create VMs with the following settings:
VM name | Folder | Availability zone | Subnet |
---|---|---|---|
net-vm |
net-folder |
ru-central1-a |
subnet-a |
dev-vm |
dev-folder |
ru-central1-b |
subnet-b |
prod-vm |
prod-folder |
ru-central1-d |
subnet-d |
Create the Linux-based net-vm
VM in net-folder
:
-
In the management console
, selectnet-folder
. -
Click Create resource and select
Virtual machine instance
. -
Under Boot disk image, in the Product search field, type
Ubuntu 22.04 LTS
and select a public Ubuntu 22.04 LTS image. -
Under Location, select the
ru-central1-a
availability zone. -
Under Network settings:
- In the Subnet field, select
subnet-a
. - Under Public IP address, leave
Auto
to assign your VM a random external IP address from the Yandex Cloud pool or if you reserved a static IP address, select it from the list.
- In the Subnet field, select
-
Under Access, select SSH key and specify your VM access credentials:
- In the Login field, specify the username:
ycuser
. -
In the SSH key field, select the SSH key saved in your organization user profile.
If there are no saved SSH keys in your profile, or you want to add a new key:
- Click Add key.
- Enter a name for the SSH key.
- Upload or paste the contents of the public key file. You need to create a key pair for the SSH connection to a VM yourself.
- Click Add.
The SSH key will be added to your organization user profile.
If users cannot add SSH keys to their profiles in the organization, the added public SSH key will only be saved to the user profile of the VM being created.
- In the Login field, specify the username:
-
Under General information, specify the VM name:
net-vm
. -
Leave all other settings unchanged and click Create VM.
Similarly, create dev-vm
and prod-vm
in the respective folders.
Note
When you create a VM, the system will assign it a public and private IP addresses Save them so you will be able to access the VM and test its connectivity.
-
Create the
vm-init.tpl
VM metadata template file:#cloud-config datasource: Ec2: strict_id: false ssh_pwauth: yes users: - name: "${USER_NAME}" sudo: ALL=(ALL) NOPASSWD:ALL shell: /bin/bash ssh_authorized_keys: - "${USER_SSH_KEY}"
-
Generate a VM metadata file:
export USER_NAME=ycuser export USER_SSH_KEY=$(cat ~/.ssh/id_rsa.pub) envsubst < vm-init.tpl > vm-config.txt
-
Create VMs:
yc compute instance create --name=net-vm --hostname=net-vm \ --zone=ru-central1-a \ --platform=standard-v3 \ --cores=2 --memory=4G --core-fraction=100 \ --create-boot-disk image-folder-id=standard-images,image-family=ubuntu-2204-lts \ --network-interface subnet-name=subnet-a,ipv4-address=auto,nat-ip-version=ipv4 \ --metadata-from-file user-data=vm-config.txt yc compute instance create --name=dev-vm --hostname=dev-vm \ --zone=ru-central1-b \ --platform=standard-v3 \ --cores=2 --memory=4G --core-fraction=100 \ --create-boot-disk image-folder-id=standard-images,image-family=ubuntu-2204-lts \ --network-interface subnet-name=subnet-b,ipv4-address=auto,nat-ip-version=ipv4 \ --metadata-from-file user-data=vm-config.txt yc compute instance create --name=prod-vm --hostname=prod-vm \ --zone=ru-central1-d \ --platform=standard-v3 \ --cores=2 --memory=4G --core-fraction=100 \ --create-boot-disk image-folder-id=standard-images,image-family=ubuntu-2204-lts \ --network-interface subnet-name=subnet-d,ipv4-address=auto,nat-ip-version=ipv4 \ --metadata-from-file user-data=vm-config.txt
Note
The commands
yc compute instance create
|create-with-container
|update
|add-metadata
support substitution of environment variable values into VM metadata. When you execute a Yandex Cloud CLI command, these values, specified in theuser-data
key in$<variable_name>
format, will be substituted into the VM metadata from the environment variables of the environment the command is executed in.To change such behavior, i.e. to provide a variable name to the VM metadata in
$<variable_name>
format rather than take the variable value from the CLI command runtime environment, use the two-dollar syntax, e.g.,$$<variable_name>
.For more information, see Specifics of providing environment variables in metadata via the CLI.
-
Save the new VMs’ public IPs as you will need them later:
NET_VM_IP=$(yc compute instance get net-vm --format=json | jq -r '.network_interfaces[0].primary_v4_address.one_to_one_nat.address') DEV_VM_IP=$(yc compute instance get dev-vm --format=json | jq -r '.network_interfaces[0].primary_v4_address.one_to_one_nat.address') PROD_VM_IP=$(yc compute instance get prod-vm --format=json | jq -r '.network_interfaces[0].primary_v4_address.one_to_one_nat.address')
-
Specify input variables:
variable "user_name" { description = "VM User Name" default = "ycuser" } variable "user_ssh_key_path" { description = "User's SSH public key file" default = "~/.ssh/id_rsa.pub" }
-
Create the
vm-init.tpl
VM metadata template file:#cloud-config datasource: Ec2: strict_id: false ssh_pwauth: yes users: - name: "${USER_NAME}" sudo: ALL=(ALL) NOPASSWD:ALL shell: /bin/bash ssh_authorized_keys: - "${USER_SSH_KEY}"
-
Describe your target VMs:
# ================= # Compute Resources # ================= data "yandex_compute_image" "vm_image" { family = "ubuntu-2204-lts" } resource "yandex_compute_disk" "boot-disk-1" { name = "boot-disk-1" type = "network-hdd" zone = "ru-central1-a" size = "20" image_id = yandex_compute_image.vm_image.id } resource "yandex_compute_disk" "boot-disk-2" { name = "boot-disk-2" type = "network-hdd" zone = "ru-central1-b" size = "20" image_id = yandex_compute_image.vm_image.id } resource "yandex_compute_disk" "boot-disk-3" { name = "boot-disk-3" type = "network-hdd" zone = "ru-central1-d" size = "20" image_id = yandex_compute_image.vm_image.id } resource "yandex_compute_instance" "net_vm" { folder_id = yandex_resourcemanager_folder.net_folder.id name = "net-vm" hostname = "net-vm" platform_id = "standard-v3" zone = "ru-central1-a" resources { cores = 2 memory = 4 } boot_disk { disk_id = yandex_compute_disk.boot-disk-1.id } network_interface { subnet_id = yandex_vpc_subnet.subnet_a.id nat = true } metadata = { user-data = templatefile("vm-init.tpl", { USER_NAME = var.user_name USER_SSH_KEY = file(var.user_ssh_key_path) }) } } resource "yandex_compute_instance" "dev_vm" { folder_id = yandex_resourcemanager_folder.dev_folder.id name = "dev-vm" hostname = "dev-vm" platform_id = "standard-v3" zone = "ru-central1-b" resources { cores = 2 memory = 4 } boot_disk { disk_id = yandex_compute_disk.boot-disk-2.id } network_interface { subnet_id = yandex_vpc_subnet.subnet_b.id nat = true } metadata = { user-data = templatefile("vm-init.tpl", { USER_NAME = var.user_name USER_SSH_KEY = file(var.user_ssh_key_path) }) } } resource "yandex_compute_instance" "prod_vm" { folder_id = yandex_resourcemanager_folder.prod_folder.id name = "prod-vm" hostname = "prod-vm" platform_id = "standard-v3" zone = "ru-central1-d" resources { cores = 2 memory = 4 } boot_disk { disk_id = yandex_compute_disk.boot-disk-3.id } network_interface { subnet_id = yandex_vpc_subnet.subnet_d.id nat = true } metadata = { user-data = templatefile("vm-init.tpl", { USER_NAME = var.user_name USER_SSH_KEY = file(var.user_ssh_key_path) }) } } # ======= # Outputs # ======= output "NET-vm" { value = yandex_compute_instance.network_vm.network_interface.0.nat_ip_address } output "DEV-vm" { value = yandex_compute_instance.dev_vm.network_interface.0.nat_ip_address } output "PROD-vm" { value = yandex_compute_instance.prod_vm.network_interface.0.nat_ip_address }
-
Run the following commands:
export TF_VAR_cloud_id=$(yc config get cloud-id) export YC_TOKEN=$(yc iam create-token) terraform apply
-
When asked to confirm the changes, enter
yes
and wait for the operation to complete.
To create a VM, use the create REST API method for the Compute Instance resource or the InstanceService/Create gRPC API call.
Check your resources connectivity
-
Connect to the
net-vm
VM over SSH:ssh ycuser@<net-vm_public_IP_address>
-
Check whether you can connect to
dev-vm
:ping -c3 <net-vm_internal_IP_address>
Result:
PING 10.127.20.4 (10.127.20.4) 56(84) bytes of data. 64 bytes from 10.127.20.4: icmp_seq=1 ttl=61 time=7.45 ms 64 bytes from 10.127.20.4: icmp_seq=2 ttl=61 time=5.61 ms 64 bytes from 10.127.20.4: icmp_seq=3 ttl=61 time=5.65 ms --- 10.127.20.4 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 5.613/6.235/7.446/0.855 ms
-
Similarly, check the connection to
prod-vm
. -
Connect to
dev-vm
over SSH and use ping to check the IP connectivity betweendev-vm
,net-vm
, andprod-vm
. -
Connect to
prod-vm
over SSH and use ping to check the IP connectivity betweenprod-vm
,net-vm
, anddev-vm
.
How to delete the resources you created
To stop paying for the resources you created, delete VMs: net-vm
, dev-vm
, and prod-vm
.