Connecting to Container Registry from Virtual Private Cloud

To work with Container Registry, cloud resources require internet access. In this tutorial, we will create a Yandex Cloud infrastructure providing access to Container Registry for the Virtual Private Cloud-hosted resources with no internet access.

Container Registry uses Object Storage to store Docker images. This solution also provides Object Storage access for Virtual Private Cloud resources.

You can see the solution structure in the diagram below.

While deploying this Yandex Cloud infrastructure, you will create the following resources:

Name	Description
cr-vpc *	Cloud network with the resources requiring access to Container Registry
cr-nlb	Container Registry load balancer accepting TCP traffic on port 443 and distributing it across its target group instances
nat-group	Load balancer target group instances providing NAT translation
s3-nlb	Object Storage load balancer accepting TCP traffic on port 443 and distributing it across its target group instances
nat-a1-vm, nat-b1-vm	VMs residing in the ru-central1-a and ru-central1-b zones and performing NAT translation ontraffic directed to and from Container Registry and Object Storage
pub-ip-a1, pub-ip-b1	VM external IP addresses for public access to the VPC cloud network
DNS zones and A records	DNS A resource records in the cr-vpc network, mapping storage.yandexcloud.net. and cr.yandex. domain names to the respective load balancers IP addresses
test-registry	Container Registry test entry
container-registry-<registry_ID>	Object Storage bucket for storing Docker images, automatically created by Container Registry for <registry_ID>
cr-subnet-a, cr-subnet-b	Cloud subnets where the ru-central1-a and ru-central1-b NAT instances reside
test-cr-vm	VM used to test access to Container Registry
test-cr-subnet-a	Cloud subnet where the test VM resides

* You can also specify an existing cloud network.

We will use Cloud DNS zones to provide access to the Container Registry and Object Storage for our cloud network VMs:

• cr.yandex.: DNS A resource record mapping the Container Registry cr.yandex domain name to the cr-nlb load balancer IP address.
• storage.yandexcloud.net.: DNS A resource record mapping the Object Storage storage.yandexcloud.net domain name to the s3-nlb load balancer IP address.

These records will direct traffic coming from your cloud network and aimed at Container Registry and Object Storage to internal load balancers that will in turn distribute it across NAT instances.

These VMs with pre-installed NAT-instance images will perform NAT translation of traffic directed to and from Container Registry and Object Storage.

We ensure fault-tolerance by placing NAT instances in multiple availability zones. You can scale the solution for higher workload by adding more NAT instances. Before doing that, take into consideration the load balancer traffic processing locality.

We will limit access to the registry to cloud resources created in this tutorial. The registry access policy will only allow access from your NAT instance public IP addresses, so you will not be able to access it from any other IP address. If you need, you can remove this limitation in Terraform settings.

For more information, see the project repository.

To deploy a cloud infrastructure providing Container Registry access for VPC cloud network resources:

1. Get your cloud ready.
2. Configure the CLI profile.
3. Set up your environment.
4. Deploy your resources.
5. Test the solution.
6. Tips for production deployment.

If you no longer need the resources you created, delete them.

Get your cloud readyGet your cloud ready

Sign up in Yandex Cloud and create a billing account:

1. Navigate to the management console and log in to Yandex Cloud or register a new account.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can navigate to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Required paid resourcesRequired paid resources

The infrastructure support cost includes:

• Fee for continuously running VMs (see Yandex Compute Cloud pricing).
• Fee for using Network Load Balancer (see Yandex Network Load Balancer pricing).
• Fee for storing pushed Docker images (see Container Registry pricing).
• Fee for public IP addresses and outgoing traffic (see Yandex Virtual Private Cloud pricing).

Required quotasRequired quotas

Make sure you have sufficient cloud quotas not used by other projects.

Resources used in our tutorial

Resource	Amount
Virtual machines	3
VM vCPUs	6
VM RAM	6 GB
Disks	3
HDD size	30 GB
SSD size	20 GB
Network load balancer	2
Load balancer target group	1
Networks	1*
Subnets	3
Static public IP addresses	2
Security groups	1
DNS zone	2
Registry	1
Service accounts	1

* Unless you use the existing network, specifying its ID in terraform.tfvars.

Configure the CLI profileConfigure the CLI profile

1. If you do not have the Yandex Cloud CLI yet, install it and get authenticated according to instructions provided.

2. Create a service account:

   Management console

   CLI

   API

   1. In the management console, select the folder where you want to create a service account.
   2. From the list of services, select Identity and Access Management.
   3. Click Create service account.
   4. Specify the service account name, e.g., sa-terraform.
   5. Click Create.

   The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

   To create a service account, run the command below and specify the sa-terraform name:

   yc iam service-account create --name sa-terraform
   

   Where name is the service account name.

   Result:

   id: ajehr0to1g8b********
   folder_id: b1gv87ssvu49********
   created_at: "2023-06-20T09:03:11.665153755Z"
   name: sa-terraform
   

   To create a service account, use the ServiceAccountService/Create gRPC API call or the create REST API method for the ServiceAccount resource.

3. Assign the administrator role for the folder to the service account:

   Management console

   CLI

   API

   1. In the management console, select your infrastructure folder.
   2. Navigate to the Access bindings tab.
   3. Select sa-terraform from in account list and click -> Edit roles.
   4. In the dialog that opens, click Add role and select the admin role.

   Run this command:

   yc resource-manager folder add-access-binding <folder_ID> \
      --role admin \
      --subject serviceAccount:<service_account_ID>
   

   To assign a role for a folder to a service account, use the setAccessBindings REST API method for the ServiceAccount resource or the ServiceAccountService/SetAccessBindings gRPC API call.

4. Set up the CLI profile to run operations on behalf of the service account:

   CLI

   1. Create an authorized key for the service account and save it to the file:

      yc iam key create \
      --service-account-id <service_account_ID> \
      --folder-id <ID_of_folder_with_service_account> \
      --output key.json
      

      Where:

      • service-account-id: Service account ID.
      • folder-id: Service account folder ID.
      • output: Authorized key file name.

      Result:

      id: aje8nn871qo4********
      service_account_id: ajehr0to1g8b********
      created_at: "2023-06-20T09:16:43.479156798Z"
      key_algorithm: RSA_2048
      

   2. Create a CLI profile to run operations on behalf of the service account:

      yc config profile create sa-terraform
      

      Result:

      Profile 'sa-terraform' created and activated
      

   3. Configure the profile:

      yc config set service-account-key key.json
      yc config set cloud-id <cloud_ID>
      yc config set folder-id <folder_ID>
      

      Where:

      • service-account-key: Authorized key file name.
      • cloud-id: Cloud ID.
      • folder-id: Folder ID.

   4. Export your credentials to environment variables:

      export YC_TOKEN=$(yc iam create-token)
      

Set up your environmentSet up your environment

Install the required toolsInstall the required tools

1. Install Git using the following command:

   sudo apt install git
   

2. Install Terraform.

Deploy your resourcesDeploy your resources

1. Clone the GitHub repository and navigate to the yc-cr-private-endpoint script directory:

   git clone https://github.com/yandex-cloud-examples/yc-cr-private-endpoint.git
   cd yc-cr-private-endpoint
   

2. Open the terraform.tfvars file and edit it as follows:

   1. Line with the folder ID:

      folder_id = "<folder_ID>"
      

   2. Line with a list of subnets allowed to access Container Registry:

      trusted_cloud_nets = ["10.0.0.0/8", "192.168.0.0/16"]
      

   terraform.tfvars variable description

   Parameter name	Change required	Description	Type	Example
   folder_id	Yes	Solution components folder ID	string	b1gentmqf1ve9uc54nfh
   vpc_id	-	ID of your cloud network with access to Container Registry. If left empty, Terraform will create a new network.	string	enp48c1ndilt42veuw4x
   yc_availability_zones	-	List of NAT instance availability zones	list(string)	["ru-central1-a", "ru-central1-b"]
   subnet_prefix_list	-	List of NAT instance subnets corresponding to availability zones from the yc_availability_zones list	list(string)	["10.10.1.0/24", "10.10.2.0/24"]
   nat_instances_count	-	Number of NAT instances. We recommend using an even number of VMs to evenly distribute them between availability zones.	number	2
   registry_private_access	-	Limits access to the registry to NAT instance IP addresses. You can remove this limitation by specifying false.	bool	true
   trusted_cloud_nets	Yes	List of cloud subnets with access to Container Registry used in the inbound traffic rule for the NAT instance security group	list(string)	["10.0.0.0/8", "192.168.0.0/16"]
   vm_username	-	NAT instance and test VM user name	string	admin
   cr_ip	-	Container Registry public IP address	string	84.201.171.239
   cr_fqdn	-	Container Registry domain name	string	cr.yandex
   s3_ip	-	Object Storage public IP address	string	213.180.193.243
   s3_fqdn	-	Object Storage domain name	string	storage.yandexcloud.net

3. Deploy your cloud resources using Terraform:

   1. Initialize Terraform:

      terraform init
      

   2. Check the Terraform file configuration:

      terraform validate
      

   3. Preview the list of new cloud resources:

      terraform plan
      

   4. Create resources:

      terraform apply
      

4. Once the terraform apply process is completed, the command will display information allowing you to connect to the test VM and run Container Registry tests. You can also see this information by running the terraform output command:

   Expand to see the deployed resource details

   Name	Description	Value (example)
   cr_nlb_ip_address	Container Registry load balancer IP address	10.10.1.100
   cr_registry_id	Container Registry ID	crp1r4h00mj*********
   path_for_private_ssh_key	Private key for SSH access to NAT and test VMs	./pt_key.pem
   s3_nlb_ip_address	Object Storage load balancer IP address	10.10.1.200
   test_vm_password	Test VM admin password	v3RCqUrQN?x)
   vm_username	NAT instance and test VM user name	admin

Test the solutionTest the solution

1. In the management console, navigate to your resource folder.

2. Select Compute Cloud.

3. Select test-cr-vm from the list of VMs.

4. In the left-hand menu, select Serial console.

5. Click Connect.

6. Enter the admin username and the password you saved earlier. To get the password again, run terraform output test_vm_password; enter the password without quotation marks.

7. Run this command:

   dig cr.yandex storage.yandexcloud.net
   

8. Check the DNS server response and make sure Object Storage and Container Registry domain names point at the load balancers IP addresses. The command output will show A-type resource records as follows:

   ;; ANSWER SECTION:
   cr.yandex.               300    IN      A       10.10.1.100
   
   ;; ANSWER SECTION:
   storage.yandexcloud.net. 300    IN      A       10.10.1.200
   

9. Check the list of available Docker images:

   docker image list
   

   Result:

   REPOSITORY    TAG       IMAGE ID       CREATED        SIZE
   golang        1.20.5    342*********   8 months ago   777MB
   hello-world   latest    9c7*********   9 months ago   13.3kB
   

10. Assign an URL to the Docker image using the following format: cr.yandex/<registry_ID>/<Docker_image_name>:<tag>. The docker command will retrieve registry ID from the environment variable:

    docker tag hello-world cr.yandex/$REGISTRY_ID/hello-world:demo
    
    docker image list
    

    Result:

    REPOSITORY                                   TAG       IMAGE ID       CREATED        SIZE
    golang                                       1.20.5    342*********   8 months ago   777MB
    cr.yandex/crp1r4h00mj*********/hello-world   demo      9c7*********   9 months ago   13.3kB
    hello-world                                  latest    9c7*********   9 months ago   13.3kB
    

    Note

    To push Docker images to Container Registry, you need to assign them URLs in this format: cr.yandex/<registry_ID>/<Docker_image_name>:<tag>.

11. Push the required Docker image to the registry:

    docker push cr.yandex/$REGISTRY_ID/hello-world:demo
    

    Result:

    The push refers to repository [cr.yandex/crp1r4h00mj*********/hello-world]
    01bb4*******: Pushed 
    demo: digest: sha256:7e9b6e7ba284****************** size: 525
    

12. In the management console, navigate to your resource folder.

13. Select Container Registry.

14. Select test-registry.

15. Make sure the hello-world repository with the Docker image appears in the registry.

Tips for solution deployment in the production environmentTips for solution deployment in the production environment

• When deploying NAT instances in two availability zones, use an even number of VMs to evenly distribute them between availability zones.

• When selecting the number of NAT instances, take into consideration the load balancer traffic processing locality.

• Once the solution is deployed, use planned downtime windows for maintenance, e.g., removing NAT instances or changing the yc_availability_zones setting, during which the system will not process user requests.

• If, with increased Container Registry workload, you see a high CPU steal time metric value, enable a software-accelerated network for this NAT instance.

• If you are using your own DNS server, create the following type A resource records in its zone file:

  Name	Type	Value
  cr.yandex.	A	<Container Registry load balancer IP address. To get it, run terraform output cr_nlb_ip_address>
  storage.yandexcloud.net.	A	<Object Storage load balancer IP address. To get it, run terraform output s3_nlb_ip_address>

• Save the pt_key.pem private key for SSH access to NAT instances to a secure location or recreate it without using Terraform.

• In our test solution, SSH access to NAT instances is disabled. To enable it, in the cr-nat-sg security group, add an inbound traffic rule allowing access on port TCP/22 from your trusted IP addresses.

• Once you tested the solution, delete the test VM and its subnet.

Delete the resources you createdDelete the resources you created

• Manually

  1. In the management console, navigate to your resource folder.
  2. Select Container Registry.
  3. Select test-registry.
  4. Select the hello-world repository.
  5. For each Docker image in the repository, click .
  6. In the menu that opens, click Delete.
  7. In the window that opens, click Delete.

• Using Terraform

  1. In the terminal window, go to the directory containing the infrastructure plan.

     Warning

     Make sure the directory has no Terraform manifests with the resources you want to keep. Terraform deletes all resources that were created using the manifests in the current directory.

  2. Delete resources:

     1. Run this command:

        terraform destroy
        

     2. Confirm deleting the resources and wait for the operation to complete.

     All the resources described in the Terraform manifests will be deleted.