Building a VM image with infrastructure tools using Packer

Written by

Updated at November 5, 2024

Prepare your cloud
- Required paid resources
Set up a working environment
Prepare the image configuration
Build the image
Create a VM from the image
How to delete the resources you created

You can use Yandex Compute Cloud to create a VM disk image with additional infrastructure tools using Packer.

Use Packer to build a VM image based on Ubuntu Linux 20.04 LTS with the parameters specified in a configuration file. Add the following tools frequently used with Yandex Cloud to the image:

Yandex Cloud CLI 0.91.0 or higher.
Terraform 1.1.9.
kubectl 1.23.
Docker 20.10.16 or higher.
Git 2.25.1 or higher.
Helm 3.9.0.
jq 1.6 or higher.
tree 1.8.0 or higher.
gRPCurl 1.8.6.
Pulumi 3.33.2.
tmux 3.0a or higher.

Using Packer, an auxiliary VM is created and run with the required software installed on it. Next, an image is built based on its boot disk. After that, the auxiliary VM and boot disk are deleted.

Follow the same steps to create your own image with the necessary software suite.

To build an image and create a VM from it:

If you no longer need the resources you created, delete them.

Prepare your cloud

Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Required paid resources

The cost of building a VM image and creating a VM from it includes:

Fee for storing built images (see Yandex Compute Cloud pricing).
Fee for VM computing resources (see Yandex Compute Cloud pricing).

Set up a working environment

Install Packer:
1. Download a Packer distribution and install it by following the instructions on the official website.
  
  You can also download a Packer distribution for your platform from a Yandex Cloud mirror.
2. When the download is complete, add the path to the folder with the executable to the PATH variable. To do this, run the following command:
```
export PATH=$PATH:<path_to_folder_with_Packer_executable>
```
  Note
  
  Yandex Cloud requires Packer 1.5 or higher.

Configure the Yandex Compute Builder plugin:

Create a file named config.pkr.hcl with the following contents:

packer {
  required_plugins {
    yandex = {
      version = ">= 1.1.2"
      source  = "github.com/hashicorp/yandex"
    }
  }
}

Install the plugin:

packer init <path_to_config.pkr.hcl>

Result:

Installed plugin github.com/hashicorp/yandex v1.1.2 in ...

Install the Yandex Cloud CLI and create a profile.

Get information about available subnets and availability zones. If you do not have any subnets, create one.

CLI

API

Run this command:

yc vpc subnet list

Result:

+----------------------+----------------------+----------------------+----------------+---------------+-----------------+
|          ID          |         NAME         |      NETWORK ID      | ROUTE TABLE ID |     ZONE      |      RANGE      |
+----------------------+----------------------+----------------------+----------------+---------------+-----------------+
| b0c29k6anelk******** | intro2-ru-central1-d | enp45glgitd6******** |                | ru-central1-d | [10.130.0.0/24] |
| e2ltcj4urgpb******** | intro2-ru-central1-b | enp45glgitd6******** |                | ru-central1-b | [10.129.0.0/24] |
| e9bn57jvjnbu******** | intro2-ru-central1-a | enp45glgitd6******** |                | ru-central1-a | [10.128.0.0/24] |
+----------------------+----------------------+----------------------+----------------+---------------+-----------------+

Save the ID of the subnet (the ID column) to host the auxiliary VM used to create the image, as well as the relevant availability zone (the ZONE column). You will need these parameters in the next steps.

Use the list REST API method for the Subnet resource or the SubnetService/List gRPC API call.

Specify the values of the variables used when building the image in the command line.
```
export YC_FOLDER_ID=$(yc config get folder-id)
export YC_ZONE="<availability_zone>"
export YC_SUBNET_ID="<subnet_ID>"
export YC_TOKEN=$(yc iam create-token)
```
Where:
- YC_FOLDER_ID: ID of the folder to host the auxiliary VM used for creating the image. Provided automatically.
- YC_ZONE: Availability zone to host the auxiliary VM used for creating the image. Previously obtained.
- YC_SUBNET_ID: ID of the subnet to host the auxiliary VM used for creating the image. Previously obtained.
- YC_TOKEN: IAM token. Required for creating VM images. Provided automatically.
Generate an SSH key pair. You will need them to create a VM and connect to it.

Prepare the image configuration

Create an HCL configuration file, such as toolbox.pkr.hcl.

In the configuration file, describe the parameters of the image to create:

# Yandex Cloud Toolbox VM Image based on Ubuntu 20.04 LTS
#
# Provisioner docs:
# https://www.packer.io/docs/builders/yandex
#

variable "YC_FOLDER_ID" {
  type = string
  default = env("YC_FOLDER_ID")
}

variable "YC_ZONE" {
  type = string
  default = env("YC_ZONE")
}

variable "YC_SUBNET_ID" {
  type = string
  default = env("YC_SUBNET_ID")
}

variable "TF_VER" {
  type = string
  default = "1.1.9"
}

variable "KCTL_VER" {
  type = string
  default = "1.23.0"
}

variable "HELM_VER" {
  type = string
  default = "3.9.0"
}

variable "GRPCURL_VER" {
  type = string
  default = "1.8.6"
}

variable "GOLANG_VER" {
  type = string
  default = "1.17.2"
}

variable "PULUMI_VER" {
  type = string
  default = "3.33.2"
}

source "yandex" "yc-toolbox" {
  folder_id           = "${var.YC_FOLDER_ID}"
  source_image_family = "ubuntu-2004-lts"
  ssh_username        = "ubuntu"
  use_ipv4_nat        = "true"
  image_description   = "Yandex Cloud Ubuntu Toolbox image"
  image_family        = "my-images"
  image_name          = "yc-toolbox"
  subnet_id           = "${var.YC_SUBNET_ID}"
  disk_type           = "network-hdd"
  zone                = "${var.YC_ZONE}"
}

build {
  sources = ["source.yandex.yc-toolbox"]

  provisioner "shell" {
    inline = [
      # Global Ubuntu things
      "sudo apt-get update",
      "echo 'debconf debconf/frontend select Noninteractive' | sudo debconf-set-selections",
      "sudo apt-get install -y unzip python3-pip python3.8-venv",

      # Yandex Cloud CLI tool
      "curl --silent --remote-name https://storage.yandexcloud.net/yandexcloud-yc/install.sh",
      "chmod u+x install.sh",
      "sudo ./install.sh -a -i /usr/local/ 2>/dev/null",
      "rm -rf install.sh",
      "sudo sed -i '$ a source /usr/local/completion.bash.inc' /etc/profile",
  
      # Docker
      "curl --fail --silent --show-error --location https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-keyring.gpg",
      "echo \"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable\" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null",
      "sudo apt-get update",
      "sudo apt-get install -y docker-ce containerd.io",
      "sudo usermod -aG docker $USER",
      "sudo chmod 666 /var/run/docker.sock",
      "sudo useradd -m -s /bin/bash -G docker yc-user",

      # Docker Artifacts
      "docker pull hello-world",
      "docker pull -q amazon/aws-cli",
      "docker pull -q golang:${var.GOLANG_VER}",

      # Terraform (classic way)
      #"curl --fail --silent --show-error --location https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-keyring.gpg",
      #"echo \"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/hashicorp.list > /dev/null",
      #"sudo apt-get update",
      #"sudo apt-get install -y terraform",
      #
      # Alternative Option
      "curl --silent --location https://hashicorp-releases.yandexcloud.net/terraform/${var.TF_VER}/terraform_${var.TF_VER}_linux_amd64.zip --output terraform.zip",
      "unzip terraform.zip",
      "sudo install -o root -g root -m 0755 terraform /usr/local/bin/terraform",
      "rm -rf terraform terraform.zip",
      # Terraform configuration file ?
      #"cat <<EOF > ~/.terraformrc \n provider_installation { network_mirror { url = \"https://terraform-mirror.yandexcloud.net/\" include = [\"registry.terraform.io/*/*\"] } direct { exclude = [\"registry.terraform.io/*/*\"] } } \n EOF",

      # kubectl
      "curl --silent --location --remote-name https://dl.k8s.io/release/v${var.KCTL_VER}/bin/linux/amd64/kubectl",
      "sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl",
      "rm -rf kubectl",

      # Helm
      "curl --silent --show-error --location --remote-name https://get.helm.sh/helm-v${var.HELM_VER}-linux-amd64.tar.gz",
      "tar zxf helm-v${var.HELM_VER}-linux-amd64.tar.gz",
      "sudo install -o root -g root -m 0755 linux-amd64/helm /usr/local/bin/helm",
      "rm -rf helm-v${var.HELM_VER}-linux-amd64.tar.gz",
      "rm -rf linux-amd64",
      # User can add own repo after login like this:
      # helm repo add stable https://charts.helm.sh/stable

      ## grpccurl
      "curl --silent --show-error --location --remote-name https://github.com/fullstorydev/grpcurl/releases/download/v${var.GRPCURL_VER}/grpcurl_${var.GRPCURL_VER}_linux_x86_64.tar.gz",
      "tar zxf grpcurl_${var.GRPCURL_VER}_linux_x86_64.tar.gz",
      "sudo install -o root -g root -m 0755 grpcurl /usr/local/bin/grpcurl",
      "rm -rf grpcurl_${var.GRPCURL_VER}_linux_x86_64.tar.gz",
      "rm -rf grpcurl",

      # Pulumi
      "curl --silent --show-error --location --remote-name https://get.pulumi.com/releases/sdk/pulumi-v${var.PULUMI_VER}-linux-x64.tar.gz",
      "tar zxf pulumi-v${var.PULUMI_VER}-linux-x64.tar.gz",
      "sudo cp pulumi/* /usr/local/bin/",
      "rm -rf pulumi-v${var.PULUMI_VER}-linux-x64.tar.gz",
      "rm -rf pulumi",

      # Other packages
      "sudo apt-get install -y git jq tree tmux",

      # Clean
      "rm -rf .sudo_as_admin_successful",

      # Test - Check versions for installed components
      "echo '=== Tests Start ==='",
      "yc version",
      "terraform version",
      "docker version",
      "kubectl version --client=true",
      "helm version",
      "grpcurl --version",
      "git --version",
      "jq --version",
      "tree --version",
      "pulumi version",
      "echo '=== Tests End ==='"
    ]
  }
}

Warning

You cannot use both the provisioner "shell" and metadata parameters in the configuration file at the same time.

Build the image

In the command line, go to the directory with the image configuration file.
```
cd <path_to_folder_with_configuration_file>
```
Make sure the image configuration file is correct using this command:
```
packer validate yc-toolbox.pkr.hcl
```
Where yc-toolbox.pkr.hcl is the configuration file name.

If the configuration is correct, you will get this message:
```
The configuration is valid.
```
Build the image with the command:
```
packer build yc-toolbox.pkr.hcl
```
Where yc-toolbox.pkr.hcl is the configuration file name.

Once the image is built, you will get a notification message:

...
==> Builds finished. The artifacts of successful builds are:
--> yandex.yc-toolbox: A disk image was created: yc-toolbox (id: fd83j475posv********) with family name infra-images

Save the ID of the built image (id). Use this ID to create a VM later.

Check if the built image is present in Yandex Cloud.

CLI

API

Run this command:

yc compute image list

Result:

+----------------------+------------+-----------+----------------------+--------+
|          ID          |    NAME    |  FAMILY   |     PRODUCT IDS      | STATUS |
+----------------------+------------+-----------+----------------------+--------+
| fd83j475posv******** | yc-toolbox | my-images | f2ek1vhoppg2******** | READY  |
+----------------------+------------+-----------+----------------------+--------+

Use the list REST API method for the Image resource or the ImageService/List gRPC API call.

Create a VM from the image

Specify the values of the variables used when creating your VM. To do this, run the following command:
```
export VM_NAME="<VM_name>"
export YC_IMAGE_ID="<image_ID>"
export YC_SUBNET_ID="<subnet_ID>"
export YC_ZONE="<availability_zone>"
```
Where:
- VM_NAME: Name of the new VM.
- YC_IMAGE_ID: ID of the image to create the VM from. Previously obtained.
- YC_SUBNET_ID: ID of the subnet to host the VM. Previously obtained.
- YC_ZONE: Availability zone for the VM. Previously obtained.
Create your VM from the built image.
CLI

API
Run this command:
```
yc compute instance create \
  --name $VM_NAME \
  --hostname $VM_NAME \
  --zone=$YC_ZONE \
  --create-boot-disk size=20GB,image-id=$YC_IMAGE_ID \
  --cores=2 \
  --memory=8G \
  --core-fraction=100 \
  --network-interface subnet-id=$YC_SUBNET_ID,ipv4-address=auto,nat-ip-version=ipv4 \
  --ssh-key <path_to_public_part_of_SSH_key>
```
Where:
- --name: Name of the new VM.
- --hostname: VM host's name.
- --zone: Availability zone.
- --create-boot-disk: Boot disk properties: size stands for size, and image-id, for the ID of the image being used.
- --cores: Number of vCPUs.
- --memory: Amount of RAM.
- --core-fraction: Basic vCPU performance in percentage.
- --network-interface: Network interface proterties, where subnet-id is the subnet ID, ipv4-address is the internal IPv4 address, and nat-ip-version is the IP specification for the egress NAT.
- --ssh-key: Public part of the SSH key.
The command outputs information about the VM created. Save the VM's public IP address:
```
...
one_to_one_nat:
  address: 62.84.122.151
...
```
Learn more about creating a VM from a custom image.
Use the create REST API method for the Instance resource or the InstanceService/Create gRPC API call.

Connect to the VM over SSH:

ssh -i <path_to_private_part_of_SSH_key> yc-user@<VM_public_IP_address>

How to delete the resources you created

To stop paying for the resources you created:

Building a VM image with infrastructure tools using Packer

Prepare your cloudPrepare your cloud

Required paid resourcesRequired paid resources

Set up a working environmentSet up a working environment

Prepare the image configurationPrepare the image configuration

Build the imageBuild the image

Create a VM from the imageCreate a VM from the image

How to delete the resources you createdHow to delete the resources you created

Was the article helpful?

Prepare your cloud

Required paid resources

Set up a working environment

Prepare the image configuration

Build the image

Create a VM from the image

How to delete the resources you created