Uploading a custom disk image to Yandex Cloud
This guide explains how to upload a Linux image file to Yandex Object Storage and use it to create an image and a VM in Compute Cloud.
Major virtualization systems are supported.
Warning
In Compute Cloud, you can only create images using files uploaded to Object Storage. You can also migrate a VM using Hystax Acura.
Preparing an image file
Make sure the following requirements for boot disk images are met:
- The
virtio-net
,virtio-blk
, andvirtio-pci
drivers are installed. If you are going to attach file storages to your VM, you also need to install thevirtiofs
driver. For details, see this guide. - The
ttyS0
terminal (COM1) is configured as the serial console. For details, see this guide. - The network interface runs correctly when starting the VM and gets the IP address via DHCP.
- The
cloud-init
package is installed and configured to work with our metadata service. Runsudo yum install cloud-init
to install the package for CentOS orsudo apt update && sudo apt install -y cloud-init
to install it for Debian or Ubuntu. - For an image based on an AMI, cloud platform verification for the VM instance is disabled in the
cloud-init
settings. For details, see this guide. - The system firewall is configured to only allow the minimum ports required for your applications to run securely and the SSH port (by default, 22 TCP).
- The SSH server starts automatically at VM startup.
- The services running your application are resilient to VM reboots.
- The boot disk uses MBR partitioning.
- The disk is mounted by its UUID rather than by name.
- The file system is not encrypted.
Supported formats: Qcow2
, VMDK
, RAW
, and VHD
.
For configuration steps, see Preparing a disk image.
Uploading an image file to Object Storage
Upload your image file to Object Storage and get a link to it:
- If you do not have a bucket in Object Storage yet, create one with restricted access.
- Upload the image to your bucket, for example, using the management console, AWS CLI, or WinSCP. Within the Object Storage context, the uploaded image is called an object.
- Get a signed link to download the image from the bucket. Use this link when creating an image in Compute Cloud.
Creating an image in Compute Cloud
Create a new image from the link you got in Object Storage:
-
In the management console, select the folder where you want to create an image.
-
Select Compute Cloud.
-
In the left-hand panel, select
Images. -
Click Upload image.
-
Name your image. The naming requirements are as follows:
- It must be 2 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- It must start with a letter and cannot end with a hyphen.
-
Add a description for the image, if required.
-
Paste the link to the image you got in Object Storage.
-
To create an optimized image, enable Optimize for deployment.
-
Click Upload.
To create a new image via the link, use the --source-uri
flag. To create an optimized image, use the --pooled
flag.
yc compute image create --name <image_name> --source-uri <image_URL> --pooled
Where:
--name
: Name to assign to the image.--source-uri
: Image link you got in Object Storage.
Add a description as required and specify the family the image belongs to:
yc compute image create \
--name ubuntu-cosmic \
--description "Ubuntu Server 18.10 (Cosmic Cuttlefish)" \
--family ubuntu \
--source-uri "https://storage.yandexcloud.net/mybucket/cosmic-server-cloudimg-amd64.vmdk"
If you know the minimum required size of a disk to create from this image, specify it in GB:
yc compute image create \
--name big-image \
--min-disk-size 20 \
--source-uri "https://storage.yandexcloud.net/mybucket/cosmic-server-cloudimg-amd64.vmdk"
Note
The min-disk-size
value must be greater than the size of the source data in the image and greater than the size of the virtual disk specified in the image.
If you set min-disk-size
below the allowed minimum, you will get an error and no image will be created. The error message will state the minimum value in bytes, e.g., Minimal disk size should be greater or equal than 10737418240
.
If you omit min-disk-size
, the minimum value will be used.
If you don't have Terraform, install it and configure the Yandex Cloud provider.
To create an image:
-
Specify the parameters of the
yandex_compute_image
resource in the configuration file.Here is an example of the configuration file structure:
resource "yandex_compute_image" "image-1" { name = "ubuntu-cosmic" os_type = "LINUX" source_url = "<image_link>" pooled = "false" }
Where
source_url
is the image link in Object Storage.For more information about the resources you can create with Terraform, see the relevant provider documentation
. -
Make sure the configuration files are correct.
-
In the command line, go to the directory where you created the configuration file.
-
Run a check using this command:
terraform plan
If you described the configuration correctly, the terminal will display a list of the resources being created and their parameters. If the configuration contains any errors, Terraform will point them out.
-
-
Deploy the cloud resources.
-
Run this command:
terraform apply
-
Confirm creating the resources.
This will create all the resources you need in the specified folder. You can check the new resources and their settings using the management console
. -
Create a new image using the create REST API method for the Image resource or the ImageService/Create gRPC API call. In the request, specify the link to the image.
Once created, the image will get the CREATING
status. Wait until its status changes to READY
before using it.
Deleting the image from Object Storage
If you have successfully created an image, you can delete the image file from Object Storage. You can also delete the bucket if it contains no objects.
Creating a VM from a pre-built image
You can create a VM with disks from a pre-built image.
Note
Disks, snapshots, and images are separate resources that work independently from each other in Compute Cloud. You can create, delete, and manage them separately. Any changes you make to the disk do not affect the snapshot of that disk or the image it was created from. Even if you delete the disk or the VM with the disk, its snapshot and image will remain unaffected.
-
In the management console
, select the folder where you want to create your VM. -
In the list of services, select Compute Cloud.
-
In the left-hand panel, select
Virtual machines. -
Click Create virtual machine.
-
Under Boot disk image:
- Go to the Custom tab.
- Click Select and select Create new in the window that opens.
- In the Contents field, select
Image
and then select the image you need from the list below. - Optionally, enable Additional in the Delete along with the virtual machine field if you need this disk automatically deleted when deleting the VM.
- Click Add disk.
-
Under Location, select an availability zone to place your VM in.
-
Add a disk:
- Under Disks and file storages, click Add.
- In the window that opens, select Disk →
Create new
. - In the Contents field, select
Image
and then select the image you need from the list below. - Enter a name for the disk.
- Select the disk type.
- Specify the required disk size and block size.
- Optionally, enable Additional in the Delete along with the virtual machine field if you need this disk automatically deleted when deleting the VM.
- Click Add disk.
-
Optionally, to encrypt a boot disk or a secondary disk, under Disks and file storages, click
to the right of the disk name and configure encryption parameters for the disk:- Select Encrypted disk.
- In the KMS key field, select the key you want to use to encrypt the disk. To create a new key, click Create new key.
Warning
You can specify encryption settings only when creating a disk. You cannot disable or change disk encryption.
If you deactivate the key used to encrypt a disk, image, or snapshot, access to the data will be suspended until you reactivate the key.
Alert
If you destroy the key or its version used to encrypt a disk, image, or snapshot, access to the data will be irrevocably lost. For details, see Destroying key versions.
-
Optionally, connect a file storage:
-
Under Disks and file storages, click Add.
-
In the window that opens, select File storage and choose the storage you want to connect from the list.
-
Click Add file storage.
-
-
-
Under Computing resources, select one of the preset configurations or create a custom one. To create a custom configuration:
- Go to the Custom tab.
- Select a platform.
- Specify the guaranteed performance and required number of vCPUs, as well as RAM size.
- Optionally, enable a software-accelerated network.
- Make your VM preemptible, if required.
-
Under Network settings:
-
In the Subnet field, enter the ID of a subnet in the new VM’s availability zone. Alternatively, you can select a cloud network from the list.
-
Each network must have at least one subnet. If your network has no subnets, create one by selecting Create subnet.
-
If there are no networks in the list, click Create network to create one:
- In the window that opens, specify the network name and select the folder where you want to create it.
- Optionally, select Create subnets to automatically create subnets in all availability zones.
- Click Create network.
-
-
In the Public IP address field, select a method for assigning an IP address:
Auto
: To assign a random IP address from the Yandex Cloud IP address pool. In this case, you can enable DDoS protection using the option below.List
: To select a public IP address from the list of previously reserved static addresses. For more information, see Converting a dynamic public IP address to static.No address
: Not to assign a public IP address.
-
Select the relevant security groups. If you leave this field empty, the default security group will be assigned to the VM.
-
Expand Additional and select a method for assigning internal addresses in the Internal IPv4 address field:
Auto
: To assign a random IP address from the pool of IP addresses available in the selected subnet.Manual
: To manually assign a private IP address to the VM.- Enable DDoS protection, if required. The option is available if you previously selected the automatic IP assignment method in the public address settings.
-
Optionally, create records for your VM in the DNS zone:
- Expand DNS settings for internal addresses and click Add record.
- Specify a zone, FQDN, and TTL for the record. When setting the FQDN, you can enable
Detect automatically
for the zone.
You can add multiple records to internal DNS zones. For more information, see Cloud DNS integration with Compute Cloud. - To create another record, click Add record.
If you want to add another network interface to your VM, click Add network interface and repeat the settings from this step for the new interface. You can add up to eight network interfaces to a single VM.
-
-
Under Access:
-
Select Access by OS Login to connect and manage access to the new VM using OS Login in Yandex Cloud Organization.
With OS Login, you can connect to VMs using SSH keys and SSH certificates via a standard SSH client or the Yandex Cloud CLI. OS Login enables rotating the SSH keys used to access VMs, providing the most secure access option.
-
If you prefer not to use OS Login, select SSH key and specify the following VM access data:
-
Under Login, enter a username.
Alert
Do not use
root
or other usernames reserved by the OS. To perform operations requiring superuser privileges, use thesudo
command. -
In the SSH key field, select the SSH key saved in your organization user profile.
If there are no saved SSH keys in your profile, or you want to add a new key:
- Click Add key.
- Enter a name for the SSH key.
- Upload or paste the contents of the public key file. You need to create a key pair for the SSH connection to a VM yourself.
- Click Add.
The SSH key will be added to your organization user profile.
If users cannot add SSH keys to their profiles in the organization, the added public SSH key will only be saved to the user profile of the VM being created.
-
If you want to add multiple users with SSH keys to the VM at the same time, specify these users' data under Metadata. You can also use metadata to install additional software on a VM when creating it.
In public Linux images provided by Yandex Cloud, the functionality of connecting over SSH using login and password is disabled by default.
-
-
Under General information, specify a name for the VM:
- It must be 2 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- It must start with a letter and cannot end with a hyphen.
Note
The VM name is used to generate an internal FQDN, which is set only once, when you create the VM. If the internal FQDN is important to you, make sure to choose an appropriate name for your VM.
-
Under Additional:
-
Optionally, select or create a service account. With a service account, you can flexibly configure access permissions for your resources.
-
Optionally, enable access to the serial console.
-
Optionally, under Backup, enable Connect and select or create a backup policy to make automatic backups of your VMs using Cloud Backup.
For more information, see Connecting Compute Cloud VMs and Yandex BareMetal servers to Cloud Backup.
-
Optionally, under Monitoring, enable Agent for delivering metrics to configure delivering metrics to Yandex Monitoring.
-
Optionally, under Placement, select a VM placement group.
-
-
Click Create VM.
The VM will appear in the list. The system automatically assigns an IP address and host name (FQDN) to a VM once it is created.
You can monitor the VM status in the serial console or serial port output.
If you do not have the Yandex Cloud CLI yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder through the --folder-name
or --folder-id
parameter.
-
See the description of the CLI command for creating a VM:
yc compute instance create --help
-
Get a list of images in the default folder:
yc compute image list
Result:
+----------------------+-----------------+--------+-------------+--------+ | ID | NAME | FAMILY | PRODUCT IDS | STATUS | +----------------------+-----------------+--------+-------------+--------+ | ... | ... | | | ... | | fd8gkcd3l6ov******** | your-test-image | | | READY | | ... | ... | | | ... | +----------------------+-----------------+--------+-------------+--------+
-
Select
ID
orNAME
of the image you need. -
Select a subnet:
yc vpc subnet list
Result:
+----------------------+---------------------------+----------------------+----------------+-------------------+-----------------+ | ID | NAME | NETWORK ID | ROUTE TABLE ID | ZONE | RANGE | +----------------------+---------------------------+----------------------+----------------+-------------------+-----------------+ | e9bnlm18l70a******** | default-ru-central1-a | enpe3m3fa00u******** | | ru-central1-a | [10.128.0.0/24] | +----------------------+---------------------------+----------------------+----------------+-------------------+-----------------+
-
Create a VM in the default folder:
yc compute instance create \ --name <VM_name> \ --zone <availability_zone> \ --network-interface subnet-name=<subnet_name>,nat-ip-version=ipv4 \ --create-boot-disk name=<disk_name>,size=<disk_size_in_GB>,image-id=<custom_image_ID> \ --ssh-key <path_to_public_key_file>
Where:
-
--name
: VM name. The naming requirements are as follows:- It must be 2 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- It must start with a letter and cannot end with a hyphen.
Note
The VM name is used to generate an internal FQDN, which is set only once, when you create the VM. If the internal FQDN is important to you, make sure to choose an appropriate name for your VM.
-
--zone
: Availability zone matching the selected subnet. -
--network-interface
: VM network interface settings:subnet-name
: Name of the selected subnet.nat-ip-version=ipv4
: Public IP address. To create a VM without a public IP address, omit this parameter.
If you want to add multiple network interfaces to your VM, specify the
--network-interface
parameter as many times as you need. You can add up to eight network interfaces to a single VM. -
--create-boot-disk
: VM boot disk settings:-
name
: Boot disk name. The naming requirements are as follows:- It must be 2 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- It must start with a letter and cannot end with a hyphen.
-
size
: Disk size in GB. -
image-id
: ID of the custom image to create the VM from. Specify the ID of the uploaded image.
-
-
--ssh-key
: Path to the file with the public SSH key. The VM will automatically create a user namedyc-user
for this key.When creating a VM from a Yandex Cloud Marketplace public image, make sure to provide an SSH key, as SSH access with a username and password is disabled by default for such images.
If you want to add multiple users with SSH keys to your VM at the same time, specify these users' data in the
--metadata-from-file
parameter.
-
When a VM is created, it is assigned an IP address and hostname (FQDN). This data can be used for SSH access.
You can make a public IP address static. For more information, see Making a VM public IP address static.
Result:
```text
id: fhmue131en37********
folder_id: b1g681qpemb4********
created_at: "2024-03-02T12:58:43Z"
name: test-vm-from-image
zone_id: ru-central1-a
platform_id: standard-v2
resources:
memory: "2147483648"
cores: "2"
core_fraction: "100"
status: RUNNING
metadata_options:
gce_http_endpoint: ENABLED
aws_v1_http_endpoint: ENABLED
gce_http_token: ENABLED
aws_v1_http_token: DISABLED
boot_disk:
mode: READ_WRITE
device_name: fhmn9n1uhutc********
auto_delete: true
disk_id: fhmn9n1uhutc********
network_interfaces:
- index: "0"
mac_address: d0:0d:1e:70:46:17
subnet_id: e9bb9n0v4h17********
primary_v4_address:
address: 10.12*.*.**
one_to_one_nat:
address: 178.154.***.***
ip_version: IPV4
gpu_settings: {}
fqdn: fhmue131en37********.auto.internal
scheduling_policy: {}
network_settings:
type: STANDARD
placement_policy: {}
```
If you don't have Terraform, install it and configure the Yandex Cloud provider.
To create a VM from a custom image:
-
In the configuration file, define the parameters of the resources you want to create:
resource "yandex_compute_disk" "boot-disk" { name = "<disk_name>" type = "<disk_type>" zone = "<availability_zone>" size = "<disk_size>" image_id = "<custom_image_ID>" } resource "yandex_compute_instance" "vm-1" { name = "vm-from-image" allow_stopping_for_update = true platform_id = "standard-v3" zone = "<availability_zone>" resources { cores = <number_of_vCPU_cores> memory = <RAM_in_GB> } boot_disk { disk_id = yandex_compute_disk.boot-disk.id } network_interface { subnet_id = "${yandex_vpc_subnet.subnet-1.id}" nat = true } metadata = { ssh-keys = "<username>:<SSH_key_contents>" } } resource "yandex_vpc_network" "network-1" { name = "network1" } resource "yandex_vpc_subnet" "subnet-1" { name = "subnet1" zone = "<availability_zone>" network_id = "${yandex_vpc_network.network-1.id}" }
Where:
yandex_compute_disk
: Boot disk description:-
name
: Disk name. The naming requirements are as follows:- It must be 2 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- It must start with a letter and cannot end with a hyphen.
-
type
: Disk type. -
zone
: Availability zone the disk will be in. -
size
: Disk size in GB. -
image_id
: ID of the custom image to create the VM from. Specify the ID of the uploaded image.
-
yandex_compute_instance
: VM description.-
name
: VM name. The naming requirements are as follows:- It must be 2 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- It must start with a letter and cannot end with a hyphen.
Note
The VM name is used to generate an internal FQDN, which is set only once, when you create the VM. If the internal FQDN is important to you, make sure to choose an appropriate name for your VM.
-
allow_stopping_for_update
: Permission to stop the VM for updates. Set totrue
if you plan to change your VM's network settings or computing resources using Terraform. The default value isfalse
. -
platform_id
: Platform. -
zone
: Availability zone the VM will be in. -
resources
: Number of vCPUs and amount of RAM available to the VM. The values must match the selected platform. -
boot_disk
: Boot disk settings. Specify the disk ID. -
network_interface
: VM network interface settings. Specify the ID of the selected subnet. To automatically assign a public IP address to the VM, setnat = true
.If you want to add multiple network interfaces to your VM, specify the
network_interface
section as many times as you need. You can add up to eight network interfaces to a single VM. -
metadata
: In metadata, provide the public key for SSH access to the VM. For more information, see VM metadata.
-
yandex_vpc_network
: Cloud network description.yandex_vpc_subnet
: Description of the subnet to connect your VM to.
Note
If you already have suitable resources, such as a cloud network and subnet, you do not need to redefine them. Specify their names and IDs in the appropriate parameters.
For more information about the resources you can create with Terraform, see the relevant provider documentation
. -
Create the resources:
-
In the terminal, change to the folder where you edited the configuration file.
-
Make sure the configuration file is correct using the command:
terraform validate
If the configuration is correct, the following message is returned:
Success! The configuration is valid.
-
Run the command:
terraform plan
The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.
-
Apply the configuration changes:
terraform apply
-
Confirm the changes: type
yes
in the terminal and press Enter.
All the resources you need will then be created in the specified folder. You can check the new resources and their settings using the management console
. -
Use the create REST API method for the Instance resource or the InstanceService/Create gRPC API call.
For information about Object Storage pricing, see Object Storage pricing policy.