Delivering USB devices to a BareMetal server or Compute Cloud virtual machine

In this tutorial, you will configure delivering USB devices to a Yandex BareMetal server via a VPN connection over a public internet segment. You will do it using the USB over IP technology and freely distributed software that comes with Linux distributions.

USB over IP allows you to transfer data from USB devices over a network (local or internet) as if they were directly connected to the client computer. This is particularly important in situations where it is difficult or impossible to physically connect USB devices to the computer.

With USB over IP:

• You can deliver USB devices to cloud services, and Compute Cloud virtual machines or BareMetal servers can act as clients for remote USB devices.
• You can connect remote printers, scanners, cameras, hardware tokens, flash drives, and other USB peripherals to VMs and servers.
• To deliver USB devices, you can use both specialized integrated system platforms and freely distributed software.
• You can place keys, tokens, and smart cards delivered to servers and VMs within a controlled perimeter with limited access.
• Connections to remote USB devices can be restricted with the help of standard network security tools.

Solution diagram:

• Remote site USB client is a Windows or Linux-based virtual machine or physical server. In this tutorial, we are going to use as a client a physical server running Linux Ubuntu 24.04 LTS leased from Yandex BareMetal.

• Remote site USB server is a Linux-based device with a connection to a local network and VPN access (if the USB device data is delivered via the internet). USB devices will be physically inserted into the USB ports of the USB server. For a server, you can use microcomputers, e.g., Raspberry Pi. In this tutorial, we will use as a server a computer running Linux Ubuntu 22.04 LTS with several USB ports.

• Software. In this tutorial, USB devices will be delivered to the client via usbip with the help of the standard set of system utilities and core modules from the linux-tools package.

• Connected USB equipment:

  • USB data drive
  • USB token

• Network delivery method. Remote USB devices will be delivered via a VPN connection over a public internet segment using WireGuard.

  The proposed WireGuard-based arrangement is for demonstration purposes only; you can use any other technology to connect your remote servers.

To deliver USB devices to a BareMetal server using USB over IP:

1. Get your cloud ready.
2. Configure a cloud network.
3. Create a virtual machine for a VPN server.
4. Create a private BareMetal subnet.
5. Lease a BareMetal server.
6. Configure VPN.
7. Configure USB over IP.
8. Test the solution.

If you no longer need the resources you created, delete them.

Getting startedGetting started

Sign up in Yandex Cloud and create a billing account:

1. Navigate to the management console and log in to Yandex Cloud or register a new account.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can navigate to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Required paid resourcesRequired paid resources

The cost of the proposed solution includes:

• Fee for using the VM public IP address (see Yandex Virtual Private Cloud pricing).
• Fee for VM computing resources and disks (see Yandex Compute Cloud pricing).
• BareMetal server lease fee (see Yandex BareMetal pricing).

Configure a cloud networkConfigure a cloud network

Create a cloud network and subnetCreate a cloud network and subnet

Create a cloud network and subnet to connect the Compute Cloud VM (VPN server) to.

Create a security groupCreate a security group

Create a security group named vpn-sg for your VM (VPN server).

Create a virtual machine for a VPN serverCreate a virtual machine for a VPN server

Create a private BareMetal subnetCreate a private BareMetal subnet

Lease a BareMetal serverLease a BareMetal server

Configure VPNConfigure VPN

To set up delivering USB devices to a BareMetal server from a remote site computer, establish a VPN connection consisting of a VPN server deployed on a Compute Cloud virtual machine and two VPN clients: one on the BareMetal server and one on the remote site computer.

In this tutorial, you will use the WireGuard open source solution to set up a VPN connection. However, you can set up your VPN connection using other tools.

Configure a VPN serverConfigure a VPN server

1. Connect over SSH to the wireguard-vpn-server virtual machine you created earlier.

2. Install WireGuard and the required dependencies:

   sudo apt update && sudo apt install wireguard resolvconf
   

3. Enable IP forwarding in the Linux kernel settings:

   1. Open the sysctl.conf configuration file:

      sudo nano /etc/sysctl.conf
      

   2. Add this line to end of the sysctl.conf file:

      net.ipv4.ip_forward = 1
      

   3. Apply the new kernel settings:

      sudo sysctl -p
      

4. Generate VPN traffic encryption keys:

   wg genkey | sudo tee server_private.key | wg pubkey | sudo tee server_public.key > /dev/null
   wg genkey | sudo tee bms_private.key | wg pubkey | sudo tee bms_public.key > /dev/null
   wg genkey | sudo tee remote_private.key | wg pubkey | sudo tee remote_public.key > /dev/null
   

   These six files were created in the current directory:

   • server_private.key: Contains the private encryption key of the VPN server.
   • server_private.key: Contains the public encryption key of the VPN server.
   • bms_private.key: Contains the private encryption key of the VPN client (BareMetal server).
   • bms_public.key: Contains the public encryption key of the VPN client (BareMetal server).
   • remote_private.key: Contains the private encryption key of the remote site VPN client.
   • remote_public.key: Contains the public encryption key of the remote site VPN client.

   Save all the encryption keys: you will need them to create WireGuard configuration files on the relevant machines.

5. Create a configuration file of the WireGuard VPN server:

   1. Open the configuration file:

      sudo nano /etc/wireguard/wg0.conf
      

   2. Add the following configuration to the file using the contents of the encryption keys you got in the previous step:

      [Interface]
      Address = 192.168.100.1/24
      ListenPort = 63665
      PrivateKey = <server_private.key_file_contents>
      
      [Peer]
      PublicKey = <bms_public.key_file_contents>
      AllowedIPs = 192.168.100.2/32
      
      [Peer]
      PublicKey = <remote_public.key_file_contents>
      AllowedIPs = 192.168.100.3/32
      

6. Run WireGuard:

   sudo wg-quick up wg0
   

   Result:

   [#] ip link add wg0 type wireguard
   [#] wg setconf wg0 /dev/fd/63
   [#] ip -4 address add 192.168.100.1/24 dev wg0
   [#] ip link set mtu 1420 up dev wg0
   

   To stop WireGuard, run this command: sudo wg-quick down wg0.

   Note

   To activate WireGuard autostarting, run this command:

   sudo systemctl start wg-quick@wg0 && sudo systemctl enable wg-quick@wg0
   

Configure VPN clientsConfigure VPN clients

1. Connect via SSH to the BareMetal server named my-usbip-client you leased earlier.

2. Install WireGuard and the required dependencies:

   sudo apt update && sudo apt install wireguard resolvconf
   

3. Enable IP forwarding in the Linux kernel settings:

   1. Open the sysctl.conf configuration file:

      sudo nano /etc/sysctl.conf
      

   2. Add this line to end of the sysctl.conf file:

      net.ipv4.ip_forward = 1
      

   3. Apply the new kernel settings:

      sudo sysctl -p
      

4. Create a configuration file of the WireGuard VPN client:

   1. Open the configuration file:

      sudo nano /etc/wireguard/wg0.conf
      

   2. Add the following configuration to the file:

      [Interface]
      PrivateKey = <bms_private.key_file_contents>
      Address = 192.168.100.2/32
      
      [Peer]
      PublicKey = <server_public.key_file_contents>
      Endpoint = <VM_public_IP_address>:63665
      AllowedIPs = 192.168.100.0/24
      PersistentKeepalive = 15
      

      Where:

      • PrivateKey: Contents of the bms_private.key file created when setting up the VPN server and containing the private encryption key of that VPN client.
      • PublicKey: Contents of the server_public.key file created when setting up the VPN server and containing the public encryption key of the VPN server.
      • <VM_public_IP_address>: Public IP address of the virtual machine with the deployed VPN server. You can look up the VM's public IP address in the management console: see the Network section's Public IPv4 address field on the VM information page.

5. Run WireGuard:

   sudo wg-quick up wg0
   

   Result:

   [#] ip link add wg0 type wireguard
   [#] wg setconf wg0 /dev/fd/63
   [#] ip -4 address add 192.168.100.2/32 dev wg0
   [#] ip link set mtu 1420 up dev wg0
   [#] ip -4 route add 192.168.100.0/24 dev wg0
   

6. Similarly, configure the WireGuard VPN client on the remote site computer; in the PrivateKey field of the wg0.conf WireGuard configuration file, specify the contents of the remote_private.key file created when configuring the VPN server with the private encryption key of the remote site VPN client.

Test the VPN connectionTest the VPN connection

At this point, the VPN connection has been established. To test it:

1. Connect over SSH to the BareMetal server named my-usbip-client and run this command:

   ping 192.168.100.3 -c 5
   

   Result:

   PING 192.168.100.3 (192.168.100.3) 56(84) bytes of data.
   64 bytes from 192.168.100.3: icmp_seq=1 ttl=63 time=29.9 ms
   64 bytes from 192.168.100.3: icmp_seq=2 ttl=63 time=30.9 ms
   64 bytes from 192.168.100.3: icmp_seq=3 ttl=63 time=35.5 ms
   64 bytes from 192.168.100.3: icmp_seq=4 ttl=63 time=30.5 ms
   64 bytes from 192.168.100.3: icmp_seq=5 ttl=63 time=28.2 ms
   
   --- 192.168.100.3 ping statistics ---
   5 packets transmitted, 5 received, 0% packet loss, time 4005ms
   rtt min/avg/max/mdev = 28.249/30.987/35.453/2.405 ms
   

   Network connectivity between the VPN clients has been established with zero packet loss.

2. Run this command in the remote site computer's terminal:

   ping 192.168.100.2 -c 5
   

   Result:

   PING 192.168.100.2 (192.168.100.2) 56(84) bytes of data.
   64 bytes from 192.168.100.2: icmp_seq=1 ttl=63 time=30.2 ms
   64 bytes from 192.168.100.2: icmp_seq=2 ttl=63 time=28.4 ms
   64 bytes from 192.168.100.2: icmp_seq=3 ttl=63 time=31.6 ms
   64 bytes from 192.168.100.2: icmp_seq=4 ttl=63 time=27.4 ms
   64 bytes from 192.168.100.2: icmp_seq=5 ttl=63 time=27.6 ms
   
   --- 192.168.100.2 ping statistics ---
   5 packets transmitted, 5 received, 0% packet loss, time 4007ms
   rtt min/avg/max/mdev = 27.381/29.041/31.600/1.608 ms
   

   Network connectivity between the VPN clients has been established with zero packet loss.

Configure USB over IPConfigure USB over IP

The USB device will be delivered to the BareMetal server via usbip.

Configure a usbip serverConfigure a usbip server

The usbip server's role will be played by a remote site computer equipped with several USB ports. In this computer's terminal:

1. Install additional packages required for usbip:

   sudo apt install linux-tools-`uname -r`
   

2. Load the kernel modules required for usbip:

   sudo modprobe usbip-core
   sudo modprobe usbip-host
   sudo modprobe vhci-hcd
   

   Where:

   • usbip-core: Main server part component.
   • usbip-host: Component responsible for USB device management.
   • vhci-hcd: Component responsible for exporting USB devices.

   Note

   To set these modules up to load automatically at system startup, open the /etc/modules-load.d/modules.conf file in any text editor and add these lines to it:

   usbip-core
   usbip-host
   vhci-hcd
   

3. Insert USB devices into the computer ports.

   In this tutorial, we will use a USB flash drive and a YubiKey USB device as an example.

4. Request a list of USB devices available for publishing:

   sudo usbip list -l
   

   Result:

   - busid 1-1.2 (0951:1666)
     Kingston Technology : DataTraveler 100 G3/G4/SE9 G2/50 (0951:1666)
   
   - busid 1-1.3 (058f:a001)
     Alcor Micro Corp. : unknown product (058f:a001)
   
   - busid 1-1.4 (0cf3:3005)
     Qualcomm Atheros Communications : AR3011 Bluetooth (0cf3:3005)
   
   - busid 1-1.5 (1050:0407)
     Yubico.com : Yubikey 4/5 OTP+U2F+CCID (1050:0407)
   
   - busid 2-1.1 (0458:6001)
     KYE Systems Corp. (Mouse Systems) : GF3000F Ethernet Adapter (0458:6001)
   

5. Publish devices with busid 1-1.2 and busid 1-1.5:

   sudo usbip bind -b 1-1.2
   sudo usbip bind -b 1-1.5
   

   Result:

   usbip: info: bind device on busid 1-1.2: complete
   usbip: info: bind device on busid 1-1.5: complete
   

6. Run the usbipd daemon:

   sudo usbipd -4 -D
   

At this point, the selected USB devices are published and available for import over the network on the usbip client side.

Configure the usbip clientConfigure the usbip client

The role of the usbip client will be played by the BareMetal server named my-usbip-client.

1. Connect over SSH to the BareMetal server named my-usbip-client.

2. Install additional packages required for usbip:

   sudo apt install linux-tools-`uname -r`
   

   Note

   If using a Yandex Compute Cloud virtual machine as a usbip client, you should additionally install the linux-image-extra-virtual package:

   sudo apt install linux-image-extra-virtual
   

3. Load the kernel modules required for usbip:

   sudo modprobe usbip-core
   sudo modprobe usbip-host
   sudo modprobe vhci-hcd
   

   Where:

   • usbip-core: Main server part component.
   • usbip-host: Component responsible for USB device management.
   • vhci-hcd: Component responsible for exporting USB devices.

4. Request a list of USB devices available for import from the usbip server:

   usbip list -r 192.168.100.3
   

   Result:

   Exportable USB devices
   ======================
   - 192.168.100.3
         1-1.5: Yubico.com : Yubikey 4/5 OTP+U2F+CCID (1050:0407)
             : /sys/devices/platform/vhci_hcd.0/usb1/1-1/1-1.5
             : (Defined at Interface level) (00/00/00)
   
         1-1.2: Kingston Technology : DataTraveler 100 G3/G4/SE9 G2/50 (0951:1666)
             : /sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.2
             : (Defined at Interface level) (00/00/00)
   

5. Import devices from the usbip client:

   usbip attach -r 192.168.100.3 -b 1-1.2
   usbip attach -r 192.168.100.3 -b 1-1.5
   

At this point, the selected USB devices have been imported over the network to the BareMetal server.

Test the solutionTest the solution

To test the connection to your remote USB devices, connect over SSH to the BareMetal server my-usbip-client and perform these test actions in the terminal:

1. Run this command to view the dmesg log:

   dmesg
   

   Result:

   Flash drive

   YubiKey device

   ...
   [522540.280156] vhci_hcd vhci_hcd.0: pdev(0) rhport(0) sockfd(3)
   [522540.280165] vhci_hcd vhci_hcd.0: devid(65539) speed(3) speed_str(high-speed)
   [522540.280177] vhci_hcd vhci_hcd.0: Device attached
   [522540.500110] usb 3-1: new high-speed USB device number 2 using vhci_hcd
   [522540.618122] usb 3-1: SetAddress Request (2) to port 0
   [522540.671557] usb 3-1: New USB device found, idVendor=0951, idProduct=1666, bcdDevice= 0.01
   [522540.671571] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
   [522540.671577] usb 3-1: Product: DataTraveler 3.0
   [522540.671581] usb 3-1: Manufacturer: Kingston
   [522540.671585] usb 3-1: SerialNumber: D067E5162216F1B14605943F
   [522540.690082] usb-storage 3-1:1.0: USB Mass Storage device detected
   [522540.690801] scsi host7: usb-storage 3-1:1.0
   [522540.691020] usbcore: registered new interface driver usb-storage
   [522540.694262] usbcore: registered new interface driver uas
   [522541.728481] scsi 7:0:0:0: Direct-Access     Kingston DataTraveler 3.0      PQ: 0 ANSI: 6
   [522541.729122] sd 7:0:0:0: Attached scsi generic sg2 type 0
   [522541.763235] sd 7:0:0:0: [sdc] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)
   [522541.775808] sd 7:0:0:0: [sdc] Write Protect is off
   [522541.775829] sd 7:0:0:0: [sdc] Mode Sense: 4f 00 00 00
   [522541.788402] sd 7:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
   [522541.890019]  sdc: sdc1
   [522541.890454] sd 7:0:0:0: [sdc] Attached SCSI removable disk
   ...
   

   A Kingston USB device, which is a block flash drive, was mounted and made available for remote access on the server side. The device was recognized as /dev/sdc.

   ...
   [1039400.471187] vhci_hcd vhci_hcd.0: pdev(0) rhport(1) sockfd(3)
   [1039400.471211] vhci_hcd vhci_hcd.0: devid(65540) speed(2) speed_str(full-speed)
   [1039400.471223] vhci_hcd vhci_hcd.0: Device attached
   [1039400.640976] vhci_hcd: vhci_device speed not set
   [1039400.697969] usb 3-2: new full-speed USB device number 4 using vhci_hcd
   [1039400.763979] vhci_hcd: vhci_device speed not set
   [1039400.820985] usb 3-2: SetAddress Request (4) to port 1
   [1039400.872797] usb 3-2: New USB device found, idVendor=1050, idProduct=0407, bcdDevice= 5.43
   [1039400.872812] usb 3-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
   [1039400.872818] usb 3-2: Product: YubiKey OTP+FIDO+CCID
   [1039400.872822] usb 3-2: Manufacturer: Yubico
   [1039400.894510] input: Yubico YubiKey OTP+FIDO+CCID as /devices/platform/vhci_hcd.0/usb3/3-2/3-2:1.0/0003:1050:0407.0003/input/input7
   [1039400.977251] hid-generic 0003:1050:0407.0003: input,hidraw2: USB HID v1.10 Keyboard [Yubico YubiKey OTP+FIDO+CCID] on usb-vhci_hcd.0-2/input0
   [1039400.987196] hid-generic 0003:1050:0407.0004: hiddev0,hidraw3: USB HID v1.10 Device [Yubico YubiKey OTP+FIDO+CCID] on usb-vhci_hcd.0-2/input1
   ...
   

   A Yubico USB device was mounted and made available for remote access on the server side.

2. Make sure you have access to data on remote USB devices:

   Flash drive

   YubiKey device

   1. Get information about the block devices of the BareMetal server:

      lsblk /dev/sdc
      

      Result:

      NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
      ...
      sdc      8:32   1  7.2G  0 disk 
      └─sdc1   8:33   1  7.2G  0 part
      

   2. Mount the /dev/sdc1 flash drive partition to the /mnt/sdc1 directory:

      mkdir -p /mnt/sdc1
      mount /dev/sdc1 /mnt/sdc1
      

   3. View the list of mounted devices and disk space available for writing:

      df -h
      

      Result:

      Filesystem      Size  Used Avail Use% Mounted on
      ...
      /dev/sdc1       7.2G   16K  7.2G   1% /mnt/sdc1
      

   4. Copy the WireGuard configuration file to the remote USB drive:

      cp -r /etc/wireguard /mnt/sdc1
      

   5. View the list of files on the remote USB drive:

      ls -la /mnt/sdc1/wireguard/
      

      Result:

      total 48
      drwxr-xr-x 2 root root 16384 Apr 20 19:46 .
      drwxr-xr-x 4 root root 16384 Jan  1  1970 ..
      -rwxr-xr-x 1 root root   247 Apr 20 19:46 wg0.conf
      

   Verification completed: your file has been successfully copied to the remote USB drive.

   1. Install the utilities required to work with the YubiKey hardware token:

      apt install yubico-piv-tool
      

   2. Run a Yubico hardware status query:

      yubico-piv-tool -a status
      

      Result:

      Version:        5.4.3
      Serial Number:  ********
      CHUID:  No data available
      CCC:    No data available
      Slot 9c:
              Algorithm:      RSA2048
              Subject DN:     CN=5-ay-yubi
              Issuer DN:      CN=ChangeMe
              Fingerprint:    15e4ec25********************************************************
              Not Before:     Feb 19 08:29:13 2025 GMT
              Not After:      Feb 16 08:29:13 2035 GMT
      PIN tries left: 3
      

   3. Get the client's public certificate from the Yubico hardware token storage by specifying in the -s parameter the Slot field value from the previous command's output:

      yubico-piv-tool -a read-cert -s 9c
      

      Result:

      -----BEGIN CERTIFICATE-----
      MIIDUjCCA*******************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ****************************************************************
      ***************************************t7Ts9P8CTUsyw=
      -----END CERTIFICATE-----
      

   Verification completed: Yubico hardware returns the correct status; the certificate data from token storage can be read without errors.

How to delete the resources you createdHow to delete the resources you created

To stop paying for the resources you created:

1. Delete the VM.
2. You cannot delete a BareMetal server. Instead, cancel the server lease renewal.