• 1. What is Cloud Hypervisor?
  • Objectives
    • High Level
    • Architectures
    • Guest OS
• 2. Getting Started
  • Host OS
  • Use Pre-built Binaries
  • Packages
  • Building from Source
  • Booting Linux
    • Firmware Booting
    • Custom Kernel and Disk Image
      • Building your Kernel
      • Disk image
      • Booting the guest VM
• 3. Status
  • Hot Plug
  • Device Model
  • Roadmap
• 4. Relationship with Rust VMM Project
  • Differences with Firecracker and crosvm
• 5. Community
  • Contribute
  • Slack
  • Mailing list
  • Security issues

1. What is Cloud Hypervisor?

Cloud Hypervisor is an open source Virtual Machine Monitor (VMM) that runs on top of the KVM hypervisor and the Microsoft Hypervisor (MSHV).

The project focuses on running modern, Cloud Workloads, on specific, common, hardware architectures. In this case Cloud Workloads refers to those that are run by customers inside a Cloud Service Provider. This means modern operating systems with most I/O handled by paravirtualised devices (e.g. virtio), no requirement for legacy devices, and 64-bit CPUs.

Cloud Hypervisor is implemented in Rust and is based on the Rust VMM crates.

Objectives

High Level

• Runs on KVM or MSHV
• Minimal emulation
• Low latency
• Low memory footprint
• Low complexity
• High performance
• Small attack surface
• 64-bit support only
• CPU, memory, PCI hotplug
• Machine to machine migration

Architectures

Cloud Hypervisor's main supported architectures are x86-64 and AArch64, with functionality varying across these platforms. The riscv64 architecture support is experimental and offers limited functionality. For more details and instructions, please refer to riscv documentation.

Guest OS

Cloud Hypervisor supports 64-bit Linux and Windows guests, such as Windows Server 2025 and Windows 11. See the Windows documentation for the tested versions.

2. Getting Started

The following sections describe how to build and run Cloud Hypervisor.

Prerequisites for AArch64

• AArch64 servers (recommended) or development boards equipped with the GICv3 interrupt controller.

Host OS

For required KVM functionality and adequate performance the minimum supported host kernel version is 6.8. Cloud Hypervisor supports recent Linux distributions that ship this kernel version or a newer one.

Use Pre-built Binaries

The recommended approach to getting started with Cloud Hypervisor is by using a pre-built binary. Binaries are available for the latest release. Use cloud-hypervisor-static for x86-64 or cloud-hypervisor-static-aarch64 for AArch64 platform.

Packages

For convenience, packages are also available targeting some popular Linux distributions. This is thanks to the Open Build Service. The OBS README explains how to enable the repository in a supported Linux distribution and install Cloud Hypervisor and accompanying packages. Please report any packaging issues in the obs-packaging repository.

Building from Source

Please see the instructions for building from source if you do not wish to use the pre-built binaries.

Booting Linux

Cloud Hypervisor boots guests in one of two ways. The first is direct kernel boot, where a kernel image is passed to --kernel. The x86-64 kernel must be built with PVH support or be a bzImage. The second is firmware boot, where a firmware image is passed to --firmware and brings up the guest's normal boot loader.

Two firmware options are supported, and which one works best depends on the guest OS. Rust Hypervisor Firmware is a lightweight Rust-based PVH firmware. The edk2 UEFI firmware is called CLOUDHV.fd for x86-64 and CLOUDHV_EFI.fd for AArch64. Prebuilt binaries for both are available at their respective releases pages, Rust Hypervisor Firmware and our edk2 fork. The edk2 fork carries customizations required to boot AArch64 guests on cloud-hypervisor. See docs/uefi.md for differences with upstream tianocore/edk2.

Firmware Booting

Cloud Hypervisor supports booting disk images containing all needed components to run cloud workloads, a.k.a. cloud images.

The following sample commands will download a recent Ubuntu cloud image, converting it into a format that Cloud Hypervisor can use and a firmware to boot the image with. Cloud images of other recent distributions work as well.

$ wget -O cloudimg-amd64.img \
  https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
$ qemu-img convert -p -f qcow2 -O raw cloudimg-amd64.img cloudimg-amd64.raw
$ wget https://github.com/cloud-hypervisor/rust-hypervisor-firmware/releases/latest/download/hypervisor-fw

The Ubuntu cloud images do not ship with a default password so it necessary to use a cloud-init disk image to customise the image on the first boot. A basic cloud-init image is generated by this script, which must be run from the repository root. This seeds the image with a default username/password of cloud/cloud123. It is only necessary to add this disk image on the first boot. The script also assigns the static guest IP address 192.168.249.2 from test_data/cloud-init/ubuntu/local/network-config to the interface with the MAC address 12:34:56:78:90:ab. The --net option below matches this MAC address and assigns the host side IP address 192.168.249.1.

$ sudo setcap cap_net_admin+ep ./cloud-hypervisor
$ ./scripts/create-cloud-init.sh
$ ./cloud-hypervisor \
	--firmware ./hypervisor-fw \
	--disk path=cloudimg-amd64.raw,image_type=raw \
		path=/tmp/ubuntu-cloudinit.img,image_type=raw \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=12:34:56:78:90:ab,ip=192.168.249.1,mask=255.255.255.0"

If access to the firmware messages or interaction with the boot loader (e.g. GRUB) is required then it necessary to switch to the serial console instead of virtio-console.

$ ./cloud-hypervisor \
	--kernel ./hypervisor-fw \
	--disk path=cloudimg-amd64.raw,image_type=raw \
		path=/tmp/ubuntu-cloudinit.img,image_type=raw \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=12:34:56:78:90:ab,ip=192.168.249.1,mask=255.255.255.0" \
	--serial tty \
	--console off

Booting: --firmware vs --kernel

The following scenarios are supported by Cloud Hypervisor to bootstrap a VM, i.e., to load a payload/bootitem(s):

• Provide firmware
• Provide kernel [+ cmdline]\ [+ initrd]

Please note that our Cloud Hypervisor firmware (hypervisor-fw) has a Xen PVH boot entry, therefore it can also be booted via the --kernel parameter, as seen in some examples.

Custom Kernel and Disk Image

Building your Kernel

Cloud Hypervisor also supports direct kernel boot. For x86-64, a vmlinux ELF kernel (compiled with PVH support) or a regular bzImage are supported. In order to support development there is a custom branch; however provided the required options are enabled any recent kernel will suffice.

To build the kernel:

# Clone the Cloud Hypervisor Linux branch
$ git clone --depth 1 https://github.com/cloud-hypervisor/linux.git -b ch-7.2.8 linux-cloud-hypervisor
$ pushd linux-cloud-hypervisor
$ make ch_defconfig
# Do native build of the x86-64 kernel
$ KCFLAGS="-Wa,-mx86-used-note=no" make bzImage -j `nproc`
# Do native build of the AArch64 kernel
$ make -j `nproc`
$ popd

For x86-64, the vmlinux kernel image will then be located at linux-cloud-hypervisor/arch/x86/boot/compressed/vmlinux.bin. For AArch64, the Image kernel image will then be located at linux-cloud-hypervisor/arch/arm64/boot/Image.

Disk image

For the disk image the same Ubuntu image as before can be used. This contains an ext4 root filesystem.

# x86-64
$ wget -O cloudimg-amd64.img \
  https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
$ qemu-img convert -p -f qcow2 -O raw cloudimg-amd64.img cloudimg-amd64.raw
# AArch64
$ wget -O cloudimg-arm64.img \
  https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-arm64.img
$ qemu-img convert -p -f qcow2 -O raw cloudimg-arm64.img cloudimg-arm64.raw

Booting the guest VM

These sample commands boot the disk image using the custom kernel whilst also supplying the desired kernel command line.

• x86-64

$ sudo setcap cap_net_admin+ep ./cloud-hypervisor
$ ./scripts/create-cloud-init.sh
$ ./cloud-hypervisor \
	--kernel ./linux-cloud-hypervisor/arch/x86/boot/compressed/vmlinux.bin \
	--disk path=cloudimg-amd64.raw,image_type=raw \
		path=/tmp/ubuntu-cloudinit.img,image_type=raw \
	--cmdline "console=hvc0 root=/dev/vda1 rw" \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=12:34:56:78:90:ab,ip=192.168.249.1,mask=255.255.255.0"

• AArch64

$ sudo setcap cap_net_admin+ep ./cloud-hypervisor
$ ./scripts/create-cloud-init.sh
$ ./cloud-hypervisor \
	--kernel ./linux-cloud-hypervisor/arch/arm64/boot/Image \
	--disk path=cloudimg-arm64.raw,image_type=raw \
		path=/tmp/ubuntu-cloudinit.img,image_type=raw \
	--cmdline "console=hvc0 root=/dev/vda1 rw" \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=12:34:56:78:90:ab,ip=192.168.249.1,mask=255.255.255.0"

If earlier kernel messages are required the serial console should be used instead of virtio-console.

• x86-64

$ ./cloud-hypervisor \
	--kernel ./linux-cloud-hypervisor/arch/x86/boot/compressed/vmlinux.bin \
	--console off \
	--serial tty \
	--disk path=cloudimg-amd64.raw,image_type=raw \
	--cmdline "console=ttyS0 root=/dev/vda1 rw" \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=12:34:56:78:90:ab,ip=192.168.249.1,mask=255.255.255.0"

• AArch64

$ ./cloud-hypervisor \
	--kernel ./linux-cloud-hypervisor/arch/arm64/boot/Image \
	--console off \
	--serial tty \
	--disk path=cloudimg-arm64.raw,image_type=raw \
	--cmdline "console=ttyAMA0 root=/dev/vda1 rw" \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=12:34:56:78:90:ab,ip=192.168.249.1,mask=255.255.255.0"

3. Status

Cloud Hypervisor is under active development. The following stability guarantees are currently made:

• The API (including command line options) will not be removed or changed in a breaking way without a minimum of 2 major releases notice. Where possible warnings will be given about the use of deprecated functionality and the deprecations will be documented in the release notes.

• Point releases will be made between individual releases where there are substantial bug fixes or security issues that need to be fixed. These point releases will only include bug fixes.

• Starting with v54, snapshot/restore and live migration from the previous two major releases to the current release ("n-2") is guaranteed to work, excluding experimental features listed below. See Version Compatibility for details. Beware that reliable usage in a heterogeneous fleet need the use of a matching CPU profile, which are available since v55.

Currently the following items and experimental features are not guaranteed across updates because:

• Devices: vfio-user, vDPA
• Snapshot/Restore or migration: CPU profiles, postcopy.

Further details can be found in the release documentation.

Firmware boot works with the cloud images of recent distributions, for example Ubuntu and Windows Server, see Windows. Linux guests with Linux 5.18 or newer support all features that Cloud Hypervisor offers, such as virtio-mem on AArch64 and the VM Generation ID device, except reading the clock from the ACPI Time and Alarm Device, which needs Linux 7.1. Direct kernel boot to userspace should work with a rootfs from most distributions, although you may need to enable exotic filesystem types in the reference kernel configuration (e.g. XFS or btrfs.)

Hot Plug

Cloud Hypervisor supports hotplug of CPUs, passthrough devices (VFIO), virtio-{net,block,pmem,fs,vsock} and memory resizing. This document details how to add devices to a running VM.

Device Model

Details of the device model can be found in this documentation.

Roadmap

The project roadmap is tracked through a GitHub project.

4. Relationship with Rust VMM Project

In order to satisfy the design goal of having a high-performance, security-focused hypervisor the decision was made to use the Rust programming language. The language's strong focus on memory and thread safety makes it an ideal candidate for implementing VMMs.

Instead of implementing the VMM components from scratch, Cloud Hypervisor is importing the Rust VMM crates, and sharing code and architecture together with other VMMs like e.g. Amazon's Firecracker and Google's crosvm.

Cloud Hypervisor embraces the Rust VMM project's goals, which is to be able to share and re-use as many virtualization crates as possible.

Differences with Firecracker and crosvm

A large part of the Cloud Hypervisor code is based on either the Firecracker or the crosvm project's implementations. Both of these are VMMs written in Rust with a focus on safety and security, like Cloud Hypervisor.

The goal of the Cloud Hypervisor project differs from the aforementioned projects in that it aims to be a general purpose VMM for Cloud Workloads and not limited to container/serverless or client workloads.

The Cloud Hypervisor community thanks the communities of both the Firecracker and crosvm projects for their excellent work.

5. Community

The Cloud Hypervisor project follows the governance, and community guidelines described in the Community repository.

Contribute

The project strongly believes in building a global, diverse and collaborative community around the Cloud Hypervisor project. Anyone who is interested in contributing to the project is welcome to participate.

Contributing to a open source project like Cloud Hypervisor covers a lot more than just sending code. Testing, documentation, pull request reviews, bug reports, feature requests, project improvement suggestions, etc, are all equal and welcome means of contribution. See the CONTRIBUTING document for more details.

Slack

Get an invite to our Slack channel, join us on Slack, and participate in our community activities.

Mailing list

Please report bugs using the GitHub issue tracker but for broader community discussions you may use our mailing list.

Security issues

Please report security issues as described in SECURITY.md. Do not file a public issue for them.