Systems and protocols · Intermediate

KVM: the boundary between virtual CPUs and a userspace VMM

Follow KVM file descriptors and execution to distinguish virtual CPUs, guest memory and device handling.

Ask Logeia about this articleTry the three questionsReview your answers

Updated

On this page
  1. Separate execution from the whole machine
  2. Follow the descriptor hierarchy
  3. Keep address spaces distinct
  4. Interpret a return from KVM_RUN
  5. Diagnose each layer separately
  6. Ask Logeia about this articleCheck your understanding

Separate execution from the whole machine

KVM exposes a Linux kernel interface for virtualization. A userspace virtual machine monitor uses that interface to create and control a guest. Executing guest instructions is only part of presenting a complete computer. Firmware, boot configuration, storage contents and device behavior still need appropriate implementations and configuration. Treating QEMU and KVM as interchangeable names obscures which component owns a failure.

For a guest reading a virtual disk, hardware-assisted CPU execution does not independently determine the bytes returned by that disk. The monitor, kernel facilities and device backends divide the work. Start an investigation by locating the missing operation within that division rather than assuming that every guest problem originates in CPU virtualization.

Follow the descriptor hierarchy

Opening /dev/kvm yields a system descriptor. KVM_CREATE_VM creates a descriptor for one virtual machine, and KVM_CREATE_VCPU on that machine produces a descriptor for a virtual CPU. These descriptors identify different objects, so an ioctl must be issued to the kind of descriptor specified by its contract. The hierarchy is useful for understanding both resource ownership and errors.

Imagine creating one machine with two virtual CPUs. That operation does not reserve two physical cores. Host scheduling and resource policies still govern when the virtual CPU threads run. Distinguish a guest-visible processor count from host capacity, placement and guaranteed execution time. Otherwise, a configuration that looks parallel inside the guest can be mistaken for dedicated physical resources.

Keep address spaces distinct

A guest physical address is not a promise to access the same numeric host physical address. In a typical userspace monitor, memory is allocated in the monitor address space and registered as backing for guest physical ranges. The guest operating system separately manages its own virtual-to-physical translation. Understanding the layers prevents misleading interpretations of an address appearing in a guest log.

A read at guest address 0x1000 therefore needs context before it can identify a host location. Examine the guest mapping, registered memory regions and applicable access permissions. A guest page fault and host memory pressure are also different observations. They can interact operationally, but one name should not substitute for evidence that identifies the layer where progress stopped.

Interpret a return from KVM_RUN

The monitor invokes KVM_RUN on a virtual CPU to advance guest execution. First check the ioctl return value. A zero return makes exit_reason available for the corresponding handling; an error requires errno-based handling. Do not interpret a stale exit_reason after an error return. Some returns require userspace to handle port or memory-mapped I/O. Other returns need different handling, including interruption or a guest shutdown condition. A return alone does not prove successful guest operating-system termination.

Nor does every hardware VM exit necessarily become a userspace return. Some events can be handled inside the kernel before execution resumes. The conceptual monitor loop is run, inspect, handle and resume; it is not a literal inventory of every internal transition. Debugging counters must be interpreted according to what they count, especially when comparing hardware events with userspace handling work.

Diagnose each layer separately

For a machine that does not boot, check access to /dev/kvm, required extensions, guest memory registration, initial virtual CPU state and boot data. Query supported capabilities instead of inferring every extension from a kernel version string. Hardware support and permission to use that support in the current environment are separate requirements, particularly inside another virtualized environment.

For performance, separate guest waiting from host scheduling and device processing. For isolation, account for the host kernel, monitor and applicable backends in the trust boundary. Using KVM alone does not establish confidential-computing properties or immunity to implementation flaws. A useful architecture description states which components are trusted and what observation would identify a failure of each one.

Reading and practice are recorded separately in this browser, for each language.

Choose a learning path