02 / The execution & I/O contract

Real CPU execution. Cooperative devices.

KVM executes guest instructions. Go handles the device model. Shared guest memory connects the two, with MMIO registers for control and packed virtqueues for the work itself.

A request crosses the boundary

Virtio is a contract between a guest driver and a host device, not an emulation of a physical controller. The guest describes buffers in RAM; the device consumes them and reports completion.

Virtio request and completion loop A guest driver publishes a descriptor chain, notifies a virtio-MMIO device through a KVM exit, and receives an interrupt after the device updates the packed ring. Guest driverrunning on a vCPU Packed ring in RAMaddr · len · id · flags MMIO notifyKVM exit → bus Go devicedisk / net / console Mark buffers usedIRQ when required CONTROL: MMIO registers DATA: descriptor-addressed memory
  1. The driver fills descriptor addresses, lengths and flags, then makes a chain available using the packed ring's availability and wrap bits.
  2. A write to QueueNotify exits guest execution into the VMM. The MMIO bus routes the access to the device that owns that address range.
  3. The device follows descriptors into guest memory and performs the backend operation. Indirect descriptors let a ring entry reference a separate descriptor table.
  4. The device returns used lengths and advances its packed-ring cursor. Event suppression determines whether to signal an interrupt; the guest reclaims completed buffers.
A conceptual request lifecycle. Net RX is backend-driven after the guest supplies receive buffers; not every operation needs a fresh notification or interrupt.

What a vCPU actually does

A vCPU has its own KVM state and a run loop associated with a host OS thread. KVM_RUN enters guest execution and returns for exits that require userspace handling, such as I/O ports and MMIO. Ordinary guest computation is not interpreted instruction-by-instruction in Go.

Guest physical RAM is mapped into the VMM and registered with KVM. The guest CPU and device handlers therefore see the same backing memory. Guest virtual addresses still pass through the guest's page tables; a descriptor contains a guest physical buffer address.

The architecture layer creates the interrupt controller and PIT, configures CPUID for the selected topology, and prepares per-vCPU state. Pause requests make the run loops leave KVM and wait at a synchronization point.

The bus is MMIO, not PCI

The virtio-MMIO bus assigns each device a 4 KiB register window starting at 0xd0000000 and an IRQ starting at 5, incrementing for each attachment. The boot loader advertises those assignments to Linux in the kernel command line.

Discover → negotiate → configure → run

  • Identity: magic, transport version and device ID tell the driver what it found.
  • Features: device and driver feature registers establish the shared capabilities.
  • Queues: queue selection, size, guest-memory addresses and ready state configure each ring.
  • Notifications: queue notify, interrupt status and interrupt acknowledgement coordinate work.
Packed queues specifically. This implementation requires the modern version, packed-ring, indirect-descriptor and event-index features. It does not implement split virtqueues. In a packed ring, availability and completion live in the same descriptor ring, with index and wrap state distinguishing successive passes.

Why the cursor is part of a checkpoint

RAM contains the descriptors, but it is not the entire queue state. The host also tracks available/used indexes and wrap bits. Restoring only RAM could process a descriptor twice or skip it. The checkpoint includes queue cursors and transport registers, and the bus's I/O gate quiesces handlers while the state is captured.

One transport, different backends

Block interprets sector requests; net moves Ethernet frames; console carries character streams. VSock provides another guest/host communication device, but live VSock state is not supported by portable checkpoints. The common MMIO/queue layer handles transport mechanics, not the meaning of each device's payload.

VM execution loopMMIO busPacked queueDevice features