Systems and protocols · Intermediate

virtio: exchanging device requests through shared queues

Trace virtqueue descriptors and ownership, separating notification, completion and persistence.

Ask Logeia about this articleTry the three questionsReview your answers

Updated

On this page
  1. A common device communication contract
  2. Descriptors identify buffers
  3. Publication transfers a responsibility
  4. Notification is not completion
  5. Investigate transport and bottlenecks
  6. Ask Logeia about this articleCheck your understanding

A common device communication contract

Virtio standardizes communication between drivers and devices. A virtual machine can use an agreed request queue instead of reproducing every register behavior of a particular historical device. Network and block devices assign different meanings to requests, while sharing mechanisms for presenting buffers. The abstraction concerns a communication contract, not a promise that all devices implement identical operations.

A virtual device also need not be serviced by one particular userspace process. Backend placement and implementation depend on the configuration. Compliant physical devices can use virtio too. Separate the standardized communication model, its transport and the component that performs an operation. That distinction is essential when attributing CPU work or deciding which component needs investigation.

Descriptors identify buffers

The driver supplies memory buffers and describes their locations, lengths and permitted access directions through descriptors. A descriptor is metadata about a buffer rather than a copy of the entire payload. Several descriptors may form one request, and device-readable memory must be distinguished from memory into which the device may write. Direction is part of the request contract.

For a conceptual block read, distinguish request information, destination data and completion status. Free descriptor count is not the same quantity as available transfer bytes or application request count. One request may consume several descriptors. Capacity planning should therefore examine actual request shapes and any indirect descriptor mechanism in use instead of assuming a fixed one-request-to-one-slot correspondence.

Publication transfers a responsibility

Consider the split-ring format as an explanatory model. The driver prepares descriptors and buffers, publishes an available request and notifies the device when required. The device processes available requests and reports used information. The driver observes the corresponding completion before recycling the affected descriptors and buffers. Other queue formats organize these details differently.

Premature reuse can make the device observe different data or write into memory assigned to another operation. Buffer lifetime and memory-ordering rules are therefore part of correctness. Writing a notification value alone is not a substitute for following the publication protocol. A queue diagram should identify when each participant may access memory, rather than showing only arrows between two components.

Notification is not completion

Notifying a device announces work; it does not establish that the work has finished or that an application-level objective has been met. An interrupt in the other direction need not correspond to exactly one request either. Notification suppression and batching affect the relationship between requests, notifications and interrupts. Count those quantities separately when interpreting instrumentation.

Imagine four requests published with one notification and three completions later reclaimed. The remaining request still owns its outstanding resources. Inferring completion count from notification count would lose that distinction. For block writes, persistence additionally depends on the device and storage-cache contract, including applicable flush operations. Queue completion should not silently be promoted to a guarantee about survival of every kind of failure.

Investigate transport and bottlenecks

PCI and MMIO are examples of transports through which a virtio device can be discovered and configured. Transport is a different axis from device type. Feature negotiation determines the capabilities actually available to a driver and device pair; a driver knowing about a feature does not establish that its current counterpart supports it. Inspect the negotiated configuration.

For slow requests, distinguish lack of free descriptors, backend waiting and delayed completion reclamation by the guest. Adding queues cannot eliminate a shared CPU or storage bottleneck. Shared memory also does not imply universally zero copying, translation or synchronization cost. Form a hypothesis about the delayed stage and collect evidence there before attributing every latency difference to the virtio abstraction itself.

Reading and practice are recorded separately in this browser, for each language.

Choose a learning path