Goal: announce completion and return ownership
After producing data, a device must still let the guest inspect the result and reuse its buffer. Follow the end of notify through inject, handle_guest_irq, and used-ring reclaim. Separate transfer, publication, interrupt reception, device acknowledgement, GIC completion, and buffer reclaim rather than compressing them into one completion event.
The IRQ-forty path from wave four now carries a block-I/O result. QEMU GICv3 delivers it to the guest handler, which reads returned state. The scope remains one trusted guest with one request outstanding. Multiple CPUs, many simultaneous requests, interrupt coalescing, and suppression are outside the tested subset.
Publish before notifying
notify writes data and status, then places the head identifier and length in a used slot. After ordering memory, it increments produced and publishes used.idx. Only afterward does it set device interrupt state and inject the IRQ. The ordering ensures that notification does not advertise a result that remains unpublished.
Volatile accesses and barriers have different roles. Volatile causes specified memory accesses to occur; ordering the visibility of multiple shared accesses is a separate concern. A working single-vCPU experiment is not evidence that barriers are unnecessary on arbitrary CPUs. List the content that must be complete before publishing idx.
Separate device ACK from GIC EOI
inject places Pending IRQ forty in ICH_LR0_EL2. handle_guest_irq reads ICC_IAR1_EL1, reads MMIO InterruptStatus, and distinguishes used notification bit zero from configuration-change bit one. Configuration change also reads device status before acknowledging the observed bits through InterruptACK. That clears the device notification cause; receiving the interrupt does not itself clear device interrupt state.
The handler then writes ICC_EOIR1_EL1 and ICC_DIR_EL1 for GIC completion and deactivation under split EOI. MMIO acknowledgement, GIC EOI, and used-ring reclaim advance distinct state. Read handle_guest_irq and identify which writes target the device and which target the interrupt controller. One completed action does not substitute for the others.
Guest reclaim: read returned state from used
After delivery, the guest orders memory and reads used.idx, the slot id and len, status, and data. GUEST_RECLAIM records those observations. It reads the values returned through shared memory rather than displaying precomputed expectations. A whole-buffer hash additionally checks the 512-byte transfer.
The used id identifies the descriptor-chain head, not the disk sector. A normal example where slot, head, and sector are all zero can conceal this distinction. In the nine-request case, slots change while the head is reused. Explain each identifier before reusing descriptors and buffers after reclaim.
Distinguish success, request error, and structural failure
A successful read completes with status zero and used.len 513. Range and write rejection complete with status one and used.len one; both still notify and are reclaimed. A cyclic chain instead marks DEVICE_NEEDS_RESET and signals configuration change without advancing used. The guest observes GUEST_CONFIG_CHANGE with status seventy-nine and InterruptStatus two, without reclaiming a used result. IRQ presence alone distinguishes neither request success nor request completion.
Investigate used advancement, device cause, GIC injection, guest reception, acknowledgement, and reclaim in order. Locate the first missing boundary before changing code. Simultaneously editing the copy loop and GIC setup in response to a missing final message makes the cause harder to identify.
| Outcome | Used publication | Request status | IRQ and reclaim |
|---|---|---|---|
| Successful read | Advances with len 513 | 0 | Present |
| Range/write rejection | Advances with len 1 | 1 | Present |
| Cyclic descriptor chain | Does not advance | No safe return target established | Configuration-change IRQ; no used reclaim; device status 79 |
Runnable Rust: notify and reclaim once
This standalone host model captures publication before notification and one reclaim per result. Both status zero and one pass through completion; unpublished or already reclaimed results return None. Running both statuses through the same sequence separates interrupt handling from request success.
The boolean is not a simulation of GIC Pending and Active states. Barriers, exception vectors, and ICC registers are absent. Use it to check a small invariant, then verify real IRQ forty and reclaim in the distributed machine recording. Keep model values separate from observed machine readback.
#[derive(Debug, Default)]
struct Completion { published:u16, reclaimed:u16, interrupt:bool, status:Option<u8> }
impl Completion {
fn publish(&mut self,status:u8) {
assert_eq!(self.published,self.reclaimed);
self.status=Some(status);
self.published=self.published.wrapping_add(1);
}
fn notify(&mut self) {
assert_eq!(self.published.wrapping_sub(self.reclaimed),1);
self.interrupt=true;
}
fn ack(&mut self) { assert!(self.interrupt); self.interrupt=false; }
fn reclaim(&mut self)->Option<u8> {
if self.published==self.reclaimed { return None; }
self.reclaimed=self.reclaimed.wrapping_add(1);
self.status.take()
}
}
fn main() {
let mut c=Completion::default();
assert_eq!(c.reclaim(),None);
c.publish(0); assert!(!c.interrupt);
c.notify(); c.ack(); assert_eq!(c.reclaim(),Some(0));
assert_eq!(c.reclaim(),None);
c.publish(1); c.notify(); c.ack();
assert_eq!(c.reclaim(),Some(1));
assert_eq!((c.published,c.reclaimed),(2,2));
println!("success=completed IOERR=completed duplicate-reclaim=none");
}Read nine requests: reuse versus duplicate processing
queueBoundary completes publication, notification, and reclaim for one request before starting another. Nine QUEUE_TAKE, USED_PUBLISH, guest IRQ receptions, and GUEST_RECLAIM observations correspond while slot zero is reused after seven. Reusing a slot is valid; overwriting before reclaim or consuming the same publication twice is not.
Check totals and per-request ordering. A matching interrupt total is insufficient if notifications correspond to the wrong requests. Preserve the case-specific source identity, ELF, and raw UART hash, and compare new captures against their own artifacts. Moving browser playback advances a saved recording rather than executing another machine run. queueSpurious is the complementary no-work control. When a duplicate or empty notification finds available.idx equal to consumed, QUEUE_IDLE is recorded and no used publication, IRQ, reset, or queue-state change follows. A missing IRQ is correct for that input. Contrast it with malformed-chain configuration notification and with successful or IOERR used-buffer notification before assigning a meaning to an empty completion list.
Exercise: inspect notification cause and reclaim position
For one normal request, tabulate changes to data, status, used element, used.idx, device interrupt, LR0, and guest reclaim position. Attach the responsible function to each row. Contrast omitting InterruptACK with omitting used.idx publication: predict which state remains and which observation exposes it. Include the request sequence number alongside the descriptor head and used slot when drawing the ownership table. The sequence distinguishes successive uses of the same resources; it is not itself the on-disk sector or the returned descriptor identifier. For a repeated notification after reclaim, the previous sequence remains complete and no new ownership transfer begins. That is different from receiving a second completion for one publication.
For an extension, repair an implementation that displays success immediately upon IRQ. Read used, validate id, len, status, and buffer, then determine the outcome. Submit comparisons for success, completed IOERR, and malformed structure, together with machine-versus-host evidence boundaries. Trace one I/O from admission to reclaim through source to connect the entire wave-five path.