Experiments and implementations · Intermediate

Observing minor and major page faults

Compare process fault counters before and after first access to an anonymous mapping, and distinguish the meaning of minor and major faults from unsupported conclusions about I/O.

Ask Logeia about this articleTry the three questionsReview your answers

Updated

On this page
  1. Use a small process-local experiment
  2. Read first, then write
  3. Interpret the categories precisely
  4. Remember what is inside the interval
  5. Do not force a major fault by harming the host
  6. Ask Logeia about this articleCheck your understanding

Use a small process-local experiment

Run on Linux with Python’s standard mmap and resource modules. Create a sixteen-mebibyte anonymous mapping without administrator privileges or an external data file. Do not clear the host’s global page cache. Record cumulative fault counters before access, after reading and after writing. The experiment concerns operating-system work associated with memory access, not an exception that necessarily terminates the program. Many page faults are handled transparently and execution resumes. Keep the allocation bounded so the experiment does not induce host-wide memory pressure. Record the Python and kernel versions when comparing runs, because allocation and mapping policy are relevant parts of the environment.

Read first, then write

Obtain the host base page size and read one byte at each corresponding offset. Verify that the new anonymous region reads as zero, then write one at the same offsets. Keep the read-phase and write-phase counter differences separate. Supplying zero-filled data to a reader and supplying writable backing to a writer need not require identical work. Obtaining the base page size avoids assuming 4096 universally, but it does not enumerate every mapping size the kernel may actually use in this region. Transparent huge pages and other policies can affect the observed counts. The code checks the values it reads and writes while leaving exact counter values as observations.

python
import mmap, os, resource
size = 16 * 1024 * 1024
page = os.sysconf('SC_PAGE_SIZE')
def faults():
    r = resource.getrusage(resource.RUSAGE_SELF)
    return r.ru_minflt, r.ru_majflt
print('page', page, 'before', faults())
with mmap.mmap(-1, size) as region:
    assert all(region[i] == 0 for i in range(0, size, page))
    print('after-read', faults())
    for i in range(0, size, page):
        region[i] = 1
    assert all(region[i] == 1 for i in range(0, size, page))
    print('after-write', faults())

Interpret the categories precisely

The minor-fault counter describes faults serviced without needing to load the page from storage; major faults require such input. Minor does not mean free: page-table changes, allocation and other kernel work can still consume time. Conversely, a zero major-fault count does not prove that the process performed no disk I/O of any kind. These are fault statistics, not a complete I/O ledger. Do not reinterpret a fault count as an elapsed duration or as the number of all physical pages owned by the process. If a performance question requires latency, add a separately defined timing measurement and keep its unit and observation interval explicit.

Remember what is inside the interval

getrusage returns cumulative values, so subtract consecutive snapshots to obtain phase differences. The same process also executes Python, allocates objects and prints results. Not every additional fault can be attributed one-to-one to an offset visited in the mapping loop. Compare first and later access, region size and access order while changing one condition at a time. Preserve the actual observations rather than adjusting them to match a page-count prediction. A disagreement can reflect measurement scope or mapping policy rather than an incorrect definition of a page fault. A separate fresh process can reduce some history effects, but it still does not turn the interpreter into a measurement-free observer.

Do not force a major fault by harming the host

A zero major-fault delta is an ordinary possible result for this anonymous-memory experiment. The procedure deliberately excludes clearing shared caches or exhausting memory to manufacture a particular counter value. Even a file mapping might avoid storage input when the required pages are already cached. Record the timestamp, region size, base page size and both counters at every phase. Success means that the program checked initialization and writes, separated observation intervals and retained the observed values. Matching somebody else’s published counter totals is not a success requirement. Any later file-backed experiment should specify its cache assumptions and isolation separately rather than silently changing the meaning of this run.

Reading and practice are recorded separately in this browser, for each language.

Learn this in order

Build your CS foundations · STEP 16

Understand the systems behind the web · STEP 6

Prepare for AI systems · STEP 6

Choose a learning path