Concepts and theorems · Intermediate

NUMA locality: place memory near the CPUs that use it

Reason about CPU placement, page allocation and bandwidth instead of treating NUMA as one latency number.

Ask Logeia about this articleTry the three questionsReview your answers

Updated

On this page
  1. Memory access is not uniform
  2. Follow first allocation and use
  3. Choose a placement policy
  4. Measure placement and demand
  5. Ask Logeia about this articleCheck your understanding

Memory access is not uniform

On a NUMA system, the route from a CPU to memory is not uniform. Memory belongs to nodes connected by an interconnect, and remote accesses can have different latency or bandwidth from local accesses. This is not a question of whether a virtual address is numerically high or low. A program can use the same pointer while its execution CPU or backing page placement changes. Analyse computation placement and data placement together. The relevant topology can be more complicated than two identical sockets, and exact costs depend on the machine and current traffic.

Follow first allocation and use

Imagine two NUMA nodes and a large array initialized entirely by a thread running on node 0. If the allocation policy and available memory place the resulting pages there, a later worker on node 1 may repeatedly access remote pages for its half of the array. Having each worker initialize its own range can improve locality under suitable first-touch behaviour. Do not turn this example into an unconditional rule: allocation mode, existing mappings, memory pressure and operating system policy matter. A read-only first access can also behave differently from the write that allocates a private page.

Choose a placement policy

Linux memory policies can prefer particular nodes or interleave allocations across a set of nodes. Local placement can reduce remote access, but concentrating every worker on one node can exhaust its capacity or bandwidth. Interleaving can spread traffic across memory controllers when a shared workload benefits from aggregate bandwidth. CPU affinity alone does not necessarily relocate pages that already exist. Page migration has costs of its own, including copying and coordination. For a short-running task, the cost of rearranging data can outweigh the benefit obtained during the remaining computation.

Measure placement and demand

Compare runs with the same input, worker count and initialization strategy. Inspect node placement and relevant remote-access indicators alongside elapsed work, rather than treating a changed counter as proof of faster execution. A compute-bound phase may gain little from improved locality. Frequently updated shared data also involves cache-coherence traffic, which is distinct from simply fetching a remote DRAM page. Record CPU migration, page sizes and frequency behaviour when explaining a result. A placement experiment is informative when the proposed mechanism matches both the observed data movement and the application’s overall outcome.

Reading and practice are recorded separately in this browser, for each language.

Learn this in order

Build your CS foundations · STEP 16

Understand the systems behind the web · STEP 6

Prepare for AI systems · STEP 6

Choose a learning path