Study guides · Intermediate

Reason about performance ceilings with Roofline

Calculate computation and bandwidth ceilings with units, distinguish nominal bytes from actual traffic, and interpret resident and larger working sets.

Ask Logeia about this articleTry the three questionsReview your answers

Updated

On this page
  1. Place computation and traffic in one model
  2. Substitute illustrative values with units
  3. Count nominal work from the array contract
  4. Separate resident and larger working sets
  5. Calibrate ceilings for the actual configuration
  6. Investigate performance below both ceilings
  7. Compare two changes within one declared model
  8. Solve changed-ceiling and byte-count exercises
  9. Ask Logeia about this articleCheck your understanding

Place computation and traffic in one model

When an array kernel is slow, the next useful change depends on whether it lacks arithmetic throughput or waits for data. Roofline compares a computational ceiling with a bandwidth ceiling and uses the lower bound as a guide to attainable performance. It does not promise that a program reaches that ceiling. Read the cache, SIMD and unit-conversion material first. Before reading benchmark speed, specify how operations are counted and which boundary the bytes cross.

Intensity I is operations divided by bytes moved across the chosen boundary, measured in FLOP/B. Performance is operations per time, commonly GFLOP/s. The original model uses operational intensity based on DRAM traffic after cache filtering. A nominal count derived from source-level loads and stores is not necessarily that traffic. Reuse inside a cache can make these quantities differ substantially. Keep the boundary explicit so a cache-resident measurement is not mistakenly described as violating a main-memory bandwidth limit.

Substitute illustrative values with units

Assume a computational ceiling P of 200 GFLOP/s, bandwidth B of 50 GB/s and intensity I of 0.5 FLOP/B. Multiplying bandwidth by intensity gives 25 GFLOP/s. Bytes cancel, leaving operations per second. The model ceiling is min(P, B×I), so it is 25 GFLOP/s here. All three inputs are teaching assumptions, not measured properties of the CPU running the companion experiments.

The intersection is P/B, or four FLOP/B. Below four, the bandwidth line is lower; above four, the 200 GFLOP/s computation ceiling is lower. At intensity eight the bandwidth product is 400, but the lower ceiling remains 200. It is not a prediction that the machine can deliver 400. Use compatible decimal GB and GFLOP units, or explicitly convert binary GiB. Similar-looking unit labels cannot be mixed without changing the numerical result. The calculation identifies a limiting model constraint rather than proving the cause of an observed program delay.

Intensity (FLOP/B)Bandwidth bound (GFLOP/s)Lower ceiling
0.52525
4200200
8400200

Count nominal work from the array contract

For single-precision out[i]=a[i]+b[i], count one addition, eight input bytes and four output bytes per element. The nominal source-level intensity is one twelfth FLOP/B. Double precision changes the nominal bytes to 24, giving one twenty-fourth. Actual traffic can include write allocation and later writeback, while cache reuse can avoid some DRAM reads. This is a useful specification-derived count, not a memory-controller measurement.

The intensity experiment reads one float, performs x=0.5x+0.25 q times in a register and writes it once. With fusion disabled, count 2q operations and eight nominal bytes, giving q/4 FLOP/B. Values of q equal to one and sixteen produce 0.25 and four. Adding arithmetic changes the workload; it is not itself an optimization solving the same problem faster. Use it to explore work per nominal transfer, and inspect compilation to verify that repeated computation has not been removed or combined in a way inconsistent with the stated count.

Separate resident and larger working sets

Repeated traversal can change the relevant boundary: a first pass may require memory traffic while later passes reuse cache contents. Nominal bandwidth from a small array can exceed a reference DRAM bandwidth without violating physics, because the quantities describe different boundaries. Hierarchical Roofline analysis can use separate bandwidths for L1, L2 and main memory. Count the complete working set, including all inputs, outputs and auxiliary state, rather than only one named array.

Save small and large cases separately. Calling a buffer large does not establish that it exceeds every relevant cache; compare size, sharing and concurrent activity. Virtualized configuration may not expose every aspect of host use, so report the actual allocation sizes and timings without manufacturing a clean residency claim. If an experiment claims to flush a cache, it must specify the operation and level. The first recorded call is not proof that all hierarchy levels began empty. This distinction matters when deciding which bandwidth ceiling can legitimately be compared with a sample.

Calibrate ceilings for the actual configuration

A machine-specific ceiling must match precision, operation-count convention, core count and instruction set. A whole-chip advertised peak is not automatically a single-thread loop's ceiling. Similarly, bandwidth should reflect sustained traffic under comparable thread count and placement. Distinguish a theoretical calculation from a dedicated empirical kernel. Variable frequency also separates fixed-clock calculations from performance measured over elapsed time.

This material keeps 200 and 50 as illustrative inputs. It does not calibrate a hardware Roofline from the companion timings, and without PMU traffic counts it does not measure DRAM intensity. The present evidence verifies output, executed instructions and nominal work and bytes. If counters are added later, record events, interval, concurrent activity, multiplexing and units. Do not merely rename an old nominal field as measured traffic. A useful model is explicit about which inputs were assumed, calculated or observed, rather than presenting every number with the same evidential status.

Investigate performance below both ceilings

Observed performance below the lower ceiling does not automatically invalidate the model. A dependency chain, absent SIMD, insufficient concurrent loads, short-call overhead or remote NUMA placement can prevent approaching it. A ceiling does not supply an implementation that uses every resource well. Verify equal correct work, then vary instructions, layout and repetition conditions one at a time to identify actionable constraints.

Reducing bytes for equal operations can move a point toward greater intensity. SIMD can improve effective arithmetic throughput. Representation conversion, copying and precision changes still belong in the complete cost and correctness contract. Adding useless arithmetic can raise a reported FLOP rate without completing the application sooner. Evaluate equal useful work by completion time and resources instead of optimizing the appearance of a GFLOP/s figure. The model is most useful when it narrows a concrete next investigation while preserving the actual task being optimized.

Compare two changes within one declared model

At intensity 0.5, doubling computation ceiling from 200 to 400 leaves the bound at 25 GFLOP/s, while doubling bandwidth from fifty to one hundred raises it to fifty. This model points toward a bandwidth-side investigation. Reducing precision to shrink bytes is a different candidate requiring the original accuracy contract to remain acceptable. A favorable bound does not authorize changing useful output. Compare a proposed resource improvement and a changed workload separately, then verify which one actually meets the application requirement.

Solve changed-ceiling and byte-count exercises

Keep P=200 GFLOP/s and B=50 GB/s, but raise intensity from 0.5 to two. The ceiling becomes min(200,100)=100 GFLOP/s. A fourfold increase in this bound does not guarantee a fourfold speedup: an implementation may still face constraints below the bound. Next use bandwidth 100 GB/s at intensity eight. The lower ceiling remains 200 because computation is now limiting in the illustrative model.

For the nominal loop reading and writing one float, let q equal eight and count each multiplication and addition separately. Sixteen operations divided by eight nominal bytes gives two FLOP/B. Can this be called measured DRAM intensity? No: cache reuse and store traffic can change actual movement. The learning outcome is the ability to calculate the bound while naming the boundary and status of its inputs. The arithmetic-intensity experiment collects samples while preserving that separation.

Reading and practice are recorded separately in this browser, for each language.

Learn this in order

Build your CS foundations · STEP 8

Prepare for AI systems · STEP 5

Choose a learning path