Amortize per-row execution overhead
Vectorized execution passes bounded batches of values between operators instead of invoking the entire operator pipeline separately for every row. During a million-row scan, repeated function calls, interpretation and type checks can consume substantial work beyond the arithmetic itself. A batch containing 1,024 values can amortize some control overhead across those values. The execution vector is a software batch; it does not mean that one CPU instruction processes all 1,024 rows simultaneously.
Trace a filter through a selection vector
For values [3, 8, 1, 9], filtering for values greater than five produces selected positions [1, 3]. A downstream operator can consult the original array through those positions or compact the survivors into [8, 9]. Selection vectors can avoid copying, but introduce indirect access. If nullness is stored in a separate validity bitmap, values, positions and validity must remain aligned. Rearranging values without the corresponding metadata can change query results rather than merely changing performance.
Distinguish batching, SIMD and parallel workers
Homogeneous contiguous values may make hardware SIMD easier to exploit, but variable-length strings, branch-heavy functions and dependent updates can limit that opportunity. Floating-point reductions can also produce different rounding when evaluation order changes. Software vectorization, SIMD instructions and multithreaded parallelism are separate design axes. A single-threaded batch engine may benefit from fewer calls without using SIMD, while a SIMD kernel may still operate inside a larger vectorized operator. Verify each claim independently.
Bound batches to preserve locality and responsiveness
Larger batches amortize more call overhead, but can enlarge intermediate state beyond cache capacity and delay the first result. A size such as 1,024 is an illustrative design choice, not a universal optimum. Column width, concurrency and operator state alter the trade-off. Compare identical query results while measuring peak intermediate memory and time to first output as well as total completion time. Tiny inputs can favor a simpler row-oriented path because setup costs dominate.
Batch boundaries are also opportunities to check cancellation. If a consumer stops after enough rows, continuing every remaining batch can waste substantial work. Decide how operators signal end-of-demand and how quickly expensive kernels can observe it. This is another reason to evaluate responsiveness along with throughput: minimizing per-row control overhead should not silently remove every practical opportunity to stop work that is no longer needed.