Identify what is reduced
Quantization maps weights, activations, or other tensors onto a limited value set, changing numerical representation. Pruning removes selected weights or structures such as channels. Unstructured zeroing and structured removal of rows, channels, or heads have different execution consequences. Both may pursue smaller models, but they change different things. State what is stored and what computation is actually avoided. Neither operation unconditionally preserves task quality, and the term compression does not establish memory reduction or speed without specifying a representation and an implementation that use the change.
Estimate dense low-bit storage
A four-by-four matrix has sixteen elements, so float32 payload is sixty-four bytes. Ideally packing every element into four bits gives eight payload bytes, excluding scales, zero points, alignment, and other metadata. A list of sixteen Python integers does not occupy eight bytes. Execution may reconstruct higher-precision values or accumulate in another dtype. Distinguish storage bits, arithmetic type, and accumulation type. The ideal payload calculation explains a possible representation advantage; it is not a measured reduction in allocated memory for a real framework or inference kernel.
Count sparse metadata
Keeping only four nonzeros in the same matrix does not change its sixty-four-byte payload if the array remains dense float32. In a simple COO representation, each nonzero might store a four-byte value and two four-byte indices, totaling forty-eight bytes for four entries. Other formats and index widths change that result. Zero fraction alone therefore does not determine storage reduction. Include structural metadata and check that the sparse representation encodes the intended matrix. This tiny accounting exercise shows why sparsity and compression ratio are related but not numerically identical concepts.
Inspect structured removal
Actually removing two of four rows creates a two-by-four matrix with thirty-two float32 payload bytes. Its output dimension also changes, requiring compatibility with subsequent layers or residual paths. Removing attention heads similarly requires corresponding projection and layout decisions. Structural removal is not the same as writing zeros into the original full-sized matrix. Record selection criteria, any retraining or correction, and resulting tensor shapes. Successful loading is only a compatibility check; it does not establish that the modified model preserves the original function or remains adequate for the intended task.
Match kernels and quality evaluation
Unstructured sparsity requires a supporting format and kernel to provide a speed benefit. Some devices support only particular sparsity patterns, so zero count alone does not predict runtime. Quantization likewise needs compatible dtype and shape support. Compare quality, subgroup failures, memory, latency, and throughput on matched evaluation data. Separate selection data from final assessment. An unchanged average task score can hide degradation in a small but important condition. Performance and quality must be measured jointly on the execution path actually used, not inferred from a theoretical count of removed values or bits.
Evaluate combined changes
Quantization and pruning can be combined, but ordering, scale recalibration, and structural changes may interact. Tabulate dense float32, ideal packed four-bit, four-nonzero COO, and row-reduced payloads, separating additional metadata. Completion means explaining the assumptions behind sixty-four, eight, forty-eight, and thirty-two bytes and why none alone determines execution speed. Use the quantization lab for numerical reconstruction error and the bandwidth guide for performance constraints. A combined compression pipeline needs its own evaluation; separate successful components do not automatically guarantee their composition’s quality, compatibility, or efficiency.