anthology-of-the-sota

LIT-512: SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

1 min read

Properties5
StatusActive
Tagsnumerics-and-precision, inference-optimization, systems-optimization
Cited as “introduced by” bySOTA-314 — Absorb quantization outliers into a high-precision low-rank branch taken from the weights, and fuse its kernels into the low-bit ones
Cited as “paper” byNOTE-257 — SVDQuant: a low-rank branch on the weights, not on the error
Cited as “source” bySOTA-314 — Absorb quantization outliers into a high-precision low-rank branch taken from the weights, and fuse its kernels into the low-bit ones, THEORY-059 — A low-rank branch corrects quantization because weight spectra are steep and quantization-error spectra are flat

Graph View

Li, Lin, Zhang, Cai, Li, Guo, Xie, Meng, Zhu and Han (2024) — ARXIV-2411.05007, read as NOTE-257.

Standing

Held for a mechanism the record’s quantization cluster does not have. SOTA-185 compensates rounding error into columns not yet quantized; SOTA-163 shrinks the scale’s blast radius with block scaling; SOTA-230 keeps adapters in 16 bits beside a 4-bit base. None of them removes the outliers before quantizing, and none applies a low-rank correction to the weight matrix itself.

Its contribution to the record is two documents. SOTA-314 takes the recommendation; THEORY-059 takes the spectral argument underneath it, which is proved rather than measured and is the part that travels.

The systems half is not optional and the paper says so. A rank-32 branch run naïvely adds 57% latency from 16-bit reads and writes around the projections. The claim only survives with the fusion, which is why the practice states both halves as one instruction.

Backlinks