Topics — The twenty-eight kinds of claim this anthology files against — the primary axis of both the practice registry and the reading list, and the same twenty-eight for each (ADR-026). The filing rule is the point: take a domain topic when the claim is ABOUT the domain; a claim merely discovered in one still takes its kind. The axis organizes the record, it does not bound it: content the list cannot place is evidence the list is short (ADR-059).
Inference optimization — serving-time decisions — batching, cache layout, compression, sparsity, distillation, sampling algorithms.
3 of 129 THEORY documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| THEORY-059 v2 | A low-rank branch corrects quantization because weight spectra are steep and quantization-error spectra are flat | Li et al. (2024), LIT-512 — two propositions bound the output error by the magnitude of weights and activations, not only by their rounding errors. So a rank-r branch helps exactly when the thing it subtracts has a few dominant singular values. A weight matrix does; a quantization error does not, which is why the same trick applied to the error rather than the weights underperforms. | Proposed |
| THEORY-105 | Soft targets transfer the teacher’s similarity structure over wrong classes, which carries more information per example than a hard label | Hinton et al. (2015), LIT-680. The small probabilities a trained model assigns to wrong classes encode how it generalizes: a BMW is rarely mistaken for a garbage truck, but far more often than for a carrot. Raising the temperature exposes that structure, and a student trained on it gets more information and less gradient variance per example. This is the account usually called “dark knowledge”, a phrase that is not in the paper. Its evidence does not rule out the simpler reading, that soft targets are a regularizer. | Proposed |
| THEORY-117 | A separator token condenses the content of the segment it closes, rather than only receiving attention a head has nowhere else to put | Chen et al. (2024), LIT-748 — SepLLM. Attention in Llama-3-8B-Instruct concentrates on punctuation and whitespace, and the paper reads that as each segment’s content being condensed into its closing separator. Its evidence is what masking costs: keeping only separators, a window and three initial tokens holds GSM8K-CoT at 77.18 against 77.79 with 47% of the KV, where StreamingLLM at the same budget gets 70.89 and keeping every fourth token instead gets 72.71. The record already holds the opposite reading — LIT-414 found heads parking no-op attention on [SEP], periods and commas, whose values are small, and THEORY-019 is the sink account it belongs to. No experiment in either paper separates the two. | Proposed |