Topics — The twenty-eight kinds of claim this anthology files against — the primary axis of both the practice registry and the reading list, and the same twenty-eight for each (ADR-026). The filing rule is the point: take a domain topic when the claim is ABOUT the domain; a claim merely discovered in one still takes its kind. The axis organizes the record, it does not bound it: content the list cannot place is evidence the list is short (ADR-059).
Inference optimization — serving-time decisions — batching, cache layout, compression, sparsity, distillation, sampling algorithms.
61 of 737 LIT documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| LIT-024 v2 | Fast Transformer Decoding: One Write-Head is All You Need | Shazeer (2019), ARXIV-1911.02150. Introduces multi-query attention. | Active |
| LIT-081 v3 | GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers | Frantar et al. (2022), ARXIV-2210.17323. One-shot weight quantization to 3-4 bits: round column by column and push each rounding error into the columns not yet done, using approximate second-order information from a small calibration set. | Active |
| LIT-103 v2 | SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills | Agrawal et al. (2023), ARXIV-2308.16369. Prefill saturates the GPU and decode starves it, so split prefills into chunks and fill the rest of each batch with decodes that ride along nearly free. | Active |
| LIT-110 v2 | Efficiently Scaling Transformer Inference | Pope et al. (2022), ARXIV-2211.05102. Inference partitioning derived rather than guessed: an analytical model of communication and memory cost picks the best parallel layout for a given model size, batch and latency target, and the answer changes with the target. | Active |
| LIT-112 v2 | Efficient Memory Management for Large Language Model Serving with PagedAttention | Kwon et al. (2023), ARXIV-2309.06180. Introduces vLLM. | Active |
| LIT-139 v2 | DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence | DeepSeek-AI (2026), ARXIV-2606.19348. Two MoE models — V4-Pro at 1.6T/49B and V4-Flash at 284B/13B — with a 1M-token context reached by a hybrid of compressed sparse attention and heavily compressed dense attention, manifold-constrained hyper-connections widening the residual stream, and Muon as the optimizer for most modules over 32–33T tokens; at 1M tokens V4-Pro needs an estimated 27% of V3.2’s per-token FLOPs and 10% of its KV cache. Multi-token prediction is carried over from V3 unchanged, as a training objective; the report does not mention speculative decoding. | Active |
| LIT-185 v3 | EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test | Li et al. (2025), ARXIV-2503.01840. Drop feature prediction for direct token prediction and fuse multi-layer features, so the draft model finally benefits from more training data: up to 6.5× speedup. | Active |
| LIT-224 | Orca: A Distributed Serving System for Transformer-Based Generative Models | Yu et al. (2022), OSDI ‘22. Introduces iteration-level scheduling — schedule at the granularity of one model iteration rather than one request — and selective batching. 36.9× throughput over FasterTransformer at equal latency on GPT-3 175B. The origin of what the field calls continuous batching. | Active |
| LIT-375 | Accelerating Large Language Model Decoding with Speculative Sampling | Chen et al. (2023), ARXIV-2302.01318. The same algorithm as LIT-376, arrived at independently two months later and demonstrated at a scale the first paper did not reach: Chinchilla, 70B, in a distributed setup, 2-2.5x decoding speedup with the target distribution preserved “within hardware numerics”. Its sharpest observation is that the resulting tokens per second “often exceeds the idealised ceiling on auto-regressive sampling speed imposed by the memory bandwidth” — which is the premise of the whole family, measured. | Active |
| LIT-376 | Fast Inference from Transformers via Speculative Decoding | Leviathan et al. (2022), ARXIV-2211.17192. Decoding is serial and memory-bandwidth-bound, so the arithmetic units are idle. Speculative decoding spends them: a cheap draft model guesses gamma tokens, the target model scores all gamma + 1 positions in one parallel pass, and an accept-reject rule keeps a prefix of the guesses. The output distribution is exactly the target’s — not approximately — and the serial count can never exceed plain autoregressive decoding. 2x-3x on T5-XXL with identical outputs. | Active |
| LIT-428 | Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention | Katharopoulos et al. (2020), ARXIV-2006.16236. Where linear attention comes from, and the identity the record argues about in twelve documents without being able to cite it: drop the softmax for a kernel feature map, associativity reorders the product to O(N), and the autoregressive case is then literally an RNN. | Active |
| LIT-468 | Self-Consistency Improves Chain of Thought Reasoning in Language Models | Wang et al. (2022), ARXIV-2203.11171. Replace greedy decoding with sample-and-marginalize: draw several reasoning paths, take the most common final answer. GSM8K +17.9, SVAMP +11.0, AQuA +12.2. The gain saturates by about five or ten samples. | Active |
| LIT-479 v2 | The Diffusion Duality | Sahoo, Deschenaux, Gokaslan, Wang, Chiu and Kuleshov (2025), ARXIV-2506.10892. Uniform-state discrete diffusion is the argmax of a Gaussian diffusion, which lets Gaussian techniques transfer: a variance-reducing curriculum doubles training speed, and consistency distillation cuts sampling from 1024 steps to 8. Masked diffusion still wins on likelihood; this wins in the few-step regime, where masked models cannot revise what they have already emitted. | Active |
| LIT-486 | Parallel Scaling Law for Language Models | Chen et al. (2025), ARXIV-2505.10475. Run the same weights over P learnably-prefixed copies of the input and aggregate the outputs: the loss falls as if the parameter count had risen by O(log P). Fitted across 0.5B–4.4B and P = 1…8 on two corpora. At batch size 1, matching a parameter-scaled model costs 22× less added memory and 6× less added latency — and P× the training FLOPs, which is the real price. | Active |
| LIT-491 | Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models | Eyring et al. (2025), ARXIV-2508.09968. Reward-guided test-time noise optimization costs 20–40 seconds a sample. Train a LoRA hypernetwork once to predict the optimized noise instead, and recover roughly half the gain at 0.1 s of added latency — 33–300× faster. The comparison that carries it: direct reward fine-tuning of the same generator makes it worse (GenEval 0.73 → 0.62 at four steps), because the KL term that would stop reward-hacking is intractable in data space and tractable in noise space. | Active |
| LIT-498 | MaskGIT: Masked Generative Image Transformer | Chang, Zhang, Jiang, Liu and Freeman (2022), ARXIV-2202.04200. Keeps VQGAN’s tokenizer and replaces its raster-order autoregressive prior with a bidirectional masked transformer decoded in parallel: predict every token, keep the most confident, re-predict the rest. 8 steps instead of 256, up to 64× faster, and better — ImageNet 256×256 FID 6.18 vs 15.78, IS 182.1 vs 78.3. The mask schedule matters and the concave family wins; cosine is best in every experiment. | Active |
| LIT-511 | Image-GS: Content-Adaptive Image Representation via 2D Gaussians | Zhang et al. (2024), ARXIV-2407.01866 — the 2D descendant of LIT-108, and the paper that does the rate accounting LIT-494 skipped. Anisotropic colored 2D Gaussians, spawned by image-gradient magnitude and grown where error persists, decoded at 0.3K MACs per pixel against C3’s 3K at comparable bitrate. Beats six neural image representations at matched model size across the bitrate range, and beats JPEG below 0.244 bpp. Read as NOTE-256. | Active |
| LIT-512 | SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models | Li et al. (2024), ARXIV-2411.05007 — 4-bit weights and activations for diffusion transformers. Shift outliers from activations into the weights by smoothing, peel the dominant singular values into a 16-bit rank-32 branch, quantize only the residual. Two propositions bound the output error by the magnitudes rather than only the rounding errors, which is why shrinking the residual works. Naïvely the extra branch costs 57% latency; fused, the system reaches 3.0× over a W4A16 baseline and 3.5× memory reduction on 12B FLUX.1. Read as NOTE-257. | Active |
| LIT-516 | From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications | Jaiswal et al. (2024), ARXIV-2407.11239 — how nearly low-rank a transformer weight matrix is depends on which matrix and how deep: Query/Key and MLP Gate converge to low rank, MLP Up/Down and Value do not, and middle blocks resist while the first and last few give way. Picking the rank per matrix instead of globally is ~6.4× better in perplexity than uniform reduction at 30% on LLaMA-2 7B and ~47× better at 40% on 13B. Read as NOTE-264. | Active |
| LIT-517 | Small Singular Values Matter: A Random Matrix Analysis of Transformer Models | Staats, Thamm and Rosenow (2024), ARXIV-2410.17770 — the singular spectrum of a trained transformer weight matrix is a Marchenko-Pastur bulk plus outliers, and in non-square matrices the outliers appear at both ends. Removing the smallest decile of the Down-Projection drops Llama-3 8B on GSM8K from 43.2% to 2.0%, while the smallest decile of the square Attention-Output matrix costs almost nothing. Magnitude order is not importance order. Read as NOTE-263. | Active |
| LIT-518 | The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction | Sharma, Ash and Misra (2023), ARXIV-2312.13558 — replacing one weight matrix by a low-rank approximation, especially an MLP matrix in a later layer, can improve accuracy: GPT-J on CounterFact goes from 13.3% to 24.1% off a single layer, with no retraining. The gains land on facts rare in the training corpus. Filed for the debate it anchors rather than read in full. | Active |
| LIT-529 | Linformer: Self-Attention with Linear Complexity | Wang, Li, Khabsa, Fang and Ma (2020), ARXIV-2006.04768 — the measurement the record needed, attached to a method it has already moved past. SVD of the attention matrix in RoBERTa-base and RoBERTa-large over 10k sentences shows a long-tail spectrum at every layer, head and task, skewing further in higher layers; Theorem 1 gives a rank Θ(log n) approximation. Read as NOTE-273. | Active |
| LIT-545 | Gemma 3n model overview | Google DeepMind (2025), model documentation with no accompanying paper. The origin of Per-Layer Embeddings (PLE): each token id looks up a small vector for every layer, and each layer gates it into the residual stream. The table is about 2.35B parameters in E2B and is kept off the accelerator. A design disclosure and not evidence: Google published no ablation. Cited by the Qwen3.8-Flash-Next report (LIT-152) as the precedent for embedding tables held in host memory. | Active |
| LIT-546 | MatFormer: Nested Transformer for Elastic Inference | Devvrit, Kudugunta, Kusupati et al. (2023), ARXIV-2310.07707. Nest four FFN widths inside one Transformer, so each smaller FFN is the first neurons of the larger. Train one width per step, sampled uniformly, and pick a different width per layer at inference to extract sizes nobody trained. Up to 850M parameters it matches four separately trained sizes at the same total compute. The small submodels beat theirs partly because shared weights see up to 4× the tokens. | Active |
| LIT-547 | Matryoshka Representation Learning | Kusupati, Bhatt, Rege et al. (2022), ARXIV-2205.13147. Add the task loss at about log(d) prefix lengths of one embedding (8, 16, …, 2048), and every prefix works as a representation. ResNet50 on ImageNet: each prefix is at least as accurate as a separately trained model of that width. Shortlisting with 16 dimensions and re-ranking with 2048 matches full-width retrieval at 128× fewer FLOPs, 14× faster in wall-clock time. At full width it costs a few tenths of a point on the web-scale models. | Active |
| LIT-552 | XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models | Dong et al. (2024), ARXIV-2411.15100. Constrained decoding sets the logits of grammar-violating tokens to −∞. Doing that for a context-free grammar means interpreting a pushdown automaton over the whole vocabulary every step. XGrammar precomputes, per automaton node, the verdict for tokens that do not depend on the rest of the stack (over 99% for JSON on Llama-3.1), and builds the mask on the CPU during the GPU forward pass. Per-token mask time is 36 µs against milliseconds for earlier engines, with end-to-end overhead of about 0.1 ms per token. | Active |
| LIT-567 | MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models | Frantar, Castro, Chen, Hoefler, Alistarh (2024), ARXIV-2408.11743. An FP16×INT4 matrix-multiply kernel, the “Marlin format” being its quantization-specific weight layout. It keeps weight-only quantization’s memory-bandwidth speed-up through batched serving: about 3.9× over FP16 (near the 3.87× ideal for 4-bit, group 128) up to batch 16–32, falling toward 1.5× at 128 as the problem turns compute-bound. In vLLM, about 2.8× lower time per output token on an A6000, and 2.3–3.2× single-GPU end-to-end at batch 16 or below, falling to 1.1–1.2× at 128. | Active |
| LIT-585 | AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration | Lin et al. (2023), ARXIV-2306.00978. Weight-only low-bit quantization that protects the ~1% of channels whose activations are largest, by scaling them up before rounding rather than keeping them in higher precision. No backpropagation and no reconstruction, so it does not overfit its calibration set. | Active |
| LIT-586 v2 | LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale | Dettmers et al. (2022), ARXIV-2208.07339. Int8 inference for the feed-forward and attention projections, at no measured cost in quality up to 175B, by isolating a small set of systematically emergent outlier feature dimensions into a 16-bit multiplication and quantizing the other 99.9% vector-wise. | Active |
| LIT-618 v2 | LTX-Video: Realtime Video Latent Diffusion | HaCohen et al., Lightricks (2024), ARXIV-2501.00103. It trades latent size for speed: a 32×32×8 VAE with 128 channels (1:192) moves patchification into the autoencoder, and the decoder does the last denoising step. It generates 5s of 768×512 video in about 2s on an H100 at 20 steps. The denoising decoder, the central idea, is supported by one internal preference study with no reported numbers. | Active |
| LIT-629 v3 | Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion | Huang et al., Adobe Research and UT Austin (2025), ARXIV-2506.08009. Self Forcing trains an autoregressive video model on its own rollouts, conditioning on self-generated context and scoring the whole clip with a distribution-level loss. On Wan2.1-1.3B it beats teacher-forced and diffusion-forced training under every objective tried, and holds quality where they degrade frame by frame. It reaches 17 fps at 0.69s latency on one H100. The comparison against slower models is confounded by a 14B teacher. | Active |
| LIT-631 v3 | From Slow Bidirectional to Fast Autoregressive Video Diffusion Models | Yin et al., MIT and Adobe (2024), ARXIV-2412.07772. CausVid distills a bidirectional video DiT into a 4-step block-causal student with distribution matching, cutting first-frame latency from 219s to 1.3s at 9.4 fps. The controlled result is that a bidirectional teacher makes a better causal student than a causal one. The authors attribute this to error accumulation, but the causal teacher is also the weaker model, and the ablation does not separate the two. The paper does not claim real time, and it reports that the student flickers more and is less diverse. | Active |
| LIT-636 v4 | Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow | Liu, Gong and Liu, UT Austin (2022), ARXIV-2209.03003. It names rectified flow: regress a velocity onto x1 − x0 along the straight interpolation, which beats VP and sub-VP probability-flow ODEs on CIFAR-10 at the same architecture (FID 2.58 against 3.93). The “fast” in the title needs reflow or distillation. Without reflow or distillation one-step FID is 378. Distillation alone gives 6.18. One reflow gives 12.21 undistilled and 4.85 distilled, and full-simulation quality gets worse. | Active |
| LIT-637 v2 | Imagen Video: High Definition Video Generation with Diffusion Models | Ho et al., Google (2022), ARXIV-2210.02303. VDM scaled into a seven-model, 11.6B cascade producing 128 frames at 1280×768. It is the first report to add a separate image-text corpus (60M internal pairs plus LAION-400M) to joint training, and it asserts the benefit without an ablation. Its tested results are that v-prediction beats ε-prediction for video super-resolution, and that distillation to 8 steps is 18× faster at a small metric cost under constant guidance and none under oscillating guidance. | Active |
| LIT-643 | One-step Diffusion with Distribution Matching Distillation | Yin et al., MIT and Adobe (2023), ARXIV-2311.18828. DMD distills a diffusion model into a one-step generator by descending an approximate KL(p_fake ‖ p_real), whose gradient is the difference of a frozen “real” score and an online “fake” score. It adds an LPIPS regression loss onto precomputed teacher ODE outputs. It reaches 2.62 FID on ImageNet-64 and 11.49 on zero-shot COCO-30k. The ablation that makes the regression loss look necessary ran its no-regression arm at a different learning rate, and its mode-collapse evidence is qualitative. | Active |
| LIT-645 | Stochastic Interpolants: A Unifying Framework for Flows and Diffusions | Albergo, Boffi and Vanden-Eijnden (2023), ARXIV-2303.08797. The earlier interpolant objective is extended with a latent Gaussian term. One learned velocity and score then give an ODE and a family of SDEs, all sharing the interpolant’s marginals, and the SDE’s diffusion coefficient can be chosen after training. For SDEs the regression losses bound the KL, and for the ODE they do not. This is the framework SiT instantiates. The evidence that SDE sampling beats ODE sampling is a 2-D checkerboard and a 128-D Gaussian mixture. The paper reports no image metric and no diffusion baseline. | Active |
| LIT-646 | Improved Distribution Matching Distillation for Fast Image Synthesis | Yin et al., MIT and Adobe (2024), ARXIV-2405.14867. DMD2 drops DMD’s regression loss. It stabilizes training by updating the fake score five times per generator step, adds a GAN loss on real images, and trains multi-step students on their own simulated inputs. The ImageNet ablation is clean: removing the loss costs 2.62 → 3.48 FID, and the update rule restores 2.61. The student is slightly less diverse than its teacher (LPIPS diversity 0.61 against 0.64). The paper does not attribute this to the reverse KL. | Active |
| LIT-650 | Competition-Level Code Generation with AlphaCode | Li et al. (2022), ARXIV-2203.07814. Top 54.3% on ten Codeforces contests, by sampling up to a million programs per problem and filtering 99% of them away on the example tests. Two things the record needs are incidental to that: HumanEval’s false-positive rate, hand-measured at 30%, and the fact that pass@k assumes every sample can be submitted and is therefore an upper bound on what a system with a submission budget achieves. | Active |
| LIT-655 | Self-attention Does Not Need O(n²) Memory | Rabe and Staats (2021), ARXIV-2112.05682. Six months before FlashAttention: attention in O(1) memory per query, self-attention in O(log n), a practical O(√n) implementation, exact rather than approximate, 59× less memory overhead at sequence length 16,384. And the number the record needed and did not have: on TPU the same algorithm is “within a few percent of the runtime of the standard implementation” — the memory saving is algorithmic and the speedup is not. | Active |
| LIT-676 v4 | DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models | Lu, Zhou, Bao, Chen, Li and Zhu (2022), ARXIV-2211.01095. The regime SOTA-203 does not mention. At a classifier guidance scale of 8.0 on ImageNet 256×256, FID at 10 function evaluations is 13.04 for first-order DDIM, 114.62 for DPM-Solver-2 and 164.74 for DPM-Solver-3 — higher order is monotonically catastrophic, and the practice’s own framing of DDIM as “the least accurate member of the family” inverts. Two causes, one of which the record already holds as SOTA-202. The fix is order 2, multistep, on the data-prediction parameterization. | Active |
| LIT-678 v2 | Diffusion Meets Flow Matching: Two Sides of the Same Coin | Gao, Hoogeboom, Heek, De Bortoli, Murphy and Salimans (2024), a blog post at diffusionflow.github.io. With a Gaussian source, flow matching is a diffusion model with the schedule α_t = 1−t, σ_t = t, and the flow-matching Euler sampler is DDIM. What separates the named recipes is three choices: the loss weighting, the network output and the sampling schedule. “Straight paths” holds only for a single-point target, and a variance-preserving schedule can be straighter for wide data. The content is derivation and toy illustration only: no trained models and no benchmarks. | Active |
| LIT-680 | Distilling the Knowledge in a Neural Network | Hinton, Vinyals and Dean (NIPS 2014 Deep Learning Workshop), ARXIV-1503.02531. This is where the word “distillation” in 112 files of this record comes from. It trains a student on the teacher’s softmax at a raised temperature, adds a down-weighted hard-label term, and scales the soft term by T². On speech, a 10-model ensemble distils into one model of the same size as each member, keeping 86% of the frame-accuracy gain and all of the 0.2-point WER gain. Three things usually quoted from it are not in it: the phrase “dark knowledge”, a high temperature on the realistic task (the best was T = 2), and an ablation of T². | Active |
| LIT-683 | Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach | Geiping et al. (NeurIPS 2025), ARXIV-2502.05171. Huginn has a 2-layer prelude, a 4-layer core iterated r times with the input re-injected every iteration, and a 2-layer coda: 3.5B parameters trained on 800B tokens with r sampled per step (mean 32) and backpropagation through only the last 8 iterations. Accuracy rises with test-time iterations and saturates per task. Recurrence is shown to buy accuracy per parameter, not per FLOP. The only controlled baseline has the same parameters and about 1/16 of the compute. “Equivalent to 50B parameters” is a FLOP count, not a comparison. | Active |
| LIT-687 v2 | Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think | Martin Garcia, Knaebel, Schmidt, de Geus, Hermans and Leibe (WACV 2025), ARXIV-2409.11355. Marigold’s “dismal” few-step depth came from the DDIM scheduler’s leading timestep spacing. The spacing hands a pure-noise input a timestep that claims an almost clean one. Switching to trailing spacing makes one step usable (NYUv2 AbsRel 5.7). The fixed single step is still behind the 50-step, 10-member ensemble on all five depth sets. What beats that ensemble everywhere is end-to-end fine-tuning as a one-step model with a task loss, and it works nearly as well from plain Stable Diffusion. | Active |
| LIT-689 | Common Diffusion Noise Schedules and Sample Steps are Flawed | Lin, Liu, Li and Yang (WACV 2024), ARXIV-2305.08891. Common schedules leave signal at the last timestep. Stable Diffusion’s x_T is 0.068·x₀ + 0.998·ε. The model learns to keep the leaked per-channel mean, and pure-noise inference then pins samples to “medium brightness”. Many samplers also never start at t = T. It proposes four fixes: zero terminal SNR, v-prediction, sampling from the last timestep, and a guidance rescale. The only numeric result applies all four at once (COCO FID 22.96 → 21.66). Every single-fix ablation is qualitative, on one to three prompts. It credits the trailing discretization to DPM-Solver, and it never calls the spacing fix “slight”. | Active |
| LIT-701 | Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution | Lou, Meng and Ermon (2023), ARXIV-2310.16834. SEDD: learn the ratios p_t(y)/p_t(x) of a discrete diffusion with score entropy, a Bregman loss that keeps them positive and yields an ELBO. The first discrete diffusion model to reach GPT-2-level zero-shot perplexity, and the absorbing (mask) process beats the uniform one on every table. The GPT-2 comparison is against the released model, trained on different data and re-evaluated without a sliding window. | Active |
| LIT-702 | Simple and Effective Masked Diffusion Language Models | Sahoo et al. (2024), ARXIV-2406.07524. MDLM: the masked-diffusion training objective reduces to a weighted average of BERT-style masked-LM losses, a continuous-time ELBO whose value is invariant to the noise schedule. It is the strongest discrete diffusion language model at its scale — beating SEDD at matched tokens — and still trails a retrained autoregressive baseline in-domain (LM1B 27.04 vs 22.32; OWT 23.21 vs 17.54), which the authors state as the limitation. | Active |
| LIT-722 | Score-Based Generative Modeling through Stochastic Differential Equations | Song, Sohl-Dickstein, Kingma, Kumar, Ermon and Poole (2020), ARXIV-2011.13456. The framework this record has been standing on without holding. Classifier guidance is Eq. 14 here, six months before ADM: the conditional reverse-time SDE adds ∇ₓ log pₜ(y|x) to the unconditional score, and §5 gives the recipe for the noise-conditioned classifier that supplies it — sample (x(0), y), noise it through the tractable forward SDE, train with a mixture of cross-entropy losses over timesteps. Also the probability flow ODE with exact likelihoods, predictor-corrector sampling, CIFAR-10 IS 9.89 / FID 2.20 / 2.99 bits/dim, and the first 1024×1024 samples from a score model. Its §I.2 imputation approximation is the “replacement” SOTA-397 exists to reject. | Active |
| LIT-748 | SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator | Chen et al. (2024), ARXIV-2412.12094 — SepLLM. Let each token attend only to a few initial tokens, the previous n tokens, and every earlier separator token (punctuation and whitespace), on the hypothesis that a segment’s content is condensed into the separator that ends it. Training-free on Llama-3-8B-Instruct it keeps GSM8K-CoT at 77.18 against 77.79 with 47% of the KV cache, where StreamingLLM at the same budget gets 70.89. Trained that way from scratch at 160M it reaches lower loss than full attention at equal FLOPs. | Active |
| LIT-750 | ZipNN: Lossless Compression for AI Models | Hershcovitch et al. (2024), ARXIV-2411.05239. Losslessly compressing a trained model’s file works almost entirely through the float exponent: about 40 of 256 values occur, and the exponent stream codes to ~33% while sign and mantissa stay near 100%. Regular BF16 models therefore shrink to ~66%, FP32 to ~83%, and models rounded after training (“clean”) to as little as 34–48%. Splitting the exponent into its own stream and coding it with Huffman alone, without LZ, beats Zstd by 17% in size and 62% in single-thread speed on Llama-3.1 BF16. | Active |
| LIT-770 v2 | DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation | Yu et al., Texas A&M and ByteDance (2026), ARXIV-2610.02188. DMAD replaces DMD’s fake-score critic with a discriminator and trains the student on a linear loss on its logit. At the unrestricted discriminator optimum that gradient equals DMD’s idealized one (App. A, checked); nothing bounds it off the optimum, where training runs with one discriminator step per student step. One-step ImageNet-64 FID is 1.24 against DMD2’s 1.28, at 570K iterations, and 1.04 with an ImageNet-pretrained discriminator scored only by FID. Four-step SDXL reaches 14.47 FID at lower CLIP. Wan2.1 VBench margins are 0.1–0.3 points. No diversity metric and no seeds are reported. | Active |
| LIT-775 | Inductive Moment Matching | Zhou, Ermon and Song, Luma AI and Stanford (2025), ARXIV-2503.07565. A one-network, from-scratch few-step model that maps the interpolant’s marginal at t to the one at s < t. It is trained by matching, with MMD over groups of samples, the distributions reached from t and from a nearby r. Consistency models are its one-sample, first-moment case, and that case collapses on ImageNet. ImageNet-256 FID is 8.05 at one step and 1.99 at eight, with guidance at sampling time, so 16 evaluations. Training is 1.2M steps at batch 4,096. CIFAR-10 2-step 1.98. One run per row. | Active |
| LIT-783 | Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency | Zheng et al., Tsinghua and NVIDIA (2025), ARXIV-2510.08431. sCM made to run on 14B text-to-image and text-to-video models, with a FlashAttention-2 JVP kernel under FSDP and context parallelism. Pure sCM loses fine detail and temporal coherence there. Adding a DMD loss at weight 0.01 (rCM) fixes it. Four-step Wan2.1 VBench is 84.43 at 1.3B against a re-implemented DMD2’s 84.56, and 84.92 at 14B; GenEval tracks the teacher at 1–4 steps. The diversity advantage over DMD2 is shown in samples only. One run per row. | Active |
| LIT-784 | Mean Flows for One-step Generative Modeling | Geng, Deng, Bai, Kolter and He, CMU and MIT (2025), ARXIV-2505.13447. Train a network for the average velocity u(z, r, t) over an interval, using the identity u = v − (t − r)·du/dt with du/dt from a Jacobian-vector product and the target under stop-gradient. With r = t it is flow matching. From scratch on ImageNet-256 with DiT-XL/2 and guidance folded into the target, one-step FID is 3.43 at 240 epochs and two-step 2.20 at 1,000. The best one-step baselines it beats, Shortcut 10.60 and IMM 7.77, are copied rows at other budgets. Behind iCT on CIFAR-10. One run per row. | Active |
| LIT-787 | One Step Diffusion via Shortcut Models | Frans, Hafner, Levine and Abbeel, UC Berkeley (2024), ARXIV-2410.12557. A flow-matching network also conditioned on step size d, trained in one run: the flow-matching loss at d = 0, and a target for step 2d built from two of its own d-steps. On DiT-B at matched compute, one model gives 6.9 / 13.8 / 20.5 FID at 128 / 4 / 1 steps on CelebA-HQ-256 and 15.5 / 28.3 / 40.3 on ImageNet-256. Progressive distillation is better at one step (14.8 and 35.6) and cannot sample at many. The re-run consistency baselines are far below their published numbers. DiT-XL reaches 10.6 in one step. One run per cell. | Active |
| LIT-789 | Multisample Flow Matching: Straightening Flows with Minibatch Couplings | Pooladian et al., Meta AI, NYU and Weizmann (2023), ARXIV-2304.14772. Flow matching with noise and data paired within each minibatch by OT (BatchOT) or a cheaper stable matching keeps the marginals exact. As batch size grows the paths straighten toward the OT map. On ImageNet-64 the FID-20 point drops from 29 Euler steps to 12, and adaptive FID from 13.93 to 12.37, at equal likelihood and 4% more time per iteration. Low-NFE numbers stay far from one-step quality: 38.86 FID at 4 Euler steps on ImageNet-32. One run per row. | Active |
| LIT-790 | Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models | Lu and Song, OpenAI (2024), ARXIV-2410.11081. Continuous-time consistency models had been unstable. A trigonometric flow parameterization, identity time input with positional embeddings, normalized tangents, learned loss weights and a Flash Attention JVP make them train to 1.5B parameters. Continuous time beats every discrete step count tried. Two-step FID is 2.06 on CIFAR-10, 1.48 on ImageNet-64 and 1.88 on ImageNet-512, against teachers at 2.01, 1.33 and 1.73. Every model, “consistency training” included, starts from the pretrained diffusion model. | Active |
| LIT-791 | Improving the Training of Rectified Flows | Lee, Lin and Fanti, CMU and Microsoft Research (2024), ARXIV-2405.20320. One reflow round is enough, argued informally, and the 2-rectified flow then trains better with a U-shaped timestep distribution and an LPIPS-Huber loss. Started from EDM, with no separate distillation stage, one-step FID is 3.07 on CIFAR-10 and 4.31 on ImageNet-64. Its own table lists DMD at 2.62 on ImageNet-64. There is no distillation-alone arm from the same teacher, and many-step quality is not compared with the pre-reflow model. | Active |
| LIT-792 | Discrete Flow Matching | Gat et al., FAIR and Weizmann (2024), ARXIV-2407.15595. Flow matching on discrete tokens as a continuous-time Markov chain, with velocities of the same form as the continuous case, any coupling, a family of schedulers, and corrector sampling that can re-mask emitted tokens. At 150M its generative perplexity beats SEDD’s in float32 (GPT-2 29.3 against 40.5 at 1,024 NFE). In double precision every discrete model’s perplexity rises 2–3.5 times and the gap narrows (97.5 against 109.5). At 1.7B on code it reaches HumanEval pass@1 6.7 against a same-data autoregressive model’s 14.3. | Active |
| LIT-796 | InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation | Liu, Zhang, Ma, Peng and Liu, UT Austin and Helixon (2023), ARXIV-2309.06380. Reflow Stable Diffusion once, then distil the reflowed model to one step with an LPIPS regression. Against distilling SD 1.4 directly, at the same 100K steps and 3.2M generated pairs, reflow first gives one-step FID 31.0 against 40.9 on COCO-2017-5k and 20.0 against 34.6 on COCO-2014-30k. The scaled model, InstaFlow-0.9B, scores 23.4 and 13.10. The teacher is a diffusion model, not a rectified flow, and the distiller is a regression, not distribution matching. One run per row. | Active |
| LIT-800 | Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM | Kozyrev and Maiboroda, Minima (2026), ARXIV-2609.04098. Public 4-bit builds of Qwen3.8-27B kept its 48 Gated DeltaNet layers in 8 or 16 bits for fear that errors in a recurrence compound. Quantizing all 496 linear layers to NVFP4 W4A4 by calibration alone stays within seed noise of BF16 on five task suites (average −0.52) at 17.5 GiB. The perplexity cost is the largest of the recipes compared, +0.72 at 4K, but it shrinks along a 32K window. In replayed layers the gate projections everyone protected are the least sensitive. State error holds at a plateau of about 12.6% over 32K tokens, and a state impulse is erased within hundreds to a few thousand steps. One model, one format. | Active |