anthology-of-the-sota

inference-optimization

Graph View

100 items with this tag.

Oct 06, 2026
LIT-800: Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
Oct 03, 2026
LIT-775: Inductive Moment Matching
Oct 03, 2026
LIT-783: Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency
Oct 03, 2026
LIT-784: Mean Flows for One-step Generative Modeling
Oct 03, 2026
LIT-787: One Step Diffusion via Shortcut Models
Oct 03, 2026
LIT-789: Multisample Flow Matching: Straightening Flows with Minibatch Couplings
Oct 03, 2026
LIT-790: Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models
Oct 03, 2026
LIT-791: Improving the Training of Rectified Flows
Oct 03, 2026
LIT-792: Discrete Flow Matching
Oct 03, 2026
LIT-796: InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation
Oct 03, 2026
SOTA-447: When distilling a large image or video model by continuous-time consistency, add a small DMD distribution-matching term
Oct 02, 2026
LIT-770: DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
Oct 01, 2026
LIT-748: SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
Oct 01, 2026
LIT-750: ZipNN: Lossless Compression for AI Models
Oct 01, 2026
THEORY-117: A separator token condenses the content of the segment it closes, rather than only receiving attention a head has nowhere else to put
- attention-techniques
- inference-optimization
Sep 26, 2026
LIT-722: Score-Based Generative Modeling through Stochastic Differential Equations
Sep 25, 2026
LIT-676: DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
Sep 25, 2026
LIT-678: Diffusion Meets Flow Matching: Two Sides of the Same Coin
Sep 25, 2026
LIT-680: Distilling the Knowledge in a Neural Network
Sep 25, 2026
LIT-683: Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
Sep 25, 2026
LIT-687: Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think
Sep 25, 2026
LIT-689: Common Diffusion Noise Schedules and Sample Steps are Flawed
Sep 25, 2026
LIT-701: Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
Sep 25, 2026
LIT-702: Simple and Effective Masked Diffusion Language Models
Sep 25, 2026
SOTA-410: For guided sampling, use a second-order multistep solver on the data-prediction parameterization, not a higher-order solver on the noise prediction
- inference-optimization
- generative-modeling
Sep 25, 2026
SOTA-411: To let a model spend more compute at inference without more parameters, train a depth-recurrent core on a randomly sampled iteration count, re-injecting the input at every iteration
Sep 25, 2026
SOTA-414: To make an image diffusion model a dense geometric predictor, fine-tune it end to end as a one-step model at t = T with a task loss
Sep 25, 2026
SOTA-416: When sampling a diffusion model in few steps, start at t = T: use trailing, not leading, timestep spacing
Sep 25, 2026
SOTA-420: When distilling a classifier, match the teacher's temperature-softened outputs at the same temperature and keep a down-weighted hard-label term
- inference-optimization
- training-optimization
Sep 25, 2026
SOTA-423: When classifier-free guidance over-exposes a latent diffusion model, rescale the guided prediction toward the conditional prediction's standard deviation
- generative-modeling
- inference-optimization
Sep 25, 2026
SOTA-433: Guide a diffusion model with a smaller, less-trained copy of itself rather than an unconditional model, and make sure the two models are degraded in the same way
Sep 25, 2026
THEORY-105: Soft targets transfer the teacher's similarity structure over wrong classes, which carries more information per example than a hard label
- inference-optimization
- training-optimization
Sep 24, 2026
LIT-618: LTX-Video: Realtime Video Latent Diffusion
Sep 24, 2026
LIT-629: Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion
Sep 24, 2026
LIT-631: From Slow Bidirectional to Fast Autoregressive Video Diffusion Models
Sep 24, 2026
LIT-636: Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
Sep 24, 2026
LIT-637: Imagen Video: High Definition Video Generation with Diffusion Models
Sep 24, 2026
LIT-643: One-step Diffusion with Distribution Matching Distillation
Sep 24, 2026
LIT-645: Stochastic Interpolants: A Unifying Framework for Flows and Diffusions
Sep 24, 2026
LIT-646: Improved Distribution Matching Distillation for Fast Image Synthesis
Sep 24, 2026
LIT-650: Competition-Level Code Generation with AlphaCode
Sep 24, 2026
LIT-655: Self-attention Does Not Need O(n²) Memory
Sep 24, 2026
SOTA-392: To get a one-step sampler from a rectified flow, distil it; treat reflow as an optional extra pass, and keep the pre-reflow model for many steps
Sep 24, 2026
SOTA-394: When distilling a causal few-step video generator, take the teacher bidirectional, not causal
Sep 24, 2026
SOTA-397: To condition an unconditionally trained diffusion model on known frames, use reconstruction guidance, not replacement
Sep 23, 2026
LIT-552: XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models
- inference-optimization
- systems-optimization
Sep 23, 2026
LIT-567: MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models
Sep 23, 2026
LIT-585: AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- numerics-and-precision
- inference-optimization
Sep 23, 2026
LIT-586: LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
Sep 23, 2026
SOTA-330: Enforce output structure by masking logits against a grammar, with masks precomputed per automaton state and built on the CPU while the GPU runs the forward pass
- inference-optimization
- systems-optimization
Sep 23, 2026
SOTA-340: Expect weight-only 4-bit quantization to speed up batched serving only while the batch keeps the matmul memory-bound, and use a kernel built to stay there
Sep 23, 2026
SOTA-355: Isolate the emergent outlier feature dimensions into 16-bit and quantize the rest to int8
- numerics-and-precision
- inference-optimization
Sep 23, 2026
SOTA-356: Protect the salient weight channels by scaling them before rounding, choosing them from the activations rather than the weights
- numerics-and-precision
- inference-optimization
Sep 22, 2026
LIT-516: From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications
Sep 22, 2026
LIT-517: Small Singular Values Matter: A Random Matrix Analysis of Transformer Models
- analysis-and-evaluation
- inference-optimization
Sep 22, 2026
LIT-518: The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction
- inference-optimization
- analysis-and-evaluation
Sep 22, 2026
LIT-529: Linformer: Self-Attention with Linear Complexity
- attention-techniques
- inference-optimization
Sep 22, 2026
LIT-545: Gemma 3n model overview
Sep 22, 2026
LIT-546: MatFormer: Nested Transformer for Elastic Inference
Sep 22, 2026
LIT-547: Matryoshka Representation Learning
- representation-and-encoding
- inference-optimization
Sep 22, 2026
SOTA-317: Do not rank singular directions by magnitude when deciding what to discard; check the small end, and check it after fine-tuning
- inference-optimization
- analysis-and-evaluation
Sep 22, 2026
SOTA-318: Choose the rank per matrix when you compress a transformer, because low-rank structure varies by component and by depth
- inference-optimization
- adaptation-and-tuning
Sep 22, 2026
SOTA-327: Train retrieval embeddings with nested losses, so a prefix of the vector can shortlist and the full vector re-rank
- representation-and-encoding
- inference-optimization
Sep 22, 2026
SOTA-328: Train one nested model instead of a family of separately trained sizes, and extract the sizes you need from it
Sep 21, 2026
LIT-468: Self-Consistency Improves Chain of Thought Reasoning in Language Models
- in-context-learning
- inference-optimization
Sep 21, 2026
LIT-479: The Diffusion Duality
Sep 21, 2026
LIT-486: Parallel Scaling Law for Language Models
Sep 21, 2026
LIT-491: Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models
Sep 21, 2026
LIT-498: MaskGIT: Masked Generative Image Transformer
Sep 21, 2026
LIT-511: Image-GS: Content-Adaptive Image Representation via 2D Gaussians
Sep 21, 2026
LIT-512: SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
Sep 21, 2026
SOTA-280: Sample several reasoning paths and take the majority answer rather than decoding one greedily
- in-context-learning
- inference-optimization
Sep 21, 2026
SOTA-289: When the sampling budget is small, prefer uniform-state discrete diffusion with consistency distillation — masked diffusion cannot revise what it has already emitted
Sep 21, 2026
SOTA-296: Add the parallel streams in a short final training stage, not from the start
- training-optimization
- inference-optimization
Sep 21, 2026
SOTA-297: Scale parallel computation with learnable input transforms, not parameters, when inference memory is the binding constraint
Sep 21, 2026
SOTA-302: Steer a distilled generator by modulating its input noise, not by fine-tuning its weights
Sep 21, 2026
SOTA-314: Absorb quantization outliers into a high-precision low-rank branch taken from the weights, and fuse its kernels into the low-bit ones
Sep 21, 2026
THEORY-059: A low-rank branch corrects quantization because weight spectra are steep and quantization-error spectra are flat
- numerics-and-precision
- inference-optimization
Sep 20, 2026
SOTA-269: Keep a knowledge base outside the weights; parameters are an expensive and lossy place to memorise facts
Sep 18, 2026
LIT-428: Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
Sep 16, 2026
LIT-375: Accelerating Large Language Model Decoding with Speculative Sampling
- inference-optimization
Sep 16, 2026
LIT-376: Fast Inference from Transformers via Speculative Decoding
- inference-optimization
Sep 16, 2026
SOTA-227: Decode with a draft model and an accept-reject rule, which is exactly lossless
- inference-optimization
Sep 16, 2026
SOTA-228: Pick the inference partitioning from where the bottleneck is, and expect it to move between prefill and decode
- inference-optimization
Sep 16, 2026
SOTA-229: Scale the draft model's training data, once nothing constrains it to predict the target's features
- inference-optimization
Sep 09, 2026
LIT-224: Orca: A Distributed Serving System for Transformer-Based Generative Models
- inference-optimization
Sep 08, 2026
SOTA-163: Quantize with block-scaled microscaling formats rather than one scale per tensor
- numerics-and-precision
- inference-optimization
Sep 08, 2026
SOTA-185: Quantize weights after training by compensating each rounding error into the columns not yet quantized
- numerics-and-precision
- inference-optimization
Sep 05, 2026
LIT-139: DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Sep 05, 2026
LIT-185: EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
- inference-optimization
Aug 24, 2026
LIT-024: Fast Transformer Decoding: One Write-Head is All You Need
Aug 24, 2026
LIT-081: GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- numerics-and-precision
- inference-optimization
Aug 24, 2026
LIT-103: SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
- inference-optimization
Aug 24, 2026
LIT-110: Efficiently Scaling Transformer Inference
- inference-optimization
Aug 24, 2026
LIT-112: Efficient Memory Management for Large Language Model Serving with PagedAttention
- inference-optimization
- attention-techniques
Aug 24, 2026
SOTA-005: Use running statistics for inference
- inference-optimization
- model-stability
Aug 24, 2026
SOTA-023: Use multi-query attention for decoder-only models to reduce memory bandwidth
- inference-optimization
Aug 24, 2026
SOTA-105: PagedAttention to accelerate batch inference for LLM sampling
- inference-optimization
Aug 24, 2026
SOTA-113: Use continuous batching for inference
- inference-optimization
Aug 24, 2026
SOTA-115: Overlap prefill and decode compute
- inference-optimization

Created with Quartz v5.0.0 © 2026

Repository
Quartz