Topics — The twenty-eight kinds of claim this anthology files against — the primary axis of both the practice registry and the reading list, and the same twenty-eight for each (ADR-026). The filing rule is the point: take a domain topic when the claim is ABOUT the domain; a claim merely discovered in one still takes its kind. The axis organizes the record, it does not bound it: content the list cannot place is evidence the list is short (ADR-059).
Adaptation and tuning — taking a trained model somewhere new — fine-tuning and transfer, preference training and safety alignment, parameter-efficient adaptation, context extension.
88 of 737 LIT documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| LIT-034 v2 | Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks | Gururangan et al. (2020), ARXIV-2004.10964. Benefits of domain-adaptive pretraining. | Active |
| LIT-035 v2 | Language Models are Few-Shot Learners | Brown et al. (2020), ARXIV-2005.14165. Introduces GPT-3. | Active |
| LIT-046 v2 | LoRA: Low-Rank Adaptation of Large Language Models | Hu et al. (2021), ARXIV-2106.09685. Freeze the pretrained weights and learn a rank-r product beside each one: the update a fine-tune applies has low intrinsic rank, so representing it in full is wasted memory. | Active |
| LIT-049 v3 | Learning to Prompt for Vision-Language Models | Zhou et al. (2021), ARXIV-2109.01134. Learnable prompt optimization. | Active |
| LIT-055 v3 | P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks | Liu et al. (2021), ARXIV-2110.07602. Prefix tuning applied at every layer rather than only at the input, which is what makes prompt tuning match full fine-tuning at ordinary model sizes instead of only at 10B+. | Active |
| LIT-072 v2 | Simple Open-Vocabulary Object Detection with Vision Transformers | Minderer et al. (2022), ARXIV-2205.06230. Open vocabulary detection. | Active |
| LIT-078 v3 | An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion | Gal et al. (2022), ARXIV-2208.01618. Efficient personalization method. | Active |
| LIT-079 v2 | DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation | Ruiz et al. (2022), ARXIV-2208.12242. Subject-driven image generation. | Active |
| LIT-082 v3 | Constitutional AI: Harmlessness from AI Feedback | Bai et al. (2022), ARXIV-2212.08073. Train harmlessness without human harm labels: the model critiques and revises its own responses against a written list of principles, and the preference model learns from those comparisons. | Active |
| LIT-089 v3 | Adding Conditional Control to Text-to-Image Diffusion Models | Zhang et al. (2023), ARXIV-2302.05543. Modular conditioning framework. | Active |
| LIT-091 v3 | T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models | Mou et al. (2023), ARXIV-2302.08453. Controllable generation. | Active |
| LIT-104 v4 | ReLoRA: High-Rank Training Through Low-Rank Updates | Lialin et al. (2023), ARXIV-2307.05695. Restart a LoRA adapter repeatedly during pretraining so low-rank updates sum to a high-rank one. The restart needs an optimizer reset and a re-warm together (SOTA-426); the paper’s own ablation shows the full-rank warm start carries most of the gain. | Active |
| LIT-118 v3 | PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding | Li et al. (2023), ARXIV-2312.04461. ID-preserving generation. | Active |
| LIT-119 | Falcon-H1-Tiny: A series of extremely small, yet powerful language models redefining capabilities at small scale | Falcon-LLM Team (2026), technical blogpost. Holds the parameter budget at 90M and pulls the other three levers — data strategy, architecture, optimizer — one at a time: pretraining from scratch on the target SFT or reasoning mix beats pretrain-then-finetune at this scale, Muon with learnable multipliers beats the AdamW recipe, and depth and SSM width beat MLP width. | Active |
| LIT-127 v2 | DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models | Shao et al. (2024), ARXIV-2402.03300. A 7B model continued-pretrained on 120B math tokens mined from Common Crawl reaches 51.7% on MATH without tools or voting, and introduces Group Relative Policy Optimization — PPO without a critic, with the baseline taken from a group of sampled outputs — the RL algorithm most later reasoning recipes use. | Active |
| LIT-128 | Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling | Falcon-LLM Team (2026), ARXIV-2601.02346. Falcon-H1-7B taken to a reasoning model by SFT and RL scaling; it matches or beats reasoning models 2× to 7× larger, and its hybrid architecture makes the parallel test-time scaling it relies on (DeepConf) cheap in inference. Its reasoning corpus is the one LIT-119’s tiny reasoning models train on. | Active |
| LIT-129 | Can Tiny Language Models Reason? | Jakimovski (2025), a write-up with code, no paper. SmolLM2-135M-Instruct taken through SFT on plain dialogue, SFT on reasoning traces in think tags, and one epoch of DPO on 50k preference pairs, on a single GPU; the DPO stage converges and the final model gains 3 to 9 points over its base on ARC-C, BBH, MMLU and IFEval. | Active |
| LIT-130 | Olmo 3 | Olmo Team (2025), ARXIV-2512.13961. Fully open 7B and 32B models released as a complete model flow — every stage, checkpoint and dataset — with a Think track built by the standard curriculum (pretrain, SFT on reasoning traces, preference tuning, RL with verifiable rewards) and an RL-Zero track that runs RLVR directly on the base model as an open benchmark for studying RL. | Active |
| LIT-164 | DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning | DeepSeek-AI (2025), ARXIV-2501.12948. Reasoning incentivised by pure RL on verifiable tasks, with no human-labelled reasoning traces: self- reflection, verification and strategy adaptation emerge rather than being demonstrated. | Active |
| LIT-167 | Understanding R1-Zero-Like Training: A Critical Perspective | Liu et al. (2025), ARXIV-2503.20783. Two corrections to the R1-Zero story: the base model may already carry what looks like emergent reasoning, and GRPO has a length bias that inflates wrong answers. Dr. GRPO removes the bias. | Active |
| LIT-168 | DAPO: An Open-Source LLM Reinforcement Learning System at Scale | Yu et al. (2025), ARXIV-2503.14476. Decoupled Clip and Dynamic Sampling Policy Optimization, published with the four techniques, the code and the dataset — 50 points on AIME 2024 from a Qwen2.5-32B base. | Active |
| LIT-169 | Direct Preference Optimization: Your Language Model is Secretly a Reward Model | Rafailov et al. (2023), ARXIV-2305.18290. Reparameterise the reward so the optimal policy has a closed form, and RLHF collapses into a single classification loss — no reward model, no sampling during fine-tuning, no RL loop. | Active |
| LIT-172 | Tulu 3: Pushing Frontiers in Open Language Model Post-Training | Lambert et al. (2024), ARXIV-2411.15124. The post-training recipe published in full — data, code, decontamination and the negative results — and the origin of Reinforcement Learning with Verifiable Rewards as a named stage. | Active |
| LIT-180 v2 | Group Sequence Policy Optimization | Zheng et al. (2025), ARXIV-2507.18071. Define the importance ratio on sequence likelihood rather than per token, and clip, reward and optimise at sequence level — which stabilises MoE reinforcement learning in particular. | Active |
| LIT-211 | Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning | Qiu et al. (2025), ARXIV-2509.24372. Evolution strategies applied to the full parameter space of an LLM, at billion scale, without dimensionality reduction — which the field had assumed impossible. Against PPO and GRPO on Countdown across Qwen2.5 (0.5B–7B) and LLaMA3 (1B–8B), ES improves over the base model by 36.4% on average against 21.3% for GRPO and 17.9% for PPO, and does it with one fixed hyperparameter set while RL got a sweep per experiment. | Active |
| LIT-230 v3 | Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO | Ba et al. (2026), ARXIV-2608.27351. The first study of what ES post-training DOES rather than whether it wins. Three findings: ES raises pass@1 and pass@k together while GRPO raises pass@1 and falls below the base model on pass@16 and pass@32 in 15 of 18 comparisons; ES’s 40x larger parameter drift is functionally sparse, concentrated in LayerNorm and attention projections, and does not produce broad forgetting; and the population size needed for stable ES falls as the model grows. | Active |
| LIT-231 v2 | Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging | Gu et al. (2026), ARXIV-2608.05541. Argues that full-parameter ES on billion-parameter LLMs is ineffective because almost all random perturbations are near-orthogonal to a useful descent direction, and replaces the search space rather than the optimizer: a handful of cheap GRPO runs supply LoRA descent directions, and CMA-ES then searches layer-wise DARE-TIES merging coefficients over their span. Beats GRPO-LoRA by about 1% with 10% fewer gradient updates — on models at 1.5B and below. | Active |
| LIT-233 v3 | Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights | Gan and Isola (2026), ARXIV-2603.12228. Measures the neighbourhood of pretrained weights and finds it dense with task-improving Gaussian perturbations, with the density rising monotonically with model scale — 0% of perturbations match base accuracy on GSM8K at 0.5B against 64% at 32B — and finds those perturbations to be specialists rather than generalists. The method built on it, RandOpt, post-trains in one parallel step and is competitive with PPO, GRPO and ES at equal training FLOPs, at the cost of K forward passes at inference. | Active |
| LIT-234 v2 | Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies | Hayes et al. (2026), ARXIV-2608.12679. Takes evolution strategies up to 32B and asks one question: what does post-training do to the breadth of the output distribution? RL raises pass@1 and lowers pass@k, eventually below the base model; ES raises both, and never falls below base at any k or scale tested. The mechanism is visible in the accuracy histogram — RL increases the number of prompts solved in none of k samples, putting a hard ceiling on pass@k that no extra sampling can lift. | Active |
| LIT-235 v2 | Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training | Hoy et al. (2026), ARXIV-2604.01499. Resolves the drift dispute by decomposing the ES update: an on-manifold component that changes the loss, and an off-manifold component that is loss-invariant and therefore performs a random walk whose squared norm grows as sigma^2 d T / N. In a landscape with many flat directions the second dominates — which is why ES moves two orders of magnitude further than GRPO, why the two update directions are nearly orthogonal, and why the two solutions are nevertheless linearly connected with no loss barrier. | Active |
| LIT-236 v2 | The Blessing of Dimensionality in LLM Fine-tuning: A Variance-Curvature Perspective | Liang et al. (2026), ARXIV-2602.00170. One geometric property accounts for two puzzles at once: fine-tuning landscapes are low-dimensional in curvature, so a handful of stiff directions carry the improvement while a near-zero bulk absorbs variance. That gives rise-then-decay dynamics under fixed hyperparameters — in GRPO as well as ES — and it gives degeneracy, where many ambient perturbations share the same projection onto the stiff subspace, so a population of about 30 keeps finding improvements from 0.5B to 7B with no rightward shift. | Active |
| LIT-237 v2 | Evolutionary Strategies lead to Catastrophic Forgetting in LLMs | Abdi et al. (2026), ARXIV-2601.20861. The first independent attempt to reproduce ES-beats-GRPO, and it does not: ES lands close but GRPO stays ahead on every task but one, at 1B and 1.5B. Its larger contribution is the forgetting curve — held-out HellaSwag falls steadily as Countdown training continues past the point where Countdown itself has converged, tracing a convex Pareto front, with ES parameter drift roughly 1000x GRPO’s and ES updates dense where GRPO’s are about 95% sparse. | Active |
| LIT-238 v2 | Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies | Schweighofer et al. (2026), ARXIV-2605.30148. Answers the forgetting result with three findings: the degradation is transient drift rather than irreversible forgetting — HellaSwag falls 8% over 300 iterations and returns to baseline by the end — it is not specific to ES, since GRPO forgets too on ProofWriter, and it is driven by the random walk Hoy et al. predicted, whose size falls with population as their equation says. Anchored Weight Decay, an L1 or L2 pull toward the initial weights applied in the update rule, buys a population-128 reduction in drift at population 30 for 1-2% runtime. | Active |
| LIT-239 v2 | ESSA: Evolutionary Strategies for Scalable Alignment | Korotyshova et al. (2025), ARXIV-2507.04453. The earliest paper in this record’s ES line, and it takes the opposite route to LIT-211: shrink the search space rather than search the full one. SFT-train LoRA adapters, take their SVD, and let CMA-ES optimize only the top singular values. On GSM8K with Qwen2.5-Math-7B it passes 0.80 accuracy in under 25 minutes against GRPO’s 50+, on 100 training examples against GRPO’s 5,978 — and it aligns a 32B model at INT4 and INT8, where backpropagation cannot run at all. | Active |
| LIT-240 v2 | ESSAM: A Novel Competitive Evolution Strategies Approach to Reinforcement Learning for Memory Efficient LLMs Fine-Tuning | Sun et al. (2026), ARXIV-2602.01003. Adds a sharpness-aware step to ES: move the parameters against the reward-weighted direction first, recompute the update at that neighbouring point, and apply it at the original — steering the solution toward flat regions. On GSM8K across Qwen2.5 at 0.5B-7B it averages 78.27% against standard ES’s 75.97%, PPO’s 77.72% and GRPO’s 78.34%, at inference-level GPU memory. | Active |
| LIT-377 v3 | Training language models to follow instructions with human feedback | Ouyang et al. (2022), ARXIV-2203.02155. InstructGPT: the paper that turned reinforcement learning from human feedback from a summarization technique into the default shape of post-training. Three steps — supervised fine-tuning on labeler demonstrations, a reward model trained on ranked outputs, then PPO against that reward with a per-token KL penalty. A 1.3B model tuned this way is preferred to 175B GPT-3, which is the result the field acted on. | Active |
| LIT-378 | QLoRA: Efficient Finetuning of Quantized LLMs | Dettmers et al. (2023), ARXIV-2305.14314. Back-propagate through a frozen 4-bit base model into 16-bit LoRA adapters: 65B fine-tuning drops from >780GB to <48GB with no loss against a 16-bit fully fine-tuned baseline. Carries a second finding the quantization headline tends to bury — LoRA on the query and value projections alone does not reach full fine-tuning at scale, and the adapter COUNT, not the rank, is what closes the gap. | Active |
| LIT-379 v2 | The Surprising Effectiveness of Test-Time Training for Few-Shot Learning | Akyürek et al. (2024), ARXIV-2411.07279. Temporarily update the model’s parameters at inference, using a loss built from the test instance’s own in-context examples. On ARC-AGI this is up to 6x a fine-tuned baseline — 53.0% with an 8B model, 61.9% ensembled with program synthesis, which the paper puts at average human performance. Read as evidence about in-context learning’s limits as much as about the technique. | Active |
| LIT-381 | Scaling Diffusion Language Models via Adaptation from Autoregressive Models | Gong et al. (2024), ARXIV-2410.17891. Diffusion language models are hard to train from scratch at scale and there are a great many trained autoregressive ones, so convert instead: continual pretraining on under 200B tokens turns GPT2 and LLaMA at 127M–7B into DiffuGPT and DiffuLLaMA. The conversion rests on a stated connection between the AR and diffusion objectives rather than on the two being retrained independently. | Active |
| LIT-382 | Large Language Models Cannot Self-Correct Reasoning Yet | Huang et al. (2023), ARXIV-2310.01798. The negative result the self-correction literature is answering: asked to revise its own reasoning with no external feedback, a model does not reliably improve, and often gets worse. Names the thing being tested — INTRINSIC self-correction — and that distinction is most of the paper’s lasting value. | Active |
| LIT-383 | Training Language Models to Self-Correct via Reinforcement Learning | Kumar et al. (2024), ARXIV-2409.12917. SCoRe: multi-turn online RL on entirely self-generated correction traces, which teaches the self-correction that prompting could not produce. +15.6% on MATH and +9.1% on HumanEval for Gemini 1.0 Pro and 1.5 Flash. Its more transferable half is the diagnosis of why supervised fine-tuning on correction traces fails. | Active |
| LIT-425 v2 | Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer | Raffel et al. (2019), ARXIV-1910.10683. T5’s paper, and the origin of C4 — the heuristically filtered Common Crawl slice that forty-six documents here train on. Filed for the corpus: the record cites C4 constantly and had no document behind it. | Active |
| LIT-432 | Finetuned Language Models Are Zero-Shot Learners | Wei et al. (2021), ARXIV-2109.01652. FLAN, and the name “instruction tuning”: fine-tune on 60+ existing NLP tasks rewritten as natural-language instructions and zero-shot performance on unseen task types improves. The rival post-training answer to LIT-377’s, and the one InstructGPT measured itself against. | Active |
| LIT-433 | Learning to summarize from human feedback | Stiennon et al. (2020), ARXIV-2009.01325. The step where the preference loop crosses into language: human comparisons between summaries, a reward model fit to them, PPO against that reward. The summaries beat the human reference summaries, and beat much larger models fine-tuned supervised — which is the result that made the loop look like a general answer rather than a control technique. | Active |
| LIT-434 | Deep reinforcement learning from human preferences | Christiano et al. (2017), ARXIV-1706.03741. The construction the record’s whole post-training spine assumes and could not cite: when the reward function is the thing you cannot write down, fit one to human comparisons instead and optimize against the fit. Atari and simulated locomotion, on feedback covering under 1% of the agent’s interactions. | Active |
| LIT-467 | Chain-of-Thought Prompting Elicits Reasoning in Large Language Models | Wei et al. (2022), ARXIV-2201.11903. Put worked reasoning steps in the few-shot exemplars and a 540B model more than doubles on GSM8K. Three ablations rule out the obvious alternative explanations, and the effect is absent — sometimes negative — below about 100B parameters. | Active |
| LIT-473 | Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost | Xu, Miikkulainen and Qiu (2026), ARXIV-2602.03120. Fine-tune a quantized model in its own integer space: accumulate the part of each evolution-strategies update that is smaller than the lattice spacing until it crosses a grid point, and rematerialize that accumulator from stored seeds so it costs kilobytes instead of an FP16 copy of the weights. Countdown, Qwen2.5 at 1.5B and 3B, INT4/INT8/W8A8. | Active |
| LIT-474 | The Coverage Principle: How Pre-Training Enables Post-Training | Chen, Huang, Golowich, Malladi, Block, Ash, Krishnamurthy and Foster (2025), ARXIV-2510.15020. Names the quantity cross-entropy is a bad proxy for: the coverage profile, the probability mass a model puts on rare high-quality responses. Proves it is necessary and sufficient for Best-of-N to succeed, that next-token prediction optimizes it faster than it optimizes cross-entropy, and that sequence-level KL carries a linear dependence on sequence length that coverage does not. | Active |
| LIT-488 | gen2seg: Generative Models Enable Generalizable Instance Segmentation | Khangaonkar and Pirsiavash (2025), ARXIV-2505.15263. Finetune Stable Diffusion or MAE end-to-end — encoder and decoder — for category-agnostic instance segmentation on indoor furnishings and cars only, and the model segments people, animals, x-rays and impressionist paintings. It approaches SAM across five domains on 0.3% of the annotated masks, and beats it ~3× on fine structures. The controls are the argument: the same backbone under a standard segmentation head scores near zero. | Active |
| LIT-491 | Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models | Eyring et al. (2025), ARXIV-2508.09968. Reward-guided test-time noise optimization costs 20–40 seconds a sample. Train a LoRA hypernetwork once to predict the optimized noise instead, and recover roughly half the gain at 0.1 s of added latency — 33–300× faster. The comparison that carries it: direct reward fine-tuning of the same generator makes it worse (GenEval 0.73 → 0.62 at four steps), because the KL term that would stop reward-hacking is intractable in data space and tractable in noise space. | Active |
| LIT-505 v2 | Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences | Rives et al. (2021), PNAS 118(15) — the protein language model the biological-sequence line runs from. A transformer trained by masked-token prediction on 250 million protein sequences, with no structural or functional labels, produces representations in which secondary structure, tertiary contacts and remote homology are linearly recoverable. Held here as the ancestor of LIT-506, which reimplements the recipe for RNA and copies its downstream head. | Active |
| LIT-506 v2 | Interpretable RNA Foundation Model from Unannotated Data for Highly Accurate RNA Structure and Function Predictions | Chen et al. (2022), ARXIV-2204.00300 — a 12-layer BERT encoder trained by masked-token prediction on 23.7 million unannotated ncRNA sequences from RNAcentral, then used as a frozen feature extractor for secondary structure, 3D contact, protein-RNA binding and UTR regulation. Beats twelve secondary-structure methods (F1 0.941 on ArchiveII600, 0.704 on bpRNA TS0). Read as NOTE-252, which records that the paper’s one clean input-swap control moves R² by 0.002 on the generalization set. | Active |
| LIT-516 | From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications | Jaiswal et al. (2024), ARXIV-2407.11239 — how nearly low-rank a transformer weight matrix is depends on which matrix and how deep: Query/Key and MLP Gate converge to low rank, MLP Up/Down and Value do not, and middle blocks resist while the first and last few give way. Picking the rank per matrix instead of globally is ~6.4× better in perplexity than uniform reduction at 30% on LLaMA-2 7B and ~47× better at 40% on 13B. Read as NOTE-264. | Active |
| LIT-536 | Do pretrained Transformers Learn In-Context by Gradient Descent? | Shen, Mishra and Khashabi (2023), ARXIV-2310.08540, ICML 2024 — the challenge from outside. It separates two hypotheses the field had been conflating: that ICL in a naturally pretrained model is gradient descent, and that transformer weights exist that simulate it. Theorem 1 says an algorithm equivalent to ICL must share its order sensitivity, and gradient descent is order-stable while ICL is not; on LLaMA-7B, ICL and fine-tuning disagree on accuracy, top-10 token overlap and overlap cosine similarity across four datasets. Read as NOTE-276. | Active |
| LIT-549 | ReFT: Representation Finetuning for Language Models | Wu, Arora et al. (2024), ARXIV-2404.03592. Freeze the model and train a low-rank edit to the hidden state at a few prompt positions: h + Rᵀ(Wh + b − Rh), with R having orthonormal rows. It uses 15–65× fewer parameters than LoRA. It leads on commonsense QA (LLaMA-7B, 80.2 against LoRA’s 74.7), leads on GPT-4-judged instruction following, and is level on GLUE. It loses on arithmetic chain of thought (GSM8K 26.0 against 37.5). Baseline numbers are copied from earlier papers. | Active |
| LIT-574 | Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models | Hase, Bansal, Kim, Ghandeharioun (NeurIPS 2023), ARXIV-2301.04213. On 652 facts known to GPT-J, causal tracing often peaks outside the layers ROME and MEMIT edit. Yet editing at those layers works regardless. In a regression of ROME’s rewrite score, the edit layer explains 94.7% of the variance, and adding the fact’s tracing effect raises it to 94.8%. Robust across editing methods (ROME, MEMIT, fine-tuning), metrics, models and datasets. Localization, as causal tracing measures it, does not tell you where to edit. | Active |
| LIT-576 | Mass-Editing Memory in a Transformer | Meng, Sen Sharma, Andonian, Belinkov, Bau (2023), ARXIV-2210.07229. MEMIT writes thousands of facts at once by spreading each update over a range of mid-layer MLPs (layers 3–8 in GPT-J) in one batched least-squares solve. At 10,000 CounterFact edits it scores 85.8 on GPT-J (ROME 50.3, fine-tuning 67.6 with generation collapse). Neighborhood success falls from 83.5 to 73.7. | Active |
| LIT-577 | Evaluating the Ripple Effects of Knowledge Editing in Language Models | Cohen, Biran, Yoran, Globerson, Geva (2023), ARXIV-2307.12976. An edit implies other facts: logical consequences, multi-hop compositions, aliases. RippleEdits tests 5K edits on six such criteria. ROME, MEMIT and MEND average 38–66 across GPT-2, GPT-J, GPT-NeoX and LLaMA. Logical generalization can be as low as 5.5–7.0. Stating the new fact in the prompt (in-context editing) scores best, beating ROME by more than 10 points on GPT-NeoX and by 29 on LLaMA. | Active |
| LIT-578 | Locating and Editing Factual Associations in GPT | Meng, Bau, Andonian, Belinkov (2022), ARXIV-2202.05262. Causal tracing: corrupt the subject tokens, then restore single hidden states to see which carry the fact. The strongest effects are at mid-layer MLPs on the last subject token. ROME writes a new fact as a rank-one update to one such MLP, treated as a linear associative memory. On CounterFact it scores 89.2 (GPT-2 XL) and 91.5 (GPT-J) against at most 68.7 for the baselines. It edits one fact at a time and degrades after about 10. | Active |
| LIT-621 v2 | Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models | Blattmann et al., LMU Munich and NVIDIA (2023), ARXIV-2304.08818. Video LDM turns a pretrained image latent diffusion model into a video model by inserting temporal layers and fine-tuning the decoder on video. Starting from the image model halves driving-scene FVD against end-to-end training (534 against 1155). The paper counts 2.2B of the text-to-video model’s 3.1B parameters as trained, and that count leaves out the spatial layers and decoder it also fine-tuned. The abstract describes training only temporal layers. | Active |
| LIT-624 v3 | Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model | Step-Video Team, StepFun (2025), ARXIV-2502.10248. A 30B text-to-video DiT on a 16×16×8 VAE that reconstructs as well as HunyuanVideo’s 8×8×4 on its own benchmark, though not on Open-Sora 2.0’s, so the model covers 204 frames. It also applies DPO to video (a 55/45 preference score, ties split) and runs at 32% MFU. It reaches parity with Movie Gen and loses to the strongest commercial model it names, despite the abstract’s “state-of-the-art”. | Active |
| LIT-633 v2 | AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning | Guo et al., CUHK, Shanghai AI Lab and Stanford (2023), ARXIV-2307.04725. Stable Diffusion 1.5 is frozen and only zero-initialized temporal “motion modules” are trained. The modules then plug into community checkpoints fine-tuned from the same base, with no tuning. Transfer is measured by one table against two baselines not built for the task. The first version’s limitations section, which reports failure on stylized checkpoints, was dropped from the second. | Active |
| LIT-647 | Sequence Level Training with Recurrent Neural Networks | Ranzato, Chopra, Auli and Zaremba, Facebook AI Research (2015), ARXIV-1511.06732. The paper that named exposure bias. MIXER anneals a cross-entropy-trained RNN toward REINFORCE on the test metric over its own samples, and beats cross-entropy by 1 to 3 points on three tasks with greedy decoding. Its baselines that avoid exposure bias without a sequence-level reward do not reliably help, so the gain is not shown to come from fixing exposure bias. | Active |
| LIT-650 | Competition-Level Code Generation with AlphaCode | Li et al. (2022), ARXIV-2203.07814. Top 54.3% on ten Codeforces contests, by sampling up to a million programs per problem and filtering 99% of them away on the example tests. Two things the record needs are incidental to that: HumanEval’s false-positive rate, hand-measured at 30%, and the fact that pass@k assumes every sample can be submitted and is therefore an upper bound on what a system with a submission budget achieves. | Active |
| LIT-670 | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding | Devlin, Chang, Lee and Toutanova (2018), ARXIV-1810.04805. The model 89 files in this record referred to and none held. Two things it is cited for here turn out not to be in it: the 15% masking rate was never swept — “in all of our experiments, we mask 15%”, and the one appendix table headed “Masking Rates” varies the 80/10/10 replacement split at a fixed 15% — and its NSP ablation removes the loss while keeping the paired-segment input format, which LIT-671 later separates and reverses. | Active |
| LIT-674 | Robust fine-tuning of zero-shot models | Wortsman, Ilharco, Kim, Li, Kornblith, Roelofs, Gontijo-Lopes, Hajishirzi, Farhadi, Namkoong and Schmidt (2021), ARXIV-2109.01903. Fine-tuning a zero-shot model buys target accuracy and costs robustness. Linearly interpolating the zero-shot and fine-tuned weights recovers both: at α = 0.5, +3.5 to +23.2 pp under six distribution shifts against the fine-tuned model while reference accuracy falls by at most 0.3 pp and usually rises. No extra cost at training or inference. And it names the precondition: averaging all the layers of two unrelated networks “typically fails, achieving no better accuracy than a randomly initialized neural network” — this works because the trajectory is shared. | Active |
| LIT-675 | Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time | Wortsman, Ilharco, Gadre, Roelofs, Gontijo-Lopes, Morcos, Namkoong, Farhadi, Carmon, Kornblith and Schmidt (2022), ARXIV-2203.05482. The dependency SOTA-407’s consensus note named as unheld. A hyperparameter sweep throws away every model but one; average them instead. The greedy recipe — sort by held-out accuracy, add each model only if held-out accuracy improves — is “no worse than the best individual model on the held-out validation set” by construction, at O(1) inference cost against an ensemble’s O(k). +0.7 pp on CLIP and +0.5 pp on ALIGN over the best sweep member. It does not move that practice’s consensus, because it is the same first author. | Active |
| LIT-687 v2 | Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think | Martin Garcia, Knaebel, Schmidt, de Geus, Hermans and Leibe (WACV 2025), ARXIV-2409.11355. Marigold’s “dismal” few-step depth came from the DDIM scheduler’s leading timestep spacing. The spacing hands a pure-noise input a timestep that claims an almost clean one. Switching to trailing spacing makes one step usable (NYUv2 AbsRel 5.7). The fixed single step is still behind the 50-step, 10-member ensemble on all five depth sets. What beats that ensemble everywhere is end-to-end fine-tuning as a one-step model with a task loss, and it works nearly as well from plain Stable Diffusion. | Active |
| LIT-691 | Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation | Ke, Obukhov, Huang, Metzger, Daudt and Schindler (CVPR 2024), ARXIV-2312.02145. Marigold fine-tunes Stable Diffusion v2’s U-Net on 74K synthetic images to denoise a depth latent conditioned on the image latent, with the VAE frozen. It samples 50 DDIM steps and ensembles 10 runs, and ranks first on average across five zero-shot depth benchmarks. Its step-count curve, one step at ≈34% NYUv2 AbsRel, was measured under diffusers’ default leading spacing, which the paper never names. That is the scheduler bug LIT-687 corrects. The claim that a generative prior is what helps is not tested. | Active |
| LIT-702 | Simple and Effective Masked Diffusion Language Models | Sahoo et al. (2024), ARXIV-2406.07524. MDLM: the masked-diffusion training objective reduces to a weighted average of BERT-style masked-LM losses, a continuous-time ELBO whose value is invariant to the noise schedule. It is the strongest discrete diffusion language model at its scale — beating SEDD at matched tokens — and still trails a retrained autoregressive baseline in-domain (LM1B 27.04 vs 22.32; OWT 23.21 vs 17.54), which the authors state as the limitation. | Active |
| LIT-715 | Surya: Foundation Model for Heliophysics | Roy et al. (2025), ARXIV-2508.14112. A 366M-parameter transformer pretrained on full-resolution (4096²) multi-instrument SDO imagery to forecast the next frame, then rollout-tuned, forecasts solar dynamics zero-shot. With LoRA fine-tuning it beats task-specific baselines on four downstream heliophysics tasks. | Deferred — seeded from the abstract and a skim on 2026-09-25; not read in full |
| LIT-734 | Fine-Tuning Language Models from Human Preferences | Ziegler et al. (2019), ARXIV-1909.08593. The first time the human-preference loop is run on a pretrained language model: a 774M GPT-2 fine-tuned with RL against a reward model fit to human choices, with a KL penalty holding it near the pretrained model. On stylistic continuation, 5,000 comparisons give a policy humans prefer 86% of the time over zero-shot. On TL;DR and CNN/DM summarization, 60,000 comparisons produce “smart copiers” that labelers like but that may be exploiting the labelers’ heuristics, the problem LIT-433 set out to fix. | Active |
| LIT-744 | Text Embeddings by Weakly-Supervised Contrastive Pre-training | Wang et al. (2022), ARXIV-2212.03533 — E5. A BERT-initialised bi-encoder trained with InfoNCE and in-batch negatives (batch 32,768) on ~270M web text pairs, cut from ~1.3B by a consistency filter that keeps a pair only if a model trained on the noisy set ranks its passage in the top 2 of 1M. With no labelled data the base model averages 42.9 nDCG@10 on 15 BEIR datasets against BM25’s 41.7, the first unsupervised model reported above BM25; after fine-tuning on MS-MARCO, NQ and NLI the 330M large model averages 61.4 on 56 MTEB English datasets, above the 4.8B GTR-xxl and Sentence-T5-xxl. | Active |
| LIT-755 | Wonderland: Navigating 3D Scenes from a Single Image | Liang et al. (2024), ARXIV-2412.12091 — Wonderland (CVPR 2025). Fine-tune CogVideoX-5B-I2V for camera control with a ControlNet branch plus LoRA, then train a reconstructor (LaLRM) that regresses 3D Gaussians directly from the video latents rather than decoded frames. Reading latents beat reading RGB at matched cost (RealEstate10K PSNR 27.10 against 25.06). The reconstructor is trained on captured RealEstate10K, ACID and DL3DV first, then fine-tuned on those plus 20K videos the video model generated, supervised by its own decoded frames; that addition moved benchmark PSNR by at most 0.09. | Active |
| LIT-757 | Promptagator: Few-shot Dense Retrieval From 8 Examples | Dai et al. (2022), ARXIV-2209.11755 — Promptagator. Prompt a 137B FLAN with at most 8 examples from the target task to write queries for its corpus, keep a generated pair only if a retriever trained on the generated pairs ranks its source passage top-1, and train a 110M dual encoder on what is left. On 11 BEIR datasets it averages 47.8 nDCG@10 against 46.6 for SPLADE v2 and 46.2 for ColBERT v2, both trained on MS MARCO. The filter is worth 2.5 points on average and lowers three of the eleven datasets. | Active |
| LIT-762 | Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models | Kandpal et al. (2023), ARXIV-2306.04529 — Git-Theta, a Git extension that versions a checkpoint per parameter group rather than as one blob. Unchanged groups (detected with a locality-sensitive hash) are stored as references; changed groups are stored as typed updates (dense, sparse, low-rank, (IA)³) that are reconstructed by walking back through history. In one T0-3B workflow it stored 41.5 GB against Git LFS’s 57.0 GB, the saving coming almost entirely from a LoRA commit (0.27 GB against 11.4); dense fine-tunes were stored nearly in full, and every add was slower than Git LFS. | Active |
| LIT-769 | Structural Decoupling: A Scaffold-Flow Theory of Generalization and Alignment | Li (2025), ARXIV-2506.20699. A theory paper with no experiments. It defines a problem’s width, the minimum number of locally feasible contexts needed to cover it, and states that width is incomparable with VC dimension and that a learner with fewer contexts than the width has an error floor no amount of within-context training removes. From the additive split of complexity into a routing term and a within-context term it argues that the parts of a model that keep context structure should not be trained by the gradients that fit predictions inside a context, and it reads hallucination, reward hacking and deceptive alignment as failures of that structure. The current version (v2, 2026) replaced a v1 about a “Context-Content Uncertainty Principle” for brains and machines. | Active |
| LIT-773 | Improving Video Generation with Human Feedback | Liu, Liu, Liang et al., CUHK, Tsinghua and Kuaishou (2025), ARXIV-2501.13918. Three parts. A 182k-pair human preference set over videos from 12 text-to-video systems, rated separately for visual quality, motion and text alignment. VideoReward, a Qwen2-VL-2B reward model trained with a Bradley–Terry loss that models ties. Flow-DPO, Diffusion-DPO rewritten for rectified flow. Carried over exactly, the loss weights the KL by β(1 − t)², which goes to zero at the noise end; that variant raises the reward’s quality scores and drops its text alignment below the base model’s. A constant β fixes it. On an internal 1B video model with LoRA, constant-β Flow-DPO beats SFT and Flow-RWR, and humans prefer it to the base 44% to 27% with 29% ties. Training labels and automatic win rates come from the same reward model, and VBench barely moves. | Active |
| LIT-774 | Diffusion Model Alignment Using Direct Preference Optimization | Wallace, Dang, Rafailov et al., Salesforce AI and Stanford (2023), ARXIV-2311.12908. DPO for diffusion models. The intractable image likelihood is replaced by a bound on the path likelihood, and the reverse chain is approximated by the forward one. The loss compares how much the model beats a frozen reference at denoising the preferred image against how much it beats it on the rejected one, at the same noise and timestep. Trained offline on 851,293 Pick-a-Pic pairs, the tuned SDXL-base is preferred to the untuned base 70.0% of the time on PartiPrompts, and to SDXL base+refiner 69%, by majority vote of five crowdworkers. β is 2000–5000 with the learning rate scaled by 1/β. Fine-tuning on the preferred images alone makes SDXL worse. Checkpoints are chosen by PickScore, which was trained on the same source. There is one run per setting and no intervals. | Active |
| LIT-776 | Training Diffusion Models with Reinforcement Learning | Black, Janner, Du, Kostrikov and Levine, UC Berkeley and MIT (2023), ARXIV-2305.13301. Treat each denoising step of a stochastic sampler as an action. Each step is then an isotropic Gaussian with an exact likelihood, so policy gradient (DDPO) can fine-tune a diffusion model on any black-box reward of the final image. On Stable Diffusion v1.4, with JPEG compressibility, LAION aesthetic score and a LLaVA + BERTScore alignment reward, both DDPO estimators beat reward-weighted regression per reward query, and the PPO-clipped one is slightly ahead. There is no KL term. Over-optimization is shown and handled by picking checkpoints by hand. Prompt sets are narrow (45–398 animals), there is no human evaluation, and the run count is not stated. | Active |
| LIT-778 | DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models | Fan, Watkins, Du, Liu, Ryu, Boutilier, Abbeel, Ghavamzadeh, K. Lee and K. Lee, Google Research, UW–Madison, UC Berkeley, Amazon and KAIST (2023), ARXIV-2305.16381. Policy gradient over the denoising steps, as in DDPO, with a KL to the pre-trained model. The KL on the final image is intractable, so each step’s KL is summed, which bounds it from above. Stable Diffusion v1.5 with LoRA is trained on ImageReward, one model per prompt, against the same reward-weighted objective trained offline on 20K samples. Online RL scores higher on ImageReward and on the held-out aesthetic predictor, and eight raters prefer it. The KL ablation is one prompt. The training reward is also the main metric, and no human compares against the base model. | Active |
| LIT-779 v2 | Flow-GRPO: Training Flow Matching Models via Online RL | Liu, Liu et al., CUHK, Tsinghua and Kuaishou (2025), ARXIV-2505.05470. GRPO on a flow-matching image model. The deterministic sampler is swapped for an SDE with the same marginals, built from the model’s own velocity, which makes each step a Gaussian policy with a closed-form KL. Rollouts use 10 steps and evaluation 40. SD3.5-M with LoRA: GenEval 0.63 → 0.95, OCR accuracy 0.59 → 0.92, PickScore 21.72 → 23.31. Without the KL penalty, quality metrics fall on two tasks and diversity collapses on the third. GenEval is trained on its own templates and scored by its own detector. Quality is judged only by reward models, and there is one run per setting. | Active |
| LIT-782 | Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation | Kirstain, Polyak, Singer, Matiana, Penna and Levy, Tel Aviv University and Stability AI (2023), ARXIV-2305.01569. A web app that generates two images per prompt and logs which one the user keeps, or a tie. The paper’s version holds 583,747 training pairs over 37,523 prompts from 4,375 users. PickScore is CLIP-H fine-tuned on those pairs with a reward-model loss. It predicts held-out user choices at 70.5% under a tie-aware accuracy where chance is 56.8%, against 68.0% for the authors’ expert annotators. Its ranking of 45 model configurations correlates 0.790 with real users’ Elo. On MS-COCO captions FID ranks nine configurations at −0.900 against human experts, because people prefer higher guidance. Every validation is in-distribution or uses colleague annotators. Diffusion-DPO trains on this data and Flow-GRPO optimizes PickScore as a reward. | Active |
| LIT-788 | ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation | Xu, Liu, Wu, Tong, Li, Ding, Tang and Dong, Tsinghua, Zhipu AI and BUPT (2023), ARXIV-2304.05977. A reward model for text-to-image preference: a BLIP backbone trained on 136,892 ranked pairs over 8,878 DiffusionDB prompts. It picks the rater’s choice 65.14% of the time, against 54.82% for CLIP similarity, about as often as one annotator agrees with another (65.3%). ReFL, added in v3, backpropagates the reward through one denoising step taken late in the chain. On Stable Diffusion v1.4 it is ranked over the base model 58.79% of the time, ahead of filtering, reward weighting and RAFT. Best-of-64 under the same reward reaches 73.33%. Its premise, that a diffusion model has no likelihood for RLHF to use, is true of the whole chain and not of a step. | Active |
| LIT-795 | Scaling Laws for Reward Model Overoptimization | Gao, Schulman and Hilton, OpenAI (2022), ARXIV-2210.10760. InstructGPT’s 6B reward model stands in for people and labels comparisons for proxy reward models of 3M to 3B parameters. Optimizing a 1.2B policy against a proxy, the gold reward rises and then falls as a function of d = √KL from the initial policy: d(α − βd) for best-of-n and d(α − β log d) for PPO. The coefficients move smoothly with the proxy’s size, so the peak is predictable. Larger proxies and more comparison data push the peak out, and a 6B policy peaks at the same KL as a 1.2B one. A KL penalty in the reward did not improve gold reward at a given KL; its effect was like early stopping, a result the authors flag as possibly hyperparameter-sensitive. RL spends far more KL than best-of-n, so KL does not compare optimization across methods. Synthetic labels, one environment. | Active |
| LIT-797 | Aligning Text-to-Image Models using Human Feedback | Lee, Liu, Ryu, Watkins, Du, Boutilier, Abbeel, Ghavamzadeh and Gu, Google Research and UC Berkeley (2023), ARXIV-2302.12192. RLHF’s recipe carried to a text-to-image model, without the RL. Two labellers mark 27,528 Stable Diffusion v1.5 images good or bad on templated count, colour and background prompts. A small head on frozen CLIP embeddings learns that label. One offline round of reward-weighted likelihood plus a pre-training loss then fine-tunes the model. Alignment wins 50% of 120 prompts by a two-thirds vote against 3% for the base. Fidelity loses: FID 13.97 to 16.76, and 3% against 20% when compared with best-of-16 sampling from the base under the same reward. One run, no diversity measure. | Active |
| LIT-799 | Olmo Hybrid: From Theory to Practice and Back | Merrill et al., Ai2 and Lambda (2026), ARXIV-2604.03444. Olmo 3 7B with its sliding-window layers replaced by Gated DeltaNet (negative eigenvalues), at 3:1 with full attention, trained to 6T tokens. It reaches Olmo 3’s MMLU in 49% fewer tokens and beats it in every domain after mid-training, though data mix, schedule and mid-training batch also changed. Scaling fits from 60M to 1B give the hybrid a lower data coefficient (83.7 against 94.9, non-overlapping CIs); the projected 1.3–1.9× token savings have CIs that include 1 at every size but 3B. A theorem shows hybrids express a task neither parent can. Removing negative eigenvalues barely changes the scaling, which weakens the expressivity explanation. | Active |
| LIT-801 | Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension | Bertsch, Soldaini, Gormley, Neubig, Hajishirzi, Lo and Groeneveld, Ai2 and CMU (2026), ARXIV-2608.10296. Twenty-six 7–8B models, 140B tokens each on identical data, extended identically to 64K. QK norm, GQA, sliding-window layers and a 4K pretraining length each cost long-context ability, and together they compound: HELMET at 32K runs from 56.4 down to 29.9. Counting how many of the four a model has predicts that score with in-sample R² 0.67 (0.61 leave-one-out). Pretraining loss predicts it at R² 0.29. QK norm is the largest single cost, about 4 to 6 points. GQA plus sliding windows costs about 9, against 1.1 for windows alone. One seed per configuration, and initialization alone moves scores by 7.7% of the range on average. | Active |