adaptation-and-tuning.
44 of 378 NOTE documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| NOTE-020 | Don’t Stop Pretraining | Two adaptive-pretraining stages, and the second is the surprise. Domain-adaptive pretraining (DAPT) on a large in-domain corpus helps; task-adaptive pretraining (TAPT) on the task’s own small unlabelled set also helps, and the two are complementary rather than alternatives. | Read |
| NOTE-024 v2 | ReLoRA: High-Rank Training Through Low-Rank Updates | Restart LoRA repeatedly during pretraining — merge the adapter, reinitialise it, prune the optimizer state, re-warm the learning rate — so a sequence of low-rank updates sums to a high-rank one. The ablation is the finding: the full-rank warm start it also requires accounts for most of the measured gain, and the restart machinery adds 0.42 perplexity on top of it. | Read |
| NOTE-026 | T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models | Argues the control is already in the frozen model and only needs aligning to an external signal, so the adapter can be small — it learns an alignment, not a generation ability. Adapters compose, and transfer unchanged to any model fine-tuned from the same base. The composition weights are manual, which the authors state as the limitation. | Read |
| NOTE-027 | Simple Open-Vocabulary Object Detection with Vision Transformers | Transfers a contrastively pretrained image-text model to open-vocabulary detection with minimal architectural change. Its transferable finding is about fine-tuning: the text encoder needs a learning rate about 100× lower than the image encoder, freezing it entirely is also wrong, and the recipe that maximises zero-shot transfer is not the one that maximises in-distribution performance. | Read |
| NOTE-034 | An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion | Personalizes a frozen text-to-image model by optimizing a single new token embedding — nothing else changes. Frames the result as a point on a distortion/editability tradeoff borrowed from GAN inversion, and adds per-image tokens so the shared concept and the incidental details separate into different embeddings. | Read |
| NOTE-036 | DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation | Fine-tunes the whole model to bind a rare token to a subject from 3–5 images, and counters the resulting language drift with a prior-preservation loss that supervises the model with its own pre-fine-tuning samples. Beats textual inversion on both subject and prompt fidelity, and needs the super-resolution stages fine-tuned too. | Read |
| NOTE-038 | PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding | Fuses an arbitrary number of reference images into one “stacked ID embedding” attached to a class word, so personalization needs no per-subject tuning at inference. Because the stack is just a set, feeding it images of different people at inference merges identities — a capability that falls out of the representation rather than being designed. | Read |
| NOTE-049 | Learning to Prompt for Vision-Language Models | Replaces hand-written CLIP prompts with continuous context vectors learned end-to-end while every pretrained parameter stays frozen. One or two shots beat prompt engineering; sixteen shots average about 15% gain, over 45% at the best. The paper also reports where it overfits and names the cause as noisy labels. | Read |
| NOTE-071 | Adding Conditional Control to Text-to-Image Diffusion Models | Adds spatial control to a frozen diffusion model by cloning its encoder into a trainable branch joined to the original by zero-initialized convolutions, so the adapter starts as an exact no-op and grows away from it. Trains robustly on datasets from under 50k to over a million. | Read |
| NOTE-075 | Hyper-ES | Argues from two lemmas that full-parameter ES cannot work at LLM scale — random perturbations concentrate near orthogonality to any useful descent direction, and the leftover orthogonal components accumulate as a random walk away from the pretrained weights — and replaces the search space with the span of a few cheap GRPO-derived LoRA directions, searched by CMA-ES. | Read |
| NOTE-076 | Understanding Evolution Strategies for LLM Reasoning | Three questions about what ES post-training does rather than whether it wins: it raises pass@1 and pass@k together where GRPO trades one for the other, its 40x larger parameter drift is functionally sparse and does not produce broad forgetting, and the population size needed for stable training falls as the model grows. | Read |
| NOTE-077 | Beyond the Best Guess | Reinforcement learning raises pass@1 and lowers pass@k, eventually below the base model, because it increases the number of prompts solved in none of k samples; evolution strategies raise both and never fall below base at any k or scale tested, up to 32B against published RL checkpoints. | Read |
| NOTE-078 | Neural Thickets | The neighbourhood of a pretrained weight vector is dense with task-improving Gaussian perturbations, the density rises monotonically with model scale, and the perturbations that help are specialists rather than uniform improvements — which together are why a single parallel round of random guessing plus majority voting can match PPO, GRPO and ES at equal training FLOPs. | Read |
| NOTE-081 | Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies | Prior-task degradation under ES is transient drift rather than irreversible forgetting, is not specific to ES, and is driven by the random walk Hoy et al. predicted — whose size falls with population exactly as their equation says. Anchored Weight Decay buys a population-128 reduction at population 30 for 1-2% runtime. | Read |
| NOTE-082 | Evolutionary Strategies lead to Catastrophic Forgetting in LLMs | The first independent replication attempt of ES-beats-GRPO, which fails — ES lands close but GRPO stays ahead on three of four settings at 1B and 1.5B — and the first measurement of what ES costs on a held-out capability: HellaSwag declines steadily as Countdown training continues past the point Countdown itself converged. | Read |
| NOTE-083 | Matching Accuracy, Different Geometry | ES and GRPO reach the same accuracy by travelling in nearly orthogonal directions two orders of magnitude apart in length, and land in the same basin with no loss barrier between them — because most of the ES displacement is a loss-invariant random walk whose squared norm grows as sigma^2 d T / N. | Read |
| NOTE-084 | The Blessing of Dimensionality in LLM Fine-tuning | One geometric property — a fine-tuning landscape whose curvature lives in a few stiff directions amid a near-zero bulk — accounts for both why a population of thirty keeps working as models grow and why the training reward rises, peaks and decays under fixed hyperparameters, in GRPO as well as ES. | Read |
| NOTE-085 | ESSA | The earliest ES-for-LLM paper in this record, and the one that accepted the dimensionality objection instead of overturning it: SFT-train LoRA adapters, take their SVD, and run CMA-ES over the top singular values only — reaching 0.80 on GSM8K in under 25 minutes against GRPO’s 50+, from 100 examples against 5,978, and aligning a 32B model at INT4. | Read |
| NOTE-086 | ESSAM | Transposes sharpness-aware minimization into zeroth order — step against the reward-weighted direction, recompute the update at that neighbouring point, apply it at the original — and closes most of standard ES’s gap to GRPO on GSM8K while keeping inference-level GPU memory. | Read |
| NOTE-161 | Training language models to follow instructions with human feedback | Making a language model bigger does not make it better at following instructions, so stop buying parameters and buy human feedback instead: demonstrations to fine-tune on, rankings to fit a reward model to, then policy optimization against that reward with a KL leash back to the supervised model. A 1.3B model tuned this way is preferred to 175B GPT-3. The cost is a measurable regression on held-out NLP tasks, and the paper supplies both the name for it and a fix. | Read |
| NOTE-164 | QLoRA: Efficient Finetuning of Quantized LLMs | Store the frozen base in 4 bits, compute in 16, and train only LoRA adapters: 65B fine-tuning goes from >780GB to <48GB with no measured loss against a 16-bit fully fine-tuned baseline. The quantization is the headline; the more useful result for anyone already using LoRA is that the standard query/value placement does not reach full fine-tuning at scale, and that the adapter count rather than the rank is what closes the gap. | Read |
| NOTE-165 | Large Language Models Cannot Self-Correct Reasoning Yet | Asked to revise its own reasoning with no external feedback, a model does not reliably improve and sometimes degrades. The durable part is the definition rather than the number: INTRINSIC self-correction, meaning no oracle, no verifier, no second model — a line that a lot of contemporary self-correction claims were quietly on the other side of. | Read |
| NOTE-166 | Scaling Diffusion Language Models via Adaptation from Autoregressive Models | Continual pretraining on under 200B tokens converts GPT2 and LLaMA at 127M-7B into diffusion language models competitive with their AR counterparts. The argument is economic before it is technical — diffusion LMs are hard to train at scale and AR checkpoints are abundant — and the capability that pays for it is infilling without prompt re-ordering. | Read |
| NOTE-167 | The Surprising Effectiveness of Test-Time Training for Few-Shot Learning | Turn the in-context examples into a loss and take gradient steps on them at inference, then discard the update. Up to 6x a fine-tuned baseline on ARC and +7.3 points on BIG-Bench Hard at 10-shot. The finding underneath is about in-context learning: the same examples are worth several times more as gradient than as context. | Read |
| NOTE-169 | Training Language Models to Self-Correct via Reinforcement Learning | SCoRe teaches self-correction with multi-turn online RL on the model’s own traces, no teacher and no extra supervision, for +15.6% on MATH and +9.1% on HumanEval. The transferable half is the diagnosis: SFT on correction traces fails by distribution mismatch or by behaviour collapse, and the method’s two stages are each aimed at one of those. | Read |
| NOTE-182 | P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks | Prompt tuning did not fail because tuning a prompt is a weak idea; it failed because the prompt was only at the input layer, where it has too few parameters and too indirect a route to the prediction. Put prefix tokens at every layer and the same family matches full fine-tuning from 330M to 10B and on hard sequence labelling, at 0.1-3% of the parameters. The paper is explicit that it is not conceptually novel — it is deep prompt tuning, already published for generation, adapted to NLU and optimized properly. | Read |
| NOTE-216 | Chain-of-Thought Prompting | Worked reasoning steps in the few-shot exemplars, nothing else. Reading it: the three ablations are the contribution — it is not the equation, not the extra tokens, and not knowledge activation — and the effect is absent or negative below about 100B parameters. | Read |
| NOTE-222 | Quantized Evolution Strategies | Reading it: the memory trick is the contribution, not the error feedback. Error feedback is 1-bit SGD’s, correctly credited; what is new is refusing to store the accumulator at all and rebuilding it from seeds, which is the only reason the method fits in the memory budget that motivated it. | Read |
| NOTE-223 | The Coverage Principle | Reading it: the strongest thing here is not the theorem but the Bernoulli example, which shows KL going to infinity while coverage stays fine on a two-outcome model. That is the whole argument in four lines, and it explains why a metric everybody watches can be anti-correlated with what they want. | Read |
| NOTE-237 | gen2seg | Read as the counter-route to SAM’s data engine. The headline (narrow finetuning, broad generalization) is well evidenced but the argument lives in the controls, which are unusually good: the same backbone under a conventional head scores ~1.5 mIoU, and swapping only the decoder recovers part of the gap. | Read |
| NOTE-240 | Noise hypernetworks | Read as the amortized counterpart to SOTA-301. The headline — recover half of test-time optimization at 1/100th the latency — is well measured. The finding that carries further is the control: direct reward fine-tuning of the same model makes it worse, and worse the more steps you sample. | Read |
| NOTE-252 | RNA-FM, and the control that does not carry the claim | Read to open the biological-sequence trunk, which this record did not hold. The structural results are large and the paper’s one clean input-swap control is not: replacing one-hot sequence with a 640-dimensional pretrained embedding in a published UTR model moves R² from 0.814 to 0.816 on the real-human set, where a 16-dimensional predicted secondary structure moves it to 0.820. No practice filed. | Read |
| NOTE-264 | Low-rank structure in a transformer is non-uniform, and a global rank is measurably the wrong choice | Read second in the spectral cluster, and it is the one that carries a recommendation. “Transformer weights are low rank” turns out to be true of some matrices and false of others, with the split systematic by component and by depth. The comparison against a uniform rank at matched compression was run and the margins are large. | Read |
| NOTE-276 | Order-sensitivity refutes it, and the models everyone measured were trained for the task | Read as the challenge from outside the setup. It separates the hypothesis that pretrained models do in-context learning by gradient descent from the hypothesis that transformer weights exist which simulate it, and shows the literature had been testing only the second. Theorem 1 then gives a contradiction from order sensitivity alone, and LLaMA-7B disagrees with fine-tuning on three metrics across four datasets. | Read |
| NOTE-290 | ReFT | A learned low-rank edit to the residual stream at a few prompt positions, with the model frozen. Up to 65× fewer parameters than LoRA. It leads on commonsense QA and on GPT-4-judged instruction following, is level on GLUE, and loses on arithmetic chain of thought. All baselines are copied from earlier papers. Read §1–6 and Appendix D; the other appendices were not read. | Read |
| NOTE-313 | Does Localization Inform Editing? | Per fact, causal tracing’s localization is unrelated to where an edit succeeds. The edit layer explains 94.7% of the variance in ROME’s success on GPT-J, and tracing adds 0.1%. Main text read, robustness appendices skimmed. | Read |
| NOTE-314 | RippleEdits | Knowledge-editing benchmarks test the edited fact and leave its consequences untested. On six ripple criteria, weight editors average 38–66, and prompting with the new fact beats them. Main text read. | Read |
| NOTE-315 | ROME | Causal tracing localizes factual recall to mid-layer MLPs at the last subject token, and a rank-one MLP edit there rewrites single facts with both generalization and specificity. Main text and the CounterFact appendix read. | Read |
| NOTE-316 | MEMIT | ROME’s idea extended to thousands of facts by spreading updates across a range of MLP layers in one batched solve. 85.8 on 10,000 CounterFact edits on GPT-J, where ROME falls to 50.3. Main text read, appendices skimmed. | Read |
| NOTE-338 v2 | Step-Video-T2V | A 30B text-to-video DiT on a 16×16×8 VAE. By the authors’ own benchmark the VAE reconstructs about as well as HunyuanVideo’s 8×8×4, and an independent measurement in Open-Sora 2.0 disagrees. Video DPO raises a tie-weighted preference score from 45% to 55% on 300 prompts. Every architecture comparison is reported without numbers, and the “state of the art” in the abstract is contradicted by the paper’s own Table 9. | Read |
| NOTE-339 | AnimateDiff | Temporal-attention modules trained on video over a frozen Stable Diffusion 1.5 plug into community checkpoints fine-tuned from the same base without further tuning. Against two baselines not built for the task it wins clearly only on smoothness: users ranked the training-free Text2Video-Zero higher on preserving the personalized domain (2.620 against 2.280), which is the property the design exists for. Version 1’s reported failure on stylized domains was dropped in version 2, and its “most checkpoints” became “any”. | Read |
| NOTE-341 | Video LDM (Align your Latents) | Video LDM makes a latent image diffusion model into a video model by inserting temporal layers and fine-tuning the decoder and upsampler on video. On driving scenes, an image-pretrained LDM with frozen spatial layers reaches FVD 534.17 against 1155.10 for the same design trained from scratch. The “images” are 1 fps frames of the same driving videos, and the pretrained arm gets 73K image steps the other does not. Decoder video fine-tuning cuts reconstruction FVD 390.88 → 32.94. | Read |
| NOTE-360 | MDLM | Masked diffusion’s ELBO is a schedule-weighted average of masked-LM losses, and with a modern recipe it is the best discrete diffusion language model at 110M parameters. It does not beat autoregression in-domain, even with the AR baseline given half the optimizer steps; the paper’s own checklist names that as its limitation. | Read |
| NOTE-376 | Surya: heliophysics foundation model | A 366M-parameter transformer pretrained on full-resolution (4096²) multi-instrument SDO imagery to forecast the next frame, then rollout-tuned, forecasts solar dynamics zero-shot. With LoRA fine-tuning it beats task-specific baselines on four downstream heliophysics tasks. | Skimmed |