Source
Kumar et al. (2024), LIT-383 — ARXIV-2409.12917.
Why not just ask
Because that was tried and measured. LIT-382 defines intrinsic self-correction — revision on the model’s own capabilities, with no oracle, verifier or second model — and finds that on reasoning it does not reliably improve and sometimes degrades.
The reason is structural rather than a prompting deficiency. With no external signal the model cannot distinguish its correct answers from its incorrect ones, so a revision pass is about as likely to overwrite a right answer as a wrong one. Where most first attempts are already right, the expected value of revision is negative.
Check which side of that line any self-correction claim sits on. If an oracle, a verifier or a critic model is in the loop, the gain is attributable to that information and not to a capacity the model has.
Why not supervised fine-tuning on correction traces
The obvious next move, and it fails in two distinguishable ways:
- Distribution mismatch — traces collected from some other policy contain that policy’s mistakes, so the model learns to repair errors it does not make.
- Behaviour collapse — training settles into a single correction mode that scores well on training prompts and does not correct at test time. This one looks like success until evaluation on new problems.
What to do instead
Train under the model’s own distribution of self-generated correction traces, with regularization aimed at collapse: a first phase of multi-turn RL on the base model to reach an initialization less prone to it, then a reward bonus that amplifies correction specifically rather than first-attempt quality.
No second model, no stronger teacher, no additional supervision — which is what keeps this a training method rather than a distillation result.
This is SCoRe, and the recommendation originates with LIT-383: Kumar et al. took the negative result on prompted self-correction as given, diagnosed the two SFT failure modes above, and showed that multi-turn online RL on self-generated traces improves the base models’ self-correction by 15.6% on MATH and 9.1% on HumanEval, on Gemini 1.0 Pro and 1.5 Flash.
Conditions
Reported on Gemini 1.0 Pro and 1.5 Flash, on MATH and HumanEval, by one group. The choice of intervention is what this practice records — train it, on-policy — and not the exact two-stage schedule, which is one recipe from one set of experiments and would be overread as a recommendation.
Self-correction also has a cost that none of these papers price: it spends extra decoding on every attempt. A capability that improves accuracy by 15.6% while doubling the tokens is a different trade from one that is free, and the record has no measurement of that trade to point at.
Where this rhymes
SOTA-130 records the R1-Zero line — skip the supervised reasoning stage, run RL with verifiable rewards on the base model. Both practices conclude a supervised stage is insufficient for a reasoning-adjacent capability and reach for online RL.
They are separate claims and stay separate. What this one adds is the
mechanism: SOTA-130 reports that skipping SFT works, and this reports why
fitting offline traces fails, with the two modes named. The mechanism is the
half that transfers to capabilities neither paper studied.