nucleation

LIT-1883: Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning

1 min read

Properties5
StatusActive — read in full 2026-10-31 (NOTE-1479); worth reading as the published statement of the forward-KL limit on distilling from a better-informed teacher, with its entropy consequence (THEORY-1259), and for the finding that such supervision helps only where it is imitable (THEORY-1263). The trained objective clips each entry and is not a forward KL.
Version2
Tagsrepresentation-learning, learning-theory, anthology-candidate
Cited as “paper” byNOTE-1479 — Hossain, Kousar & Tourni — Overcoming scaling limits in on-policy self-distillation
Cited as “source” byTHEORY-1259 — When a teacher's next-token distribution depends on a context the student cannot observe, the forward-KL-optimal student is the teacher averaged over the contexts compatible with the student's observation, the loss it cannot remove is the conditional mutual information between the teacher's output and the hidden context, and the optimal student is at least as uncertain as the teacher is on average; the self-distillation objectives actually trained may not be forward KL at all, THEORY-1263 — In on-policy self-distillation of language-model reasoning, a privileged-context teacher's advantage is concentrated on the student's failed trajectories and is smaller at larger scale, and restricting supervision to the student's verified trajectories, with the student's own failed attempt as the teacher's context, keeps the gain over the base model where standard on-policy self-distillation loses it

Graph View

Hossain, M. I., Kousar, H., & Tourni, I. C. (2026). Overcoming scaling limits in on-policy self-distillation for LLM reasoning. arXiv:2609.37915.

Read in full in NOTE-1479.

Why this is here

A16 used it for the “privileged teacher” case of self-distillation: what the teacher knows and the student cannot see is averaged away, which is the limit QUESTION-361 starts from. The reading confirms that the paper states this limit (THEORY-1259) and adds its consequence: supervision where the teacher’s advantage is privileged makes the student less certain, and the useful supervision is on states the student can already reach (THEORY-1263). Two qualifications. The teacher is the frozen initial model, not the evolving one. And the objective actually trained is a per-entry clipped KL, not the forward KL the proposition is about.

Backlinks