nucleation

LIT-1884: Why Self-Training Helps and Hurts: Denoising vs. Signal Forgetting

1 min read

Properties5
StatusActive — read 2026-10-31 (NOTE-1471), body and appendices, with the proofs skimmed; worth reading for an exact account of iterated reconstruction through fresh low-rank views, in which forgetting the ancestor first improves and then destroys fidelity to the world, and every student fits its teacher exactly on its own sample (THEORY-1269).
Version2
Tagslearning-theory, mathematical-statistics, anthology-candidate
Cited as “paper” byNOTE-1471 — Wu, Yang & Sun — Self-training: denoising vs. signal forgetting
Cited as “source” byTHEORY-1269 — In overparameterized linear regression, iterated self-training with fresh covariates, noiseless pseudo-labels and minimum-norm students applies a fresh random orthogonal projection to the estimate each round, so noise from the first fit decays geometrically while signal along a direction of variance s decays at the slower rate s/(s + τ); risk is U-shaped under anisotropy and monotone under isotropy, tends to the risk of predicting zero, and every student fits its teacher exactly on its own sample

Graph View

Wu, M., Yang, A. Y., & Sun, Q. (2026). Why self-training helps and hurts: Denoising vs. signal forgetting. arXiv:2602.14029.

Read in NOTE-1471.

Why this is here

With LIT-1895, A16’s evidence that recursive reconstruction can lose as well as correct. It is the formal neighbour of CLAIM-081 and CLAIM-270 on forgetting the ancestor. The reading makes the neighbourhood precise. The chain forgets its ancestor entirely, and since the ancestor carries noise, forgetting it first improves fidelity to the source and then destroys it. A single projection repeated would change nothing; the loss comes from drawing a new one each round (THEORY-1269). Each student also fits its teacher exactly on its own sample while differing from it elsewhere, an illustration for CLAIM-517.

Backlinks