Li (2025) — ARXIV-2506.20699
Key takeaways
Two versions, two papers. v1 (25 June 2025) was On Context-Content Uncertainty Principle. Its argument is that inference should go from low-entropy content (priors, schemas) to high-entropy context (input). From that it derives four layers of principles: structure before specificity, precision-weighted attention, asymmetric learning rates (“learn content slowly, specificity quickly”), structure-first curricula, and attractor memory. It then presents the framework as a synthesis of brain theories: predictive coding, the free energy principle, active inference and attractor dynamics. The v1 abstract mentions “computational simulations”, but the v1 body is lemmas, theorems and an appendix of proofs, with no simulation in it. v2 (8 June 2026) keeps the author and the arXiv id and replaces the content with Structural Learning Theory (StrLT). The rest of this note is about v2, the version the id now resolves to. v1 is held, with a close reading, in the companion record nucleation, on the owner’s instruction to file this paper in both (see issue #180).
- Width as a second complexity axis. A cell is feasible when one local predictor is both contractive and low-risk on it. Width is the minimum number of feasible cells that cover the input space. Theorem III.2 states that width and VC dimension are incomparable. One witness is a bouquet of circles, where width grows with the number of loops while the local predictor class stays fixed. The other is polynomial regression on one interval, where VC dimension grows with degree while the width stays one.
- Phase transition at the true width (Theorem III.3). With at least as many cells as the width, the problem splits into ordinary per-cell learning problems at the usual metric rate. With fewer, every learner has a structural error floor. Theorem III.4 adds a coupon-collector lower bound on the samples needed to identify every basin, so sample complexity splits into a structural term and a per-cell term.
- Estimating width. The contractive-similarity kernel weights a graph edge by geometric proximity and by agreement of predictions. Its Laplacian therefore separates basins that are connected in input space but incompatible in prediction (Theorem III.6), which an ordinary graph Laplacian cannot do. Penalized structural ERM then selects the true width almost surely, under a positive structural gap and stated regularity assumptions (Theorem III.7).
- The decoupling principle. Theorem IV.5 states that structural Rademacher complexity is a routing term plus a per-cell term, with no cross-term. The paper reads that as the reason to keep separate mechanisms for the scaffold (context, routing, boundaries) and the flow (prediction inside a context), and not to train the scaffold with the flow’s gradients. The paper’s own example is mixture-of-experts: routers and experts trained jointly on one task loss “blur” structural discovery and within-expert learning. Catastrophic forgetting is read as structural under-resolution.
- Safety readings, all argued and none measured. Deceptive alignment is cast as scaffold-flow decoupling, with grokking cited as the benign existence proof that a flow fit fast can sit on a structure that consolidates slowly. The paper says grokking has the same shape as the width transition and a different mechanism, because it happens inside a single cell. A miscalibrated reward model is read as a closure gate that writes a wrong invariant into the scaffold. Hallucination and misalignment are both read as boundary-resolution failures, and the paper’s evaluation proposal is to test boundary regions rather than the interiors of familiar basins. It also proposes an “anesthesia-mode” deployment, in which a system may use its scaffold but not consolidate new structure.
- Not ablated, not run. The paper has no experiment. §VI says that large-scale empirical evaluation of width estimation “remains open”. It says that existing continual-learning benchmarks conflate width with VC dimension, and it proposes benchmarks with known width as the test. Each claim about deployed models is an interpretation inside the framework.
Standing in the anthology
What the record can use here is a framework and its predictions. The paper has no result, and nothing in the record depends on it.
The nearest documents are the grokking theories. THEORY-069 holds that delayed generalization moves with training-set size, initialization scale and kernel alignment. THEORY-070 explains it as the lazy-to-rich transition. This paper uses grokking as a within-cell instance of its phase-transition shape and adds no mechanism to either account. Its width transition is a statement about how many contexts the learner has, and the error floor it predicts sits on that count, not on training time. THEORY-074 separates a well-defined Bayesian phase transition from a trajectory transition. The width transition belongs to neither: Theorem III.3 is a capacity statement about the number of cells, and the paper does not connect it to either kind.
On routing, THEORY-005 holds that dense feed-forward layers settle into an expert partition early in pre-training, which bears on the paper’s claim that structure and within-context prediction have different timescales. SOTA-148 balances mixture-of-experts load with a routing bias updated from load rather than from a gradient on the loss. That is a step in the direction this paper’s decoupling principle points, taken for a different reason (load balance, not structure), so it is a resonance and not support. On forgetting, LIT-237 measures a forgetting curve under evolution strategies. This paper offers a reading of such curves, structural under-resolution, and no measurement of one.
The predictions that could become evidence are the error floor below true width and the CS-operator width estimate, both on benchmarks with known width. Until someone runs them this is a seed, not a gap.
Unread — no NOTE.