Yang (2026) — ARXIV-2601.05732, read as NOTE-258.
Standing
A repair of SOTA-136’s branch, not a rival to it. SOTA-169’s promotion condition says in as many words that another paper proposing a fourth constraint would not move the trunk. This is not one: it keeps the doubly-stochastic constraint exactly as LIT-140 states it and changes how the matrix is built. So the exclusion does not apply — and the trunk stays where it is anyway, because nothing here isolates width or ships in a released model.
Its diagnosis is the third distinct one in this dispute, and it disagrees with the other two about where the fault lies. LIT-151 and LIT-181 both argue the doubly-stochastic set is the problem — bounded above but not below, or degenerate near the identity. This argues the set is fine and the approximation to it is not.
What it does not do is measure stream distinctness, which is the one measurement SOTA-136’s promotion condition asks for. That makes it the third document to look like it should settle the question and not — after LIT-139 and LIT-152.
Against unconstrained hyper-connections (LIT-141) it is a direct
measurement. HC, mHC and mHC-lite are reimplemented side by side in nanoGPT
with n = 4 streams, each initialized to reduce to a plain residual. HC’s
gradient norms are substantially larger than either constrained variant’s
(Figure 2, L model, FineWeb-Edu). In Table 1, at the 24-layer L size HC’s
validation loss (3.248 OpenWebText, 3.244 FineWeb-Edu) is slightly worse than
the plain residual’s (3.242, 3.240), where mHC-lite reaches 3.198 and 3.185;
at S and M, HC beats the residual and is within 0.03 of mHC-lite. mHC-lite
also trains at higher throughput than HC in the authors’ PyTorch
implementation. Single runs, about 1.3B tokens each.