Source
Zhu et al. (2024), LIT-141 — Hyper-Connections; then LIT-140 (mHC), LIT-151 (oHC) and LIT-181 (spectral-sphere), each proposing a different constraint; and LIT-152, a third-party design study running mHC in its own harness.
Replace the single residual stream with n parallel streams (n = 2 in most published runs) and learn the mixing between them, so the network can tune connection strength across depth and exchange information laterally. The motivation is the seesaw between vanishing gradients and representation collapse that fixed residual variants cannot escape.
The original learned the mixing freely, and the three papers since all agree that free mixing is the part that breaks: it discards the identity mapping a residual guarantees. What they agree on is the shape of this practice — widen, and constrain the mixing somehow. What they disagree about is the constraint.
The trunk, and why it is filed separately from its branches
- SOTA-136 — constrain to the doubly-stochastic manifold (mHC).
Proposedandcontested: two groups attacked the constraint within a month, by different arguments and with opposite remedies.
- SOTA-137 — mix freely (the original). Superseded by the above.
Filing the branches and not the trunk left the record in an odd position: the disputed part of the design had a node and the agreed part did not. That is backwards, and it is why the record could recommend a layer ratio defined in terms of a structure it had never stated.
The disagreement is genuinely about the constraint, not about the width. LIT-151 argues the doubly-stochastic set is bounded above but not below, so mixing can only shrink what distinguishes the streams; LIT-181 argues the same set is degenerate near the identity and cannot subtract at all. Different arguments, same verdict, opposite remedies — SO(n) rotation against a spectral sphere. Neither disputes that widening helps.
Conditions, and why this is Proposed rather than Active
The record has one production deployment (DeepSeek-V4, at 1.6T, using the mHC constraint) and four research papers. Nothing isolates what the width buys independently of which constraint was chosen, so the trunk’s evidence is four papers that each measured a different bundle and agreed on the sign.
This is also, deliberately, the practice that ADR-015 declined to file — on
the grounds that “converting a hedge into an assertion is an editorial act”.
Proposed is the hedge, so filing it here converts nothing: it gives the
agreement a node that can carry a promotion condition and accumulate a
consensus reading, which prose in a note cannot do.
Known implementations
- DeepSeek-V4 at 1.6T, via the mHC constraint.
- Against, by silence: Llama, Qwen and Mistral ship a single residual stream.