Source
Lahoti et al. (2026), LIT-165 — ARXIV-2603.15569.
The framing is inference-first and the critique is aimed at its own family: many recent linear models trade quality and capability for algorithmic efficiency — failing on tasks such as state tracking — and their theoretically linear inference is often hardware-inefficient in practice. Linear is not automatically fast and not automatically enough.
Three changes, each read off the state-space view of linear models:
- A more expressive recurrence, derived from SSM discretisation.
- A complex-valued state update, which is what enables richer state tracking.
- A multi-input multi-output (MIMO) formulation, which improves quality without increasing decode latency.
What it measured
The three changes are proposed together as Mamba-3 in LIT-165, and every number here is that paper’s own measurement against its own baselines.
At 1.5B, +0.6 downstream accuracy over the next best model — named as Gated DeltaNet — and the MIMO variant +1.2 further, for +1.8 total. Across state-size experiments it matches Mamba-2’s perplexity at half the state size, which is the efficiency claim that matters for an architecture with a fixed memory budget.
What it changes about the record’s own summary
SOTA-132 interleaves linear-attention layers with global attention and treats Gated DeltaNet and KDA as the modules to build with. This is the only note in the state-space line that measures against Gated DeltaNet as the incumbent, and it is ahead at 1.5B. So “linear attention won” is not the right summary of the last two years: the SSM family kept moving in parallel, and which module a hybrid should use is open again.
The line: Mamba’s selectivity (LIT-161) → the duality and Mamba-2 (LIT-162) → gated delta rule (LIT-137) → channel-wise gating (LIT-133) → here.
Conditions, and why this is Proposed
One group, one paper, 1.5B, and nothing in the record ships it. The three
changes are also reported as a bundle plus one ablation (MIMO), so a reader
adopting this is adopting all three on one team’s word — which is what
promote_when: asks somebody to unpick.
Known implementations
- None in the record.