anthology-of-the-sota

SOTA-147: Compress the KV cache into one shared latent vector instead of sharing key and value heads

2 min read

Properties7
StatusActive
Version2
Consensusemerging
Tagsattention-techniques
SourceLIT-174 — DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Introduced byLIT-174 — DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Compared againstSOTA-109 — Prefer GQA to MQA or MHA

Graph View

The KV cache is what makes long-context serving expensive, and the field has two answers to it.

SOTA-109 is the first: share key and value heads across groups of query heads, so there are fewer of them to store. MLA is the second: compress keys and values into one low-rank latent vector, cache that, and project back up when attention runs. The cache stops scaling with head count at all.

The distinction worth holding is what each gives up. GQA gives up head diversity in the keys and values. MLA keeps every head distinct and gives up storing them uncompressed. Neither gives up exact attention over all positions — which is what separates both of them from the linear-attention line (SOTA-132, SOTA-135), where a fixed-size recurrent state gives up exactness itself.

The evidence

Against the same team’s dense 67B (LIT-174): 93.3% smaller KV cache, 5.76× maximum generation throughput, 42.5% lower training cost.

Then four generations of adoption rather than one result:

  • LIT-160 — DeepSeek-V3 at 671B.
  • LIT-139 — V4, with the sparse-attention indexer built on top of it.
  • LIT-131 — Kimi K3 gates it.
  • LIT-133 — Kimi Linear keeps MLA for the one layer in four that stays global, which is the layout SOTA-132 recommends.

That last one is the reason this practice was overdue: the record already recommends a layout defined in terms of MLA layers while never having said what an MLA layer is.

Why Active but only emerging

Active because the production evidence is unusually strong for an architecture choice — not one result but four models across two labs, at scales from 236B to 1.6T, each of which could have dropped it and did not.

emerging because two labs is not the field. Llama, Qwen and Mistral ship GQA, and no third group has adopted MLA. The record is recommending this slightly ahead of consensus and the second axis is where it says so (ADR-015).

What this does not say

It does not retire SOTA-109. GQA remains the right default for a model that is not paying MLA’s implementation cost, which is most models — the two are compared_against rather than one superseding the other.

Backlinks