Yang et al., Qwen Team (2025) — ARXIV-2505.09388
Key takeaways
- Dense and MoE models from 0.6B to 235B in one family.
- The named innovation is unifying thinking and non-thinking modes in a single model, switchable by query or chat template — removing the need to keep a chat-optimised model and a dedicated reasoning model side by side, which was the standing arrangement.
- A thinking budget: the caller allocates inference compute per request,
trading latency against quality by task. This is the mechanism that appears
in the record a generation later as
reasoning_efforton Qwen3.8-27B (LIT-135), defaulting to its deepest setting. - Smaller models are built by leveraging knowledge from the flagship, which is presented as a substantial reduction in the compute needed to produce a competitive small model — the record’s tiny-model half (SOTA-123) argues the opposite route from scratch.
- Multilingual support from 29 to 119 languages. Apache 2.0 throughout.
Standing in the anthology
The generation the Qwen line in the record descends from, and the missing middle of it: LIT-136 is Qwen3-Next’s architecture, LIT-135 is Qwen3.8-27B’s model card, LIT-152 is the design report — and this is the report the whole series is named after. It is also what GSPO (LIT-180) is credited with improving.
Two things in it bear on practices the record holds. The thinking budget is the origin of the control LIT-135 documents, so the “over-thinks by default” observation has a lineage. And flagship-to-small distillation is a direct alternative to SOTA-123’s claim that tiny specialists should be pretrained from scratch on the target mixture — two roads to a small capable model, with no comparison between them in the record.
No practice of its own: a model report is adoption, not evidence
(ADR-017). It describes a recipe in full and isolates none of it, so what
it supplies is that somebody shipped these choices — which is a claim about
the field, and belongs in the consensus_note: of the practices whose
designs it carries. Its value here is as the dense baseline the Qwen hybrids are measured against, which is a role notes fill and practices do not.
Two 2026 papers use it as a reference point. The long-context ablations (LIT-801) pretrain a Qwen 3-style architecture on Olmo 3 data, with headwise QK norm, GQA and 36 layers. Extended with the same recipe as the rest of the pool, it scores 46.9 on HELMET at 32K against 52.4 for the Llama 3 architecture. Headwise QK norm is the variant the paper finds slightly worse than layerwise. Olmo Hybrid (LIT-799) uses Qwen3 8B as a dense row in its open-weight table: trained on 36T tokens against 6T, it is ahead on four of the five OlmoBaseEval averages and behind on GenQA.