Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah and Neel Nanda (2024) — ARXIV-2408.05147
Key takeaways
- A JumpReLU SAE encodes an activation x as f(x) = z ⊙ H(z − θ), with z = W_enc x + b_enc and a learned threshold θ_i > 0 for each of M ≫ n latents. It decodes as W_dec f + b_dec and trains on ‖x − x̂‖² + λ‖f(x)‖₀ (Eqs. 1–4). Unlike TopK, which keeps a fixed number of latents per token, the threshold lets the number of active latents vary by token. The gradient through θ comes from a straight-through estimator with a kernel-density bandwidth ε, set to 0.001 after sweeps (§2.2, footnote 5).
- The release covers every layer and sublayer of Gemma 2 2B and 9B and three layers of 27B, at widths 2¹⁴ to 2²⁰, each at several sparsities, trained on 4–16B tokens each (Table 1). It also includes a suite of transcoders, a “feature-splitting” ladder of widths at fixed λ, and SAEs trained on the instruction-tuned 9B. The build used over 20% of GPT-3’s training compute and about 20 PiB of stored activations (§1).
- Fidelity is measured as delta LM loss and FVU at a given mean L0 (§4.1). Residual-stream SAEs cost more LM loss than MLP or attention SAEs at similar FVU, because the residual stream is the bottleneck every later layer reads. Base-model SAEs reconstruct instruction-tuned activations almost as well as SAEs trained on them, by delta loss though less clearly by FVU (§4.5, Appendix C.4). Wider SAEs shift latent firing frequencies lower but keep a cluster of ultra-high-frequency latents (Fig. 5). The authors state that “there is no consensus on what constitutes a reliable metric for the quality of a sparse autoencoder” (§4).
Standing in the record
Filed on 2026-10-03 at the owner’s request, beside work on Fisher-spectrum block structure and model comparison by evidence that is being filed in parallel. The owner’s phrase for that line is “the object being thresholded”, and this paper is the case where the threshold is literally the model. A JumpReLU latent is kept or zeroed by a learned per-latent cut, and the training objective prices each active latent at a flat λ. That makes the SAE a sparse dictionary selected by a penalised count, the same shape as an information criterion that charges a fixed cost per retained component. The paper does not use that language and offers no principled way to set λ or the width M. Its evaluations are Pareto curves, and its open-problems list asks for exactly the measures a model-comparison reading would supply. NOTE-482 sets out the comparison; it is my reading, not the paper’s.
Its premise is the record’s existing account of features. Elhage et al.’s Toy Models of Superposition (LIT-323) is the hypothesis SAEs are built to undo: more features than dimensions, stored as near-orthogonal directions, recoverable when sparse. Engels et al. (LIT-322) found circular, irreducibly multi-dimensional features by clustering SAE dictionary directions. This report lists “how can SAEs efficiently capture the circular features of Engels et al.” as an open problem (§5), and calls truly non-linear features the SAE’s “dark matter”. Nanda et al. (LIT-345), with Lieberum as a co-author, is the earlier mechanistic reading of a network in a known basis, there a Fourier one, that SAEs try to recover without being told the basis.
The anthology holds the work that tests these SAEs. Kantamneni et al., “Are Sparse Autoencoders Useful?” (ANTH-LIT-568), probes with Gemma Scope’s latents and finds they add almost nothing over baselines on 113 datasets. Gao et al.’s scaling study of TopK SAEs (ANTH-LIT-571) is the architecture JumpReLU is set against here. And Chen et al. on dynamical versus Bayesian phase transitions in the toy model of superposition (ANTH-LIT-544) is where superposition meets singular learning theory’s free energy. That is the bridge from SAE features to the evidence-based model comparison the owner is collecting.
It carries anthology-candidate because SAEs are machine-learning
interpretability tooling, which anthology topics hold. The report carries
some practice advice (data sharding for SAE training, bfloat16 inference
being harmless), but none of it is why it is here.