Source
Rouhani et al. (2023), LIT-197 — ARXIV-2310.10537.
An MX block is a vector of k elements sharing one scale. The shipped formats use block size 32 and an E8M0 scale — eight bits of exponent, no mantissa — so the scale is a power of two and applying it is an exponent adjustment rather than a multiply. The element type is independent: MXFP8 (E4M3 or E5M2), MXFP6 (E2M3 or E3M2), MXFP4 (E2M1), MXINT8.
Why block scaling and not per-tensor. A single scale for a whole tensor has to cover its whole dynamic range, so a few large values cost every small one its precision. Thirty-two elements is a small enough neighbourhood that outliers are contained locally. That is the same problem massive activations and quantization outliers pose, met by shrinking the scope of the scale rather than by changing the model — which is why it composes with everything else instead of competing with it.
The format and the evidence are both LIT-197’s: the paper specifies the MX formats and recommends them over per-tensor scaling on the strength of results across more than two dozen benchmarks, summarised by width in the table below, including what it claims as the first sub-8-bit training to FP32 accuracy.
What each width can bear, which is the part to read before choosing one
| Width | Regime |
|---|---|
| 8-bit MX | Direct-cast inference on FP32 pretrained models, minimal loss, no calibration or fine-tuning |
| 6-bit MX | Close to FP32 after quantization-aware fine-tuning or a PTQ method |
| 6-bit MX | Training — weights, activations and gradients — to FP32 accuracy with no recipe change |
| 4-bit weights | A minor drop |
The 6-bit training result is the headline and the paper claims it as the first sub-8-bit instance. The clause that matters operationally is “without modifications to the training recipe”: the paper is explicit that user friction is a third axis alongside accuracy and efficiency, and that a format nobody can adopt without rewriting their recipe does not count as practical. That is stated as a result rather than an aside, and it should be read as one.
What “FP4” means in the record’s other notes
DeepSeek-V4 and Kimi K3 both say FP4 and MXFP4, and both mean this: a 4-bit element inside a 32-element block with a shared power-of-two scale, not a bare 4-bit float. Reading those reports without this practice makes their precision decisions look like implementation details rather than choices.
The decision this does not make for you
Which width goes where in the pipeline is a separate question, and SOTA-160 is the practice about it: quantization damage grows with how much data the model was trained on, so the pretraining budget and the quantization plan are coupled. This practice says what the format can bear in each position; that one says the position is not free to choose late.
A 4-bit serving instance
Kozyrev and Maiboroda (LIT-800) quantized every linear layer of a 27B Gated DeltaNet hybrid to NVFP4 W4A4 by calibration alone. NVFP4 is the finer-grained relative of the MX formats: 16-element blocks with an E4M3 scale rather than 32 with a power-of-two one. The inputs carry the residual stream’s usual outliers, a median-layer max/RMS of 48–368 depending on the projection. Activation quantization error is nonetheless 7.5–9.2% for every layer role, because each outlier sets the scale only for its own block of 16. Task accuracy stayed within seed noise of BF16, at a perplexity cost (+0.72 at 4K). It is one model and the authors’ own checkpoint, so it is an instance of the mechanism this practice names, at a width the table above lists only for weights, not a replication of the source’s ablations.
Known implementations
- DeepSeek-V4 — FP4 quantization-aware training on MoE expert weights and the indexer QK path, applied in post-training.
- Kimi K3 — MXFP4 quantization-aware post-training; the release is in MXFP4.