Source
Ma et al. (2024), LIT-380 — ARXIV-2402.17764.
The claim
The record’s quantization practices all act on a finished model — SOTA-185 compensates rounding error into the columns not yet quantized, SOTA-163 chooses the block-scaled format to round into. This is the other branch: constrain the weights during training and never hold a high-precision checkpoint at all.
BitNet b1.58 does it with ternary weights, {-1, 0, +1}, 8-bit activations, and
BitLinear in place of nn.Linear. At matched size and token budget it
reaches FP16 perplexity and end-task accuracy from 3B upward.
The branch enters the record with LIT-380, where Ma et al. trained BitNet b1.58 ternary from scratch against LLaMA at equal size and tokens and argued for fixing the format before the run rather than compressing after it. Its perplexity comparison is the evidence: 12.87 against 12.33 at 700M, 11.29 against 11.25 at 1.3B, 9.91 against 10.04 at 3B, which is where the “from 3B” qualifier comes from.
Why this is not just another compression ratio
Two arguments, and they are worth separating because only one is measured on hardware you can buy.
Measured. Weight memory and bandwidth fall with the format, which is ordinary and real: 2.22GB at 3B, a 3.55× reduction, and at 70B on two 80GB A100s the batch ceiling goes 16 → 176 and throughput 333 → 2977 tokens/s.
Argued. With ternary operands the matrix multiply reduces to integer addition and the multiplications disappear, which is where the energy claim comes from. That is a fact about arithmetic and a bet about silicon: it assumes kernels, and eventually hardware, built for the format. The paper says as much. On stock GPU matmul the memory saving survives and the arithmetic saving does not.
Conditions
Below 3B it does not hold, and the paper’s own table says so — 700M perplexity 12.87 against 12.33, 1.3B 11.29 against 11.25, 3B 9.91 against 10.04. Behind, level, ahead. This is a practice for models large enough to have been on the right side of that crossover.
It is not available retroactively. There is no conversion path here; the decision has to be made before the run, which makes it the most expensive commitment in the quantization family to get wrong. Where a model already exists, SOTA-185 and SOTA-163 are the practices that apply.
Proposed, not Active. The measurements are real and the branch is
unrepresented in this record, which is reason enough to file it; but one
group, one architecture, and nothing at the frontier ships this way, so the
record should not yet be telling anyone to plan a pretraining run around it.
Why it is worth watching rather than dismissing
LIT-186 reports post-training quantization damage rising with the number of training tokens — far enough that extra pretraining data becomes harmful for a model that will be quantized later.
SOTA-160 already tells a reader to treat the token budget and the quantization plan as one decision.
This practice is the other way out of that bind. If the format is fixed before training there is no compression step to be damaged, and the token budget stops trading against the serving precision. The two papers never cite each other; the tension is the record’s own to notice.