Source
Dao et al. (2023), LIT-106 — ARXIV-2307.08691.
The same block boundary, from the mask’s side
SOTA-107 says the sequence should be a multiple of the block size; this says what to do when it cannot be. A mask expressed at token granularity still has to be applied blockwise, so a block that is partly masked is computed in full and then discarded — the kernel’s unit of work is the tile, not the token.
Rounding the mask out to block boundaries does not save that work. What it does is let the kernel skip blocks that are entirely masked, which is where the saving actually is: in causal attention roughly half the blocks are fully above the diagonal, and a block-aligned mask lets them be dropped rather than computed and zeroed.
Why this is rejected
The paper does not say it. padding, padded, pad, block boundar and
divisible all appear zero times in LIT-106.
What is written above is a correct account of how a blockwise kernel handles a mask, and it is an inference rather than a citation. Retired with
SOTA-107, which fails the same way from the other end: two practices about block alignment, both technically defensible, neither attributable.
The durable claim — express a structured mask so a kernel can skip fully masked blocks — is real, and needs a source that makes it before the record can recommend it.
Condition, and the cost that is easy to miss
Rounding a mask outward makes it less restrictive at the edges. That is fine for a causal mask, whose block-aligned form is still causal at block granularity and is handled exactly by the kernel’s own diagonal logic. It is not fine for an arbitrary mask where the difference between token- and block-alignment changes which positions attend to which — there the practice would silently alter the model.
So this is a recommendation about how to express a structured mask so the kernel can exploit it, not a licence to round any mask. Document-packing masks, the common case where it matters, are structured in exactly the way that makes it safe.
Like SOTA-107, this is a property of blocked implementations rather than
something LIT-106 states — which both bodies said before either was read
against the paper, and which is now the reason both are retired.