Source
Albalak et al. (2023), LIT-117 — ARXIV-2312.02406.
Not a recommendation, and not this paper’s
“Monitor domain coverage during training” states no threshold and no
response, which is the same gap as the retired SOTA-073 and SOTA-074: nothing a reader can do differently on reading it.
It is also in tension with its own source. Under ODM the sampling distribution is supposed to move away from uniform coverage — that is what the bandit is for. A domain being under-sampled is the method working, not a condition to detect, so “coverage” is not even the right quantity to watch.
What LIT-117 does report is the bandit itself: an Exp3 policy over the 22 Pile domains, rewarded by per-domain training loss, which reaches the next best method’s final perplexity in 19% fewer iterations at 1B parameters. It recommends no monitoring of any kind. The attribution came from a takeaway bullet in this record that did not describe the paper, and it is kept here because a retired practice keeps its provenance.
What the real concern would be
There is one, and it is worth stating because it is the reason somebody wrote this bullet: a bandit can collapse onto a few arms and starve a domain entirely, and a domain seen almost never is a capability the model will not have. That is a genuine failure mode of the method, and the guard is the exploration floor in the bandit’s policy rather than a dashboard.
So the useful practice is “set an exploration floor so no domain is starved”, which is a claim about the algorithm’s configuration. As written, this asks someone to watch a number that the method is deliberately moving.
The cluster is resolved as of #114. SOTA-103 survives, restated as what ODM actually does; SOTA-102 is superseded into it; SOTA-101 and this one are retired. Four practices, one method, and none of the four described it.