Optimization (2) — optimizers, schedules, batch size: 001 · 002
Stability (0) — normalization, initialization, gradient handling.
Systems (1) — hardware, kernels, memory: 003
By status: In the anthology (2) · Attic (1) · Superseded (0)
What the status column means in this scheme — the words are luria’s, the meanings are this project’s.
| Status | Means | |
|---|---|---|
Active | In the anthology | Worth reading, whether or not a practice was drawn from it |
Rejected | Attic | Retired with a reason, and the reason is kept |
Superseded | Superseded | Later work covers this better, and the note names it |
| # | Title | Summary | Status |
|---|---|---|---|
| LIT-001 | Adam: A Method for Stochastic Optimization | Kingma et al. (2014), ARXIV-1412.6980. Adaptive moment estimation, still the default optimizer a decade later. | Active |
| LIT-002 | Hard Negative Mining in Contrastive Learning | ARXIV-2104.08892. Atticked: the technique is real, the benefit is not consistent enough to recommend. | Rejected — needs per-case tuning, no consistent cross-domain benefit |
| LIT-003 | Scaling Data-Parallel Training Past the Communication Wall | A lab technical report, never posted to arXiv and without a DOI. Its url: is its source, which is all the source group asks for. | Active |