Russakovsky et al. (2014) — ARXIV-1409.0575
Key takeaways
- The task, the splits and the metric that “ImageNet accuracy” refers to:
1,000 object classes, roughly 1.2M training images, 50K validation, 100K
test, scored as top-1 and top-5 classification. This is
ImageNet-1K - The challenge is a slice of the database, not the database.
LIT-422has millions of images over tens of thousands of synsets; the 1,000 classes here were selected, and the selection is what makes the numbers comparable across years - It measures and publishes its own human baseline, which is the part most worth carrying: the widely repeated “human error on ImageNet” figure comes from a small trained-annotator study reported here, not from a population measurement. Comparisons of the form “model X now exceeds human performance” inherit that study’s scope
Standing in the anthology
Carries no practice, deliberately — ADR-032.
This is the instrument. LIT-422 is the source material, and the split
between them is why both are filed — see that note for the three-way
“ImageNet” ambiguity in this record and the two earlier cases of the same
shape.
What this record uses it for is not vision. It is the standard heavy
workload against which optimizer, compression and distributed-training
claims are made. Seventy-four documents say “ImageNet”; most of them mean
this, though the count cannot be split cleanly without reading each one. A
sample of what rests on it: NOTE-157’s SlowMo top-1 gains (69.94% → 73.24%),
NOTE-096’s 270–600× gradient compression “without loss of accuracy”,
NOTE-148’s 25.5% large-batch speedup on ViT-S/16, SOTA-221’s tuned
Adagrad/Adam/SGD comparison at 16K batch, and SOTA-165’s validation of
Distributed Shampoo.
The bound to carry into all of those is that ILSVRC is a classification
workload of a particular size and shape, and a result established on it is
established there. Some of the citing documents mark this themselves —
SOTA-222 notes its own experiments are CIFAR-10 while the claim is pitched
at ImageNet scale. The ones that do not are relying on the reader knowing it.
Unread — no NOTE.