Kirillov et al. (2023) — ARXIV-2304.02643
Key takeaways
- The data engine is the transferable part. Three stages — assisted-manual, semi-automatic, fully automatic — with the model retrained between them, so each stage’s labels train the model producing the next stage’s
- 1.1B masks over 11M images, the great majority from the automatic stage; each stage materially cheaper per label than the one before
- The model and the dataset are built together, which is the method; the mask count is the consequence
- Two preconditions the loop needs: the model must be useful before the dataset exists, and confidence must be informative enough to route work between model and human
- A promptable formulation is what makes the model useful at stage one — it can be steered rather than needing to be right unprompted
Standing in the anthology
The source of SOTA-186.
Read in full for #114; the reading is NOTE-018. The practice is confirmed.
This note’s standing has always been that the practice drawn from it is not
about segmentation but about the annotation loop — a data-pipeline claim,
under ADR-026 (which supersedes ADR-020 and carries that decision
forward). The reading supports that and adds the two preconditions,
which the practice does not state and a reader most needs.
It also names what the record is relying on and the paper does not test: that the loop generalises past segmentation. Masks are unusually cheap to verify, and a task needing expert judgement per label does not inherit the economics.
gen2seg (LIT-488) measured itself against this model on five datasets drawn from its evaluation suite, with 3.7M finetuning masks against 1.1B. At one prompt point it roughly matches SAM on COCO-Large (57.6 against 57.0 mIoU) and beats it on iShape’s fine structures (51.4 against 16.8), but loses heavily on COCO-Medium (38.8 against 59.5) and COCO-Small (8.5 against 56.9). Its generality comes from a generative prior rather than from the data engine.