Source
Sun et al. (2021), LIT-387 — ARXIV-2104.00680.
The claim
The classical order is detect, describe, match, and the paper’s word for it is sequentially. That ordering is the problem, not the algorithms filling it: everything downstream is conditioned on what the detector proposed, so the first stage sets the ceiling.
And the first stage fails somewhere specific — low-texture regions, “where feature detectors usually struggle to produce repeatable interest points”. Those are walls, floors, sky and road, not an unlucky corner case, and they are where reconstruction most needs the help. A matcher cannot recover a correspondence that was never proposed, so improving the matcher does not touch this failure at all.
So: establish dense matches at a coarse level, refine the good ones, and let attention rather than a cost volume do the work.
That recipe, and the argument against the detector, are LIT-387’s. Sun et al. proposed LoFTR as detector-free matching, credited the Transformer’s global receptive field for dense matches in low-texture areas, and reported it outperforming the prior state of the art by a large margin and ranking first on two public visual localization benchmarks among published methods.
The part that generalises past graphics
Descriptors become conditioned on both images. Self and cross attention mean a location’s representation depends on the image it is being matched to.
A detect-then-describe pipeline structurally cannot do this. Description runs per image, before any pair exists, so a descriptor has to be a summary useful against every possible partner — conservative by construction. That cost is invisible while the stage exists, because nothing in the pipeline can measure what a less conservative descriptor would have bought.
A stage computed before its consumer exists must be conservative, and the price is only legible once the stage is removed. That is the transferable claim, and it is not about vision.
Conditions
Dense matching costs more compute than sparse detect-and-match, and where the budget is tight — embedded, real-time on modest hardware — detecting first is still the right trade. Nothing in this record measures where that crossover falls.
The evidence is one paper for the recommendation itself, which would normally
make this unreplicated. It is filed converged on structural grounds
instead: SOTA-236 records systems that remove the matching stage entirely
and recover correspondence as an output, which is a step that only makes sense
if the detector was already gone. The later line presupposes this one.
The line this belongs to
Three steps, each removing the stage the previous step exposed as the limit:
- LIT-386 (SuperGlue) — learn the matcher instead of hand-designing the heuristics, and keep the detector.
- This — remove the detector; the ceiling it imposed was the real limit.
- LIT-385 and LIT-384 (SOTA-236) — remove matching as a stage; predict geometry directly and recover matches from the output.
Each removal was justified by a failure the earlier framing could not express, which is what makes this a line rather than a sequence of improvements.