Wang et al. (2023) — ARXIV-2312.14132
Key takeaways
- The problem it refuses. Multi-view stereo in the wild first needs camera intrinsics and extrinsics — “tedious and cumbersome to obtain, yet they are mandatory to triangulate corresponding pixels in 3D space, which is the core of all best performing MVS algorithms.” DUSt3R operates “without prior information about camera calibration nor viewpoint poses.”
- Pointmap regression instead of triangulation. Pairwise reconstruction is cast as regressing pointmaps, “relaxing the hard constraints of usual projective camera models”. The projective model stops being a constraint the method must satisfy and becomes something the output happens to be consistent with.
- Monocular and binocular unify. One formulation covers both, which the classical framing cannot — a single view has no baseline to triangulate across.
- More than two images are handled by a global alignment step expressing all pairwise pointmaps in a common reference frame.
- The inversion, which is the reason to file it. From the 3D output one can “seamlessly recover pixel matches, relative and absolute camera”. Every quantity the classical pipeline requires before it can start is a by-product of this one after it finishes.
- Standard Transformer encoders and decoders, so pretrained models carry over.
Standing in the anthology
The first paper in this record on the geometry-estimation half of 3D vision. The anthology has held the rendering and reconstruction half for some time — LIT-064, LIT-086, LIT-108, LIT-109, feeding SOTA-205 — and all of it assumes camera poses somebody else supplied. #154 is the audit that found the gap: a reader arriving with a set of images and no poses found four papers that assume the thing they came to ask about.
Sources SOTA-236 together with LIT-384, which is the same claim made feed-forward and general two years later.
Also gives LIT-111 (OnePose) the lineage it lacked. That note’s recipe builds an SfM point cloud and matches against it with a generic network — machinery this record held only as a step inside somebody else’s method, never as a subject with papers of its own.