Mildenhall et al. (2020) — ARXIV-2003.08934
Key takeaways
- A scene is a function, and the weights are the scene. A fully-connected network maps a continuous 5D coordinate (3D position plus 2D viewing direction) to volume density and view-dependent RGB. No voxels, no mesh, no convolution
- Differentiable volume rendering is what makes it trainable. Classical volume rendering accumulates samples along a camera ray; because that is differentiable, the only supervision required is a set of images with known camera poses
- Positional encoding is not optional. The paper is explicit that the basic implementation “does not converge to a sufficiently high-resolution representation” — mapping the input coordinate into a higher-dimensional space is what lets the MLP represent high-frequency scene content, and the ablation without it is oversmoothed
- Hierarchical sampling is the other half, allocating the MLP’s capacity toward space that actually contains visible content rather than sampling the ray uniformly
- Multiview consistency is enforced architecturally, not by a loss: density is a function of position alone, while colour is a function of position and direction. Eight 256-channel layers produce density and a feature vector; one more 128-channel layer takes that plus the ray direction and emits colour
Standing in the anthology
The trunk under the record’s one scene-representation practice, and the thing that practice exists to replace. SOTA-205 says to “replace a large coordinate network with a compact explicit structure and a small decoder” — and the large coordinate network it means is this one. Its four sources are all reactions to this paper: Instant-NGP (LIT-064), K-Planes (LIT-086), 3D Gaussian Splatting (LIT-108) and NeuS2 (LIT-109).
Carries no practice, deliberately. The record’s position on coordinate
networks is SOTA-205’s, which is a recommendation against this design;
filing a competing practice would be recommending both. Same shape as
LIT-428, where the record holds SOTA-167’s position on linear attention
and the trunk note carries none of its own.
Filed because the practice was unreadable without it. SOTA-205 recommends
replacing something the record could not point at, and two of its four
sources name this paper in their own notes. DP-007 — what everyone agrees
on has no author, and “the coordinate MLP” was the shared assumption the
whole cluster was arguing with.
Its positional encoding was measured from outside by LIT-551 (SIREN), which fits an image with a ReLU network using this paper’s encoding as a baseline: of the networks compared, only that one and SIREN reproduce the image, and only SIREN also reproduces its gradient and Laplacian. The comparison is of the input encoding, not of the volume-rendering pipeline, and SIREN describes the two papers as concurrent.
Read — NOTE-188.