Topics — The twenty-eight kinds of claim this anthology files against — the primary axis of both the practice registry and the reading list, and the same twenty-eight for each (ADR-026). The filing rule is the point: take a domain topic when the claim is ABOUT the domain; a claim merely discovered in one still takes its kind. The axis organizes the record, it does not bound it: content the list cannot place is evidence the list is short (ADR-059).
Vision and graphics — neural rendering, reconstruction, perception, visual foundation models.
35 of 449 SOTA documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| SOTA-205 v2 | Replace a large coordinate network with a compact explicit structure and a small decoder | Four independent groups, three structures. Where a field is queried pointwise and the signal is spatially sparse, the capacity belongs in an addressable structure that training optimises directly, not in a network evaluated per point. | Active |
| SOTA-236 | Predict scene geometry directly instead of solving for cameras first and triangulating | Wang et al. (2023), LIT-385 — ARXIV-2312.14132; Wang et al. (2025), LIT-384 — ARXIV-2503.11651. Given uncalibrated, unposed images, regress the 3D structure and let camera parameters and pixel matches fall out of it, rather than estimating calibration and pose first so that triangulation becomes possible. The quantities the classical pipeline needs as inputs are by-products of this one. | Active |
| SOTA-237 | Match densely without a keypoint detector, because the detector fails where matching is hardest | Sun et al. (2021), LIT-387 — ARXIV-2104.00680. Establish dense matches coarse-to-fine instead of detecting keypoints, describing them and matching the descriptors. The detector sets the pipeline’s ceiling, and it fails in low-texture regions — walls, floors, road — where it cannot emit repeatable points, so no downstream matcher can recover a correspondence that was never proposed. | Active |
| SOTA-250 | Pretrain by predicting representations of masked regions, not their pixels | Assran et al. (2023), LIT-216 — ARXIV-2301.08243. Predict the representations of masked target blocks from a context block, with a learned target-encoder, instead of reconstructing pixels. No hand-crafted augmentations, and a ViT-H/14 reaches strong downstream performance in under 1200 GPU-hours — faster than a ViT-S/16 trained with iBOT. | Active |
| SOTA-251 | Train at low resolution and raise it only during the decay phase of a warmup-stable-decay schedule | Assran et al. (2025), LIT-215 — ARXIV-2506.09985. Spend warmup and the constant phase on short, low-resolution clips; raise resolution and clip length only in the final decay. Up to 8x less pretraining compute than training at full resolution throughout, for the same end state. | Proposed |
| SOTA-252 | Make the full-resolution path affordable instead of upsampling, when the shortcut is what breaks correctness | Abdal et al. (2023), LIT-113 — ARXIV-2311.17857. Prior 3D GANs render small and upsample in 2D because volume rendering is too slow at training resolution — but a 2D upsampler is multi-view inconsistent by construction. Efficient Gaussian rendering makes native 512x512 affordable, and the inconsistency leaves with the upsampler rather than being mitigated. | Active |
| SOTA-253 | Add a new instance by reconstructing it, not by training on it | Sun et al. (2022), LIT-111 — ARXIV-2205.12257. Split what the system knows about a specific object from what it has learned in general. Build the object’s structure once by reconstruction; match against it with a network that never saw the object or its category. Adding an object is then a scan, not a training run. | Active |
| SOTA-299 | Finetune the whole generative model, encoder and decoder, for a dense perception task instead of attaching a head to a backbone | Khangaonkar and Pirsiavash (2025), LIT-488 — the freshly-initialized mask head is the part that cannot generalize, because it has only ever seen the finetuning categories. Keep every parameter generatively pretrained, encode the target as an image, and a model finetuned on furniture and cars segments people, x-rays and paintings. The same backbone under a conventional head scores 1.4–2.4 mIoU. | Proposed |
| SOTA-331 | Pass low-dimensional coordinate inputs through sinusoids of sampled frequencies before an MLP, and tune the frequency scale rather than the distribution | Tancik et al. (2020), LIT-550, with NeRF (LIT-435) — an MLP on raw coordinates never fits high frequencies. Map the input to [cos 2πBv, sin 2πBv] with B ~ N(0, σ²) and tune σ on held-out data: too small blurs, too large aliases, and the distribution’s shape does not matter. Gaussian features beat no mapping and log-spaced positional encoding on all seven image, shape, CT, MRI and view-synthesis tasks tested. | Active |
| SOTA-332 | When an implicit representation will be supervised through its derivatives, use sine activations with the SIREN initialization rather than a ReLU network | Sitzmann et al. (2020), LIT-551 — a sine network’s derivative is another sine network, so gradients and Laplacians of the fit are well behaved where a ReLU network’s second derivative is zero. Initialize hidden weights U(±√(6/n)) and scale the first layer by ω₀ = 30 (tune it to the signal), or deep sine networks do not train. It fits images, video, SDFs and PDE solutions from derivative supervision. | Proposed |
| SOTA-333 v4 | For autoregressive generation of continuous sequences, train with an independent noise level per token and condition the rollout on slightly noised history | Chen et al. (2024), LIT-554 — train a causal model to denoise tokens that each carry an independent noise level. At rollout, treat the generated history as slightly noisy, so accumulated errors look like training noise. The same model then samples autoregressively, plans with horizon-wide guidance, and keeps near-future tokens cleaner than far ones. Long-rollout stability is shown on video qualitatively, with a small RNN. Planning and robot gains are measured. | Proposed |
| SOTA-335 | When generated content must move continuously, as in video or animation, build the generator alias-free: treat features as continuous signals and low-pass filter around every nonlinearity | Karras et al. (2021), LIT-559 — fine detail in GAN output sticks to pixel coordinates because the generator aliases. Apply each nonlinearity at 2× temporary resolution and low-pass filter back, start from Fourier features rather than a learned constant, remove per-pixel noise, and handle boundaries properly. On FFHQ-U, translation equivariance goes to 63 dB (and rotation to 40 dB for the -R variant) at StyleGAN2’s FID. | Proposed |
| SOTA-336 | Train high-resolution GANs at a fixed topology with output skips in the generator and a residual discriminator, not by progressive growing | Karras et al. (2019), LIT-560 — progressive growing makes each resolution briefly the output, so it learns output-level detail and leaves features stuck to pixel positions. A fixed generator that sums upsampled RGB outputs from every resolution, with a residual discriminator, keeps the coarse-to-fine emphasis without changing topology. FFHQ FID 4.34 → 3.31, and the phase artifacts are gone. | Proposed |
| SOTA-338 v2 | Train the GAN discriminator on frozen multi-scale pretrained features with fixed random channel and scale mixing | Sauer et al. (2021), LIT-562 — project real and generated images through a frozen EfficientNet, mix channels and scales with fixed random convolutions, and train one small discriminator per scale. It reaches StyleGAN2’s best LSUN-Church FID after 1.1M images instead of 88M. The speed-up is measured in FID, and at equal FID on FFHQ a CLIP-space distance and human raters prefer StyleGAN2 (LIT-563). | Proposed |
| SOTA-339 | For unpaired image-to-image translation of appearance, constrain both mappings with a cycle-consistency loss | Zhu et al. (2017), LIT-564 — learn X → Y and Y → X together, and penalize ‖F(G(x)) − x‖₁ and ‖G(F(y)) − y‖₁ alongside the two adversarial losses. With GAN losses alone the mapping may ignore its input. With the cycle it keeps content and changes appearance. It is scoped to colour and texture changes, and the authors report failure on geometric ones. | Proposed |
| SOTA-341 | Condition an image generator on each training image’s original size instead of discarding or upsampling small images | Podell et al. (2023), LIT-566 — embed each training image’s original height and width, like the timestep, and give it to the model. Then keep the small images that a minimum-resolution filter would drop (39% of SDXL’s pretraining data below 256²) without teaching the model their blur. At inference, set the size you want. On class-conditional ImageNet 512²: FID-5k 43.84 discarding, 39.76 keeping unconditioned, 36.53 keeping with size conditioning. | Proposed |
| SOTA-358 v3 | Drop the domain inductive bias once pre-training data is large enough, and keep it when it is not | Active | |
| SOTA-359 v2 | Supervise vision from the caption, and match image to caption rather than predicting its words | Active | |
| SOTA-383 | Fix the sample count before comparing FID values, and settle a close comparison with an unbiased estimator rather than a tighter error bar | Bińkowski et al. (2018), LIT-615 — the FID estimator is biased, the bias depends on the distribution being measured, and no unbiased estimator exists. So two FID numbers are comparable only at equal n, and a small standard deviation is not evidence of a converged estimate: at d=2048 and 50,000 samples their construction reversed a true ordering in 100 trials out of 100, with standard deviations of 0.2 and 0.5. | Active |
| SOTA-386 v4 | Show a video diffusion model images before and alongside video | Proposed | |
| SOTA-388 v2 | Train a plain ViT on ImageNet-1k with average pooling, fixed 2D sin-cos positions, a 1024 batch and light augmentation | Beyer et al. (2022), LIT-628 — five changes, none novel, take ViT-S/16 on ImageNet-1k from 66.8% to 76.5% at 90 epochs and 80.0% at 300. Global average pooling, fixed 2D sin-cos position embeddings, batch 1024, RandAugment at level 10 and Mixup at p = 0.2. 6h30 on a TPUv3-8 for the 90-epoch run. | Active |
| SOTA-390 v3 | In a video diffusion transformer, attend over space and time jointly rather than factorizing, and budget for the cost | Proposed | |
| SOTA-393 | Freeze the image model’s spatial layers when adding time if checkpoint compatibility matters; do not expect a quality gain from it | Girdhar et al. (2023), LIT-635, and Guo et al. (2023), LIT-633. Freezing a text-to-image model’s spatial layers and training only temporal ones keeps the model a drop-in for checkpoints fine-tuned from the same base. Emu Video’s one controlled test finds freezing costs nothing and gains little (55.0 / 58.1 against fine-tuning). The reason to freeze is compatibility, not quality. | Proposed |
| SOTA-394 v3 | When distilling a causal few-step video generator, take the teacher bidirectional, not causal | Yin et al. (2024), LIT-631. A block-causal 4-step student distilled from a bidirectional teacher beats one distilled from a causal teacher, at the same initialization, data and step count (Table 4). The causal teacher is also the weaker model before distillation, so the result shows the arrangement works and does not isolate why. | Proposed |
| SOTA-395 | Post-train an autoregressive video generator on its own rollouts, and score the whole rollout | Huang et al. (2025), LIT-629. Training a causal video generator on context it generated itself, with a distribution-level loss over the whole clip, beats teacher-forced and diffusion-forced training under every objective tried (Table 2). It holds quality as the number of autoregressive steps grows, where they degrade. Evidence is VBench on 5s clips, and the exposure-bias mechanism is inferred, not measured. | Proposed |
| SOTA-396 | When a latent video model reuses an image autoencoder or upsampler, fine-tune them on video so they see time | Blattmann et al. (2023), LIT-621. An image autoencoder decodes each frame independently and flickers. Fine-tuning its decoder on video, with a video discriminator, cuts reconstruction FVD on three datasets: 390.88 to 32.94, 35.82 to 18.66, and 73.78 to 25.55. Making the upsampler temporal cuts FVD from 165.98 to 45.39. Reconstruction FID worsens on one dataset and improves on two. | Proposed |
| SOTA-397 v2 | To condition an unconditionally trained diffusion model on known frames, use reconstruction guidance, not replacement | Ho, Salimans et al. (2022), LIT-627. Extending a video autoregressively from a model trained without conditioning, reconstruction guidance (steering each denoising step toward consistency with the known frames) gives FVD 136 against 451 for replacing the known frames’ noisy latents (Table 6). The model and guidance weight are held fixed. | Proposed |
| SOTA-398 v2 | Pretrain a vision transformer on packed, aspect-preserved images at sampled resolutions, not on fixed square crops | Dehghani et al. (2023), LIT-657. Do not resize or crop images to one square resolution. Keep each image’s aspect ratio and sample its resolution per example, favouring small side lengths. Pack the patches into fixed-length sequences with per-example attention masks. At matched compute this beats square fixed-resolution ViT pretraining, and matches the best ViT with 4x less compute. Most of the gain is throughput. | Proposed |
| SOTA-400 | Append a few register tokens to a ViT so it does not commandeer background patches for scratch space | Active | |
| SOTA-401 | Anchor the patch Gram matrix to an early teacher when training long, because dense features decay while global metrics improve | Proposed | |
| SOTA-414 | To make an image diffusion model a dense geometric predictor, fine-tune it end to end as a one-step model at t = T with a task loss | Martin Garcia et al. (WACV 2025), LIT-687. Fix the timestep at T, feed zeros as the noise, decode the prediction through the frozen VAE, and train on the task’s own loss: affine-invariant L1 for depth, angular error for normals. From 20K iterations on 74K synthetic images, the one-step model beats Marigold’s 50-step, 10-member ensemble on depth AbsRel on all five zero-shot sets, and on normals by 2.6–3.0° of mean error. Starting from plain Stable Diffusion is nearly as good. | Proposed |
| SOTA-416 v2 | When sampling a diffusion model in few steps, start at t = T: use trailing, not leading, timestep spacing | Martin Garcia et al. (WACV 2025), LIT-687. DDIM’s “leading” spacing never visits the final timestep. So at one step, a pure-noise input is labelled with a timestep that claims an almost clean sample. With the same weights and only the spacing changed, Marigold’s single-step depth goes from noise to NYUv2 AbsRel 5.7. The mismatch vanishes as the step count approaches T, which is why it goes unnoticed at 50 steps. | Active |
| SOTA-429 | Drop half the image patches when training a CLIP-style model, spend the saving on more pairs and a larger batch, and unmask only for a short final tune | Li et al. (2022), LIT-706 — FLIP. Remove 50% of image patches and run the ViT on the rest, with CLIP’s loss and nothing else. The saving buys a 2× larger batch at the same memory and 2× the samples per hour; ViT-L/16 on LAION-400M reaches its unmasked reproduction’s accuracy more than 3× faster. The accuracy comes from what the saving buys: at equal batch, masking is parity. | Proposed |
| SOTA-434 | In a vision backbone, carry position with a relative bias inside the attention window and do not add an absolute position embedding on top | Liu et al. (2021), LIT-723, Table 4. One architecture, one recipe, three tasks. A relative position bias inside the attention window is best or tied-best on all three. An absolute position embedding is worth +0.4 ImageNet top-1 and -0.2 COCO box AP and -0.6 ADE20K mIoU, and adding it on top of the relative bias costs 2.1 mIoU while changing ImageNet nothing. The recommendation is the sign, not the size: this is the one design choice in the table that a classification benchmark scores backwards. | Proposed |
| SOTA-440 | Supervise a feed-forward 3D reconstructor with a frozen camera-controlled video model’s renderings instead of captured multi-view data | Bahmani et al. (2025), LIT-743 — Lyra trains a 3D Gaussian Splatting decoder only on a frozen video model’s RGB decodings along six camera paths per image, with no captured multi-view data. In its one controlled arm, training on real RealEstate10K + DL3DV instead scores PSNR 19.08 against 24.77, and adding the real data to the generated data gives 24.74, no better. Both are scored against the teacher’s own renderings on its own prompts, which is the distribution the generated supervision was drawn from — so the arm shows the generated data is sufficient there, not that it beats captured data on real scenes. | Proposed |