Source
Bahmani et al. (2025), LIT-743 — Lyra, §3, §6 (Tables 1 and 2) and Appendix C.
The claim
Captured multi-view scenes are scarce and narrow; RealEstate10K and DL3DV are what scene-level reconstructors train on, and they generalize poorly outside them. A camera-controlled video diffusion model trained on internet video can render any image along any camera path. So generate the multi-view data instead: sample camera trajectories for each input image, let the frozen video model produce the views, and train the reconstructor to match them.
LIT-743 does this with GEN3C as teacher: 59,031 generated images, six trajectories of 121 frames each, 354,186 videos, and no captured multi-view data at all. The student is a 3DGS decoder that reads the video model’s latents, and its renderings are trained against the RGB decoder’s frames with MSE and LPIPS, plus a depth loss against ViPE video depth and an opacity penalty.
The evidence, and what each part can show
The arm that isolates the supervision source (Table 2), scored by PSNR / SSIM / LPIPS against the video model’s own renderings on out-of-distribution prompts:
| supervision | PSNR | SSIM | LPIPS |
|---|---|---|---|
| generated only (Lyra) | 24.77 | 0.837 | 0.224 |
| captured only (RealEstate10K + DL3DV) | 19.08 | 0.659 | 0.413 |
| generated + captured | 24.74 | 0.823 | 0.236 |
The paper reads the third row as confirming “that self-distillation is diverse and consistent enough to learn a reconstruction model”. That is the supportable half. The large gap to the second row is not evidence that generated supervision beats captured supervision: the test targets are the teacher’s frames, so a student trained on the teacher is being scored on its training distribution and a student trained on real scenes is being scored on how well it reproduces the teacher’s hallucinations. The captured-only model is never scored on RealEstate10K or DL3DV, the benchmarks where it would have the advantage.
The benchmark comparison (Table 1), single image to 3D on RealEstate10K, DL3DV and Tanks-and-Temples, scored against real held-out views: Lyra leads on every metric — RealEstate10K PSNR 21.79 against Bolt3D’s 21.54, DL3DV 20.09 against Wonderland’s 16.64, Tanks-and-Temples 19.24 against 15.90. This is the real-scene test, but it compares whole systems: the baselines differ in generator, decoder and training data at once, and their numbers are taken from their papers because “no source code is available”. It shows the recipe produces a strong system; it does not say how much of that is the supervision.
The authors’ own limit: “the scale and consistency of our generated scenes are bounded by the capacity of our camera-controlled video diffusion model”. A student trained only on a teacher’s frames inherits every 3D inconsistency in them; the depth loss exists because RGB supervision alone produced flat geometry.
What was already done before the source
Wonderland (LIT-755) already supervised a feed-forward 3D reconstructor with a
camera-controlled video model’s outputs. Its reconstructor reads the video
model’s latents, as Lyra’s does, and for the 20K videos it generated “the
decoded video frames from these latents provide supervision views”. So
using a video model’s renderings as training targets for a reconstructor
is not new with the source. What Wonderland did not do is drop captured
data. It trained 200K iterations on RealEstate10K, ACID and DL3DV first and
added the generated videos only to the 100K-iteration high-resolution
stage, mixed with the same captured sets. That is the opposite proportion
from this practice, which is why introduced_by stays with the source.
Wonderland’s ablation of that addition (Table A3) is the record’s only measurement, on real held-out views, of what generated supervision adds on top of captured supervision: PSNR 17.06 to 17.15 on RealEstate10K, 16.62 to 16.64 on DL3DV and 15.85 to 15.90 on Tanks-and-Temples, one run each. That is near zero, and it answers a different question from the one the promotion condition asks. It says the generated videos did not hurt a model already trained on captured data. It does not say they can replace it.
Conditions
- The student decodes the teacher’s latents. At inference Lyra still runs the video model and replaces only its RGB decoder. Whether generated views are as good a supervisor for a reconstructor that takes images directly is untested here; the pixel-space version of this student ran out of memory on 726 views per scene, which is why the design reads latents.
- The teacher has to be 3D-consistent first. The source chose GEN3C because it builds an explicit 3D cache to keep views consistent, and still had to refine its masks; a video model without that would supply inconsistent targets, and nothing here measures how much inconsistency the student tolerates. Fusing across the six trajectories is learned for this reason; without it PSNR falls to 17.73, the largest drop in Table 2.
- One teacher and one group, with no variance reported for any cell.
Known implementations
- Lyra (NVIDIA)