Key takeaways
- The design is an ablation, not a model. Starting from DiT — discrete time, score prediction, variance-preserving interpolant — the paper walks one variable at a time to SiT — continuous time, velocity prediction, linear interpolant — with the architecture, parameter count and GFLOPs held exactly fixed. That is what makes the attribution possible.
- What each step buys: continuous time, marginal. Score → velocity prediction, significant. Variance-preserving → linear or generalized-VP interpolant, significant. A tuned stochastic sampler on top, further gain.
- The mechanism offered for the interpolant result is transport cost. Path length falls when moving from the VP interpolant to GVP or linear, and the singularity the VP interpolant carries at one endpoint disappears.
- The diffusion coefficient is a sampler hyperparameter, not a training one. In score-based diffusion it is conventionally tied to the forward process; the paper observes it need not be, because it affects neither the velocity nor the score. So it can be tuned after training, with no retraining, and tuning it tightens control of the KL divergence between model and target.
- Learn the velocity and express the score from it; only one of the two needs estimating.
- SiT-XL reaches 2.06 FID at ImageNet 256×256 and 2.62 at 512×512, and beats DiT uniformly across model sizes and throughout training.
Standing in the anthology
It is the controlled experiment under the rectified-flow recommendation. LIT-449 takes the straight-line path to 8B and text-to-image and is the reason to believe the choice survives scale; this is the reason to believe the choice is the cause, because everything else was held fixed. The two are load-bearing in different ways and the practice (SOTA-266) needs both.
The backbone is LIT-448’s, unchanged. SiT takes DiT’s latent-patch transformer at the same model sizes, parameter counts and GFLOPs, and changes only the training formulation. It moves from DiT’s discrete-time, score-prediction, variance-preserving setup to continuous time, velocity prediction and a linear interpolant, plus a sampler whose diffusion coefficient is tuned after training. On that shared backbone SiT-XL reaches 2.06 FID at ImageNet 256 where DiT-XL/2 reported 2.27.
The diffusion-coefficient finding is separable, and it is not new here. The freedom is the stochastic-interpolants framework’s (LIT-645), which shares three of this paper’s authors and states it without an image metric; this paper is where it is measured. Every sampler practice the record holds — SOTA-203 on higher-order ODE solvers, SOTA-207 on when to take the deterministic sampler — treats the sampler as something you choose subject to what you trained. This says one of its parameters was never downstream of training at all, and that a free tuning pass exists which most practitioners are not taking. SOTA-265 records it.
What it does not establish is anything outside class-conditional ImageNet at 256 and 512, which is the limitation LIT-449 exists to remove — and which that paper names explicitly when it says the advantages so far had been “limited to class-conditional models”.
Three papers filed later quote SiT-XL/2 as the many-step reference on ImageNet-256 without re-running it. Inductive Moment Matching (LIT-775) lists it at 2.15 with guidance 1.5, against IMM’s 1.99 at eight guided steps, which is 16 evaluations against SiT’s 250 × 2. MeanFlow (LIT-784) lists it at 2.06, and reaches 2.20 at two steps after 1,000 epochs and 3.43 at one; its text calls SiT 2.15. STARFlow (LIT-785) sets its 2.40 against SiT’s 2.06, the nearest diffusion row to a latent normalizing flow.