Liu, Gong and Liu, University of Texas at Austin (2022) — ARXIV-2209.03003
Key takeaways
- The method is one regression. Sample t uniformly and set x_t = t·x1 + (1−t)·x0. Regress v(x_t, t) onto x1 − x0 (eq. 1, Alg. 1). §2.3.1 says the linear interpolation “should be recommended as a default choice”. This is the name SD3 (LIT-449) and SOTA-266 use.
- The theory holds for exact minimizers. Marginals are preserved under any interpolation (Thm. 3.3). Convex transport cost does not increase (Thm. 3.5). Reflow straightens paths at O(1/K) (Thm. 3.7). Probability-flow ODEs and DDIM are nonlinear rectified flows, and reflow does not straighten them (Prop. 3.11 for the equivalence; §2.3 and a 2-D toy in Fig. 4 for the non-straightening).
- At full simulation the straight path beats the VP probability-flow ODE. With the same DDPM++ architecture and RK45 sampling, FID is 2.58 for 1-rectified flow, against 3.93 for the VP ODE and 3.16 for sub-VP (Table 1a). The 2,000-step VP SDE scores 2.55. Training-budget parity is not stated. DDPM (LIT-036) enters the table twice. The paper calls the VP SDE equivalent to DDPM and the VP ODE the continuous-time limit of DDIM derived from it, so on the shared DDPM++ U-Net the straight path beats DDPM’s deterministic sampler (2.58 against 3.93) and ties its stochastic one only at 2,000 steps (2.55). Table 1b also quotes DDPM’s own 1,000-step FID of 3.21, from the literature on a different architecture.
- Few steps need reflow or distillation. At one Euler step, undistilled 1-rectified flow scores FID 378. Distilled with no reflow it scores 6.18. 2-rectified flow, after one reflow, scores 12.21, and 4.85 when distilled (Table 1a). Distillation does most of the work.
- Reflow costs quality at many steps. 2- and 3-rectified flow score FID 3.36 and 3.96 at full simulation, against 2.58 before reflow (§5.2).
Where the hedges are
Per DP-010:
- “High quality results even with a single Euler discretization step” (abstract) describes 2-rectified flow, after a second training pass. The base method’s undistilled one-step FID is 378, and 6.18 when distilled. The 4.85 headline also needs distillation with an LPIPS loss (App. A).
- The straightness theorems are about the reflowed coupling. The single-pass model, which is what everybody trains, is not straight (Fig. 10).
- Budget parity with the VP baselines is not stated. It is unclear whether they were retrained or taken from Song et al. Treat Table 1a as partly controlled.
- Buried and useful: Fig. 5’s last column runs VP with a linear α_t, which gives the same curve at uniform speed. This toy separates path speed from path curvature, the mechanism experiment SOTA-266 says nobody has run. It is 2-D and qualitative only.
Standing in the anthology
This is the origin of the name “rectified flow”, and one of the two concurrent origins of the straight path, with Flow Matching (LIT-630). Stochastic interpolants (LIT-644), three weeks later, share the simulation-free objective but use a trigonometric path and credit the linear one to this paper. SD3 is “Scaling Rectified Flow Transformers”, so this is the paper SD3 could not stand without.
What travelled from it is the single-pass objective with a straight path. The reflow procedure, which is what the title and abstract are about, did not become standard. Few-step video generation today comes from separate distillation. Anything citing this paper for “few-step generation” should say whether it means reflow or distillation. The paper’s own numbers (6.18 distilled without reflow, 4.85 with it) put most of the gain on distillation.
DMD (LIT-643) later sets its own one-step distillation against this paper’s best one-step number: 3.77 unconditional CIFAR-10 FID against 4.85 for distilled 2-rectified flow (DMD App. Table 6). The figures are copied across papers rather than rerun, so they place the two methods but do not test one against the other.
Five papers filed later take reflow up or test it.
- InstaFlow (LIT-796), by the same senior authors, extends the
reflow-then-distil pipeline to text-to-image. At matched budget on Stable
Diffusion 1.4, reflow first beats distilling directly, one-step FID-5k
31.0 against 40.9, the same ordering as this paper’s 4.85 against 6.18
with a wider gap. Its teacher is a diffusion model’s curved ODE, not a
rectified flow. It was listed among this note’s implementations until
2026-10-03; the relation is now held as its
extends, which says which part of this paper it builds on. - Lee et al. (LIT-791) extend the reflow stage itself. One round, trained with U-shaped timesteps, an LPIPS-Huber loss and EDM initialization, takes 2-rectified flow’s one-step CIFAR-10 FID from this paper’s 12.21 to 3.07 with no distillation stage. Those setup differences make it no test of the pipeline here.
- Shortcut Models (LIT-787) keeps this paper’s straight-path velocity regression as its d = 0 term and runs reflow as a matched baseline on DiT-B at 256 px. One reflow scores 23.2 and 44.8 FID at one step on CelebA-HQ and ImageNet, behind progressive distillation from the same teacher at 14.8 and 35.6.
- The two minibatch-OT papers set reflow against straightening by coupling. In 2-D, two and three rounds reach path energies of 0.069–0.149 and 0.055–0.129, against OT-CFM’s 0.018–0.087 in one simulation-free run (LIT-798). On ImageNet a reflowed model trained longer scores 5.55 and 13.02 adaptive FID, against BatchOT’s 4.68 and 12.37 at 32 and 64 px with no reflow (LIT-789).
Read in full as NOTE-337.