Lipman et al., Meta AI (FAIR) and Weizmann Institute (2022) — ARXIV-2210.02747
Key takeaways
- The objective is simulation-free. Regressing onto a conditional vector field, defined per sample, has the same gradients as the intractable marginal flow-matching objective (Thms. 1–2, requiring only p_t(x) > 0). So a CNF trains the way a diffusion model does, without ODE solves in the loop.
- Diffusion is one choice of path. Every Gaussian path has a closed-form field (Thm. 3), with VE and VP diffusion as special cases. The “OT” path is a linear interpolation with a σ_min floor (eq. 20). It is defined on all of [0, 1], while VP must stop short of one end (App. E.1). This is the endpoint singularity SOTA-266 mentions.
- The path comparison is controlled. With the same ADM U-Net, hyperparameters and epochs (Table 1), FID on CIFAR-10 is 6.35 for FM-OT, 8.06 for FM-diffusion and 7.48 for DDPM. On ImageNet-64 it is 14.45, 16.88 and 17.36. Comparing FM-OT with FM-diffusion isolates the path. Comparing FM-diffusion with score matching isolates the objective. DDPM (LIT-036) is a baseline in the same table, on the same U-Net: FM-OT beats it on FID at all three sizes (6.35 against 7.48, 5.02 against 6.99, 14.45 against 17.36) with roughly half its adaptive solver evaluations (142 against 274 on CIFAR-10), while FM with the diffusion path loses to it on CIFAR-10 (8.06 against 7.48). DDPM may have had more iterations than the FM runs (see the budget hedge below).
- It needs fewer evaluations for the same error. FM-OT reaches a given ODE error in “roughly only 60% of the NFEs” (Fig. 7, ImageNet-32). The numbers are given only in the figure.
- Timesteps are uniform (eq. 5: t ~ U[0,1]). The straight path wins here without a logit-normal.
Where the hedges are
Per DP-010:
- The budget statements disagree. §6.1 contradicts itself in one sentence (“same number of training iterations, where baselines are allowed more iterations”). App. E fixes epochs for FM-OT, FM-Diffusion and Score Matching and says nothing about DDPM or ScoreFlow. The two statements conflict only for Score Matching. The OT-against-diffusion path comparison is matched under every reading. Either way the budget was equal to or favoured the baselines.
- “Consistently better likelihood” is 0.01 bpd over the next-best model at ImageNet-32 and 64, and a tie with DDPM at K = 1 on ImageNet-32 (3.62, Table 4).
- “OT” describes the conditional path only. §4 says “this by no means imply that the marginal VF is an optimal transport solution”. The learned trajectories are not shown to be straight.
- The controlled comparison stops at 64 px. The ImageNet-128 result is compared against GANs, with a different batch size and iteration count.
- σ_min is never given a value.
Standing in the anthology
This is one of two concurrent origins of the straight data-to-noise path. The other is Rectified Flow (LIT-636), about a month earlier in autumn 2022. Albergo and Vanden-Eijnden’s stochastic interpolants (LIT-644) appeared between the two and share the simulation-free regression objective, but not the straight path: its interpolant is trigonometric, the linear one appears only in a remark crediting Liu et al., and its related work describes this paper and Rectified Flow as the ones that “focus on straight interpolants”. Before this filing, 13 files in the record mentioned “flow matching” and none held the paper. Among them were SOTA-266, which recommends the path, and the Movie Gen, HunyuanVideo and Step-Video reports, which train with it.
Its bearing on SOTA-266 is specific. The practice said that rectified flow with uniform timesteps does not win. Table 1 here is a controlled, uniform-timestep, pixel-space comparison in which the straight path does win. The two results do not conflict, because SD3’s finding is about ranking against well-tuned ε-prediction in latent text-to-image at scale. They do show that the uniform version’s loss is a property of that setting and not a general fact.
Three later papers re-run or quote it. The minibatch-OT paper (LIT-798) could not reproduce Table 1’s CIFAR-10 FID of 6.35 from this paper’s text: the stated procedure gave 11.53, and it lists what the text leaves unspecified. Under its own recipe, which beats the original, this paper’s straight path scores 3.655 adaptive against the VP path’s 4.335, and 4.640 against 7.772 at 100 Euler steps, the same ordering as Table 1. Multisample Flow Matching (LIT-789), with Lipman among its authors, reruns the path comparison on ImageNet: the diffusion path scores 6.36 and 15.11 adaptive FID against the straight path’s 5.04 and 13.93 at 32 and 64 px. Both papers keep the straight conditional path and change only how noise and data are paired.
TarFlow (LIT-794) quotes this paper’s ImageNet-64 likelihood, 3.31 bits/dim, against its own 2.99, and an unconditional ImageNet-64 FID of 13.93 cited to this paper. This note records 14.45 for FM-OT. 13.93 is the straight-path figure in Multisample Flow Matching’s reproduction, which may be where TarFlow took it from; TarFlow does not say.
Read in full as NOTE-340.