Boffi, Albergo and Vanden-Eijnden, Courant Institute (2024) — ARXIV-2406.07507. Read at v2 (3 Jun 2025), main text and Appendices A–D. v1 (11 Jun 2024) was titled “Flow Map Matching”; v2 added the subtitle.
Key takeaways
- The object is the two-time map (§3.2, Def. 3.4, Props. 3.5–3.6). X_{s,t} carries any ODE solution from time s to time t. It satisfies a Lagrangian equation, ∂t X{s,t}(x) = b_t(X_{s,t}(x)), and an Eulerian one, ∂s X{s,t} + b_s·∇X_{s,t} = 0. Its composition rule X_{t,τ}∘X_{s,t} = X_{s,τ} is the consistency property, stated over two times. A consistency model is the one-time special case that always maps to the data end (App. C.2, Eq. C.12).
- Two distillation losses, one minimizer, different behaviour (§3.3, Cors. 3.7–3.8). Lagrangian map distillation (LMD) penalizes ∂_t X − b(X). Eulerian map distillation (EMD) penalizes the Eulerian residual, which needs the map’s spatial Jacobian. EMD is the continuous-time limit of consistency distillation (App. C.2, Eq. C.18). Both control the squared 2-Wasserstein distance to the teacher (Props. 3.9–3.10). The LMD bound carries a Lipschitz-constant factor; the EMD bound does not, but involves ∇X.
- The Lagrangian loss wins by a wide margin (Table 1, Fig. 4B). On CIFAR-10 with a 5.53-FID teacher, LMD gets 7.13 FID at two steps and 6.04 at four, against EMD’s 48.32 and 44.35. Teacher-FID is 1.27 and 1.05 against 34.19 and 30.74. On CIFAR-10 and ImageNet-32, LMD’s loss falls an order of magnitude faster. The authors attribute EMD’s failure to the spatial gradient of a nearly singular map (§3.4).
- Direct training, with no teacher (§3.5, Prop. 3.11). The loss replaces the unknown velocity with the interpolant’s time derivative, evaluated at X_{s,t}(X_{t,s}(I_t)), plus an invertibility term. Learning the full one-step map directly was “challenging”. Training on a strip |t − s| ≤ 1/K and then distilling the K-step map into one step (§3.7, Lemma 3.13) is what worked.
- Few-step numbers without a teacher (Table 2, ImageNet-32, strip 0.25). FID 16.90, 14.48, 12.61 and 9.68 at 4, 6, 8 and 20 steps, against minibatch-OT flow matching’s 38.86, 22.08, 15.64 and 7.71.
- Consistency training is not gradient descent on a loss (§3.6, Prop. 3.12). Replacing the teacher velocity by the interpolant’s derivative does not give an equivalent loss, because the squared term is quadratic in it. Consistency models put a stop-gradient there. The flow map is then a critical point of the update, with no guarantee it is stable or attractive.
Where the hedges are
Per DP-010:
- “Sample quality comparable to flow matching while reducing generation time by a factor of 10–20” (abstract). The supporting number is CIFAR-10 LMD: 6.04 at four steps against a 5.53 teacher run with an adaptive fifth-order solver. The paper does not count the teacher’s evaluations. One-step image quality appears only as training curves (Fig. 4B), not in a table.
- “Consistency models do not benefit from multistep sampling” (§2). Consistency Models (LIT-093) and iCT (LIT-786) both report two-step sampling improving on one-step on ImageNet-64: 6.20 to 4.70 for consistency distillation, 3.25 to 2.77 for iCT-deep. What a one-time map cannot do is stop at an intermediate time; it can still re-noise and denoise again.
- The 2D figure and its text disagree. §4.1 says the one-step map learned over all of [0, 1]² “performs worse” than the four-step strip map. Fig. 3’s caption says it “performs as well as SI”, with KL 0.017 against SI’s 0.016 and the four-step map’s 0.008.
- The teachers were “trained with limited compute” (§4.2). The experiments are 32 × 32, one U-Net, four GPUs, and 1–2.5 × 10⁵ steps (Table 4). No consistency, trajectory or progressive-distillation model is run on the same setup.
- The ImageNet-32 baselines are quoted, not re-run (Table 2 caption). FMM edges ahead only below about 8 steps. At 20 steps minibatch OT is better.
- LMD is better in distribution and worse at keeping the teacher’s coupling (App. D, Table 3, Fig. 6). On the checkerboard its KL is lower than EMD’s, but it sends more points to a different checker than the teacher does.
Standing in the anthology
It extends the stochastic-interpolant framework (LIT-645), from the same group. The velocity it distils or learns is that framework’s b_t(x) = E[İ_t | I_t = x] (Prop. 3.2), and every loss is an expectation over the interpolant I_t. The paper adds a second time to the learned object.
On ImageNet-32 it quotes DDPM (LIT-036) and minibatch-OT flow matching (LIT-789) from the latter’s tables. At 20 steps DDPM scores 63.08 FID and BatchOT 7.71; at 4 steps, 362.37 and 38.86. Its own direct training lands between them, 9.68 and 16.90.
It is the theory the other two-time papers in this batch cite. MeanFlow (LIT-784) notes that the flow map is its average velocity times the interval. Shortcut Models (LIT-787) is the discrete, binary-step version of the same composition rule. Appendix C.4 writes progressive distillation (LIT-067) as matching one map step to two steps of the previous map.
On SOTA-206 it is a theoretical argument, with thin numbers behind it. A two-time map can be used at any step count, and §1 calls post-training step choice a central advantage. The image results run two to twenty steps.
Filed without a NOTE: the takeaways come from one full reading of v2’s main text and appendices. The proofs in Appendix B were followed but not checked line by line. Figs. 3–4 are read only where the text or captions give values.