flows-and-transport.
7 of 378 NOTE documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| NOTE-196 | Scaling Rectified Flow Transformers | A 61-way sweep over diffusion and rectified-flow formulations settles on a straight-line path with logit-normal timestep sampling — rectified flow with uniform timesteps does not win. Adds a resolution-dependent timestep shift, MMDiT’s per-modality weights with joint attention, and QK-RMSNorm for high-resolution stability. 8B, weights released. | Read |
| NOTE-197 v2 | SiT: Scalable Interpolant Transformers | Walks one variable at a time from DiT to SiT with the architecture, parameter count and GFLOPs held fixed: discrete to continuous time, score to velocity prediction, variance-preserving to linear interpolant, ODE to a tuned SDE sampler. Velocity and the linear interpolant carry most of the gain, and the sampler’s diffusion coefficient turns out to be choosable after training. | Read |
| NOTE-310 | FLUX.1 Kontext | The FLUX.1 architecture, and an editing model built by appending context latents to the token sequence. It is a system report: human preference against other systems, latency, one reconstruction table, and no ablations. Main text and appendix A read. | Read |
| NOTE-333 v2 | Wan | An open 14B text-to-video recipe whose evidence is mostly systems measurement: 2D context parallelism cuts communication overhead from over 10% to under 1%, and caching and 8-bit kernels give 1.62× and 1.27×. Its three modelling ablations run on 1.3B text-to-image. Two are read from training loss, and on FID one of them favours the rejected option. The applications in §5 are demonstrated almost entirely by figures. | Read |
| NOTE-334 | Movie Gen | At 5B, flow matching beats v-prediction diffusion on quality by 3.3 annotation σ, but on text alignment by only 1.9σ. Video-native captions beat captions stitched from three frames on alignment by 2.9σ. Every σ in the paper is rater variance, and each ablation arm was trained once. The Llama 3 block beats a DiT block that differs only in norm, activation and bias. The scaling claim rests on image-only runs covering less than a quarter of a decade of compute. | Read |
| NOTE-337 | Rectified Flow | Regressing a velocity onto x1 − x0 along the straight interpolation gives an ODE that preserves the marginals and does not increase any convex transport cost, and retraining on its own couplings (“reflow”) provably straightens it at O(1/K). On CIFAR-10 the single-pass model scores FID 2.58 against 3.93 for the VP probability-flow ODE at the same architecture, with training budget unstated. Most of the one-step result comes from distillation (378 → 6.18 with no reflow), and reflow adds 6.18 → 4.85. | Read |
| NOTE-340 v2 | Flow Matching | Regressing onto a per-sample conditional vector field trains a continuous normalizing flow without simulation, with the same gradient as the intractable marginal objective. With the U-Net, hyperparameters and epochs held fixed, the straight “OT” path beats the VP diffusion path under that objective at CIFAR-10 and ImageNet 32/64 with uniform timesteps (FID 6.35 against 8.06 on CIFAR-10). The objective’s own advantage over score matching is mixed, and the likelihood gains at ImageNet scale are 0.01 bpd. | Read |