Step-Video Team, StepFun (2025) — ARXIV-2502.10248
Key takeaways
- Compression doubles in every dimension, with no loss on its own benchmark. A dual-path VAE (pixel-unshuffle convolutions plus a channel-averaging shortcut) compresses 16×16×8. It reconstructs at SSIM 0.978, PSNR 39.37 and rFVD 3.61, against HunyuanVideo’s 8×8×4 at 0.971, 39.56 and 4.17. Cosmos, at the same 16×16×8 ratio, gets rFVD 40.33 (Table 15). Open-Sora 2.0’s Table 1 (LIT-634) measures the same VAE well below HunyuanVideo’s: LPIPS 0.082 against 0.046, PSNR 28.719 against 30.240. The latent is 64 channels against 16 (per LIT-634), so the information ratio is 2×, not 8×. Step-Video’s own numbers present it as a counterexample to CogVideoX’s finding that 16×16×8 is “extremely difficult”. Open-Sora’s measurement and the 4× larger channel count weaken that reading. Wan (LIT-619) also plots this VAE, listed at 64 latent channels, against its own, HunyuanVideo’s and others on PSNR and speed over 200 videos (Fig. 7). The values are not printed, so the plot neither confirms nor refutes Table 15.
- The model. A 30B DiT with 48 layers, full 3D attention and cross-attention to two text encoders (a CLIP and a bilingual LLM). It uses AdaLN-single, 3D RoPE and QK-norm, and trains with flow matching. The backbone is DiT’s (LIT-448), which the paper names as its base. It keeps DiT’s adaLN timestep modulation but drops the class label, shares the modulation across layers as AdaLN-single to cut its cost, and adds a cross-attention layer for text between self-attention and the FFN (§4.3).
- Video DPO works, modestly. DPO (LIT-169) is applied with fixed initial noise, and the DPO model scores 55 against 45 over the baseline on 300 prompts (§9.7). This is a preference score, with “no preference” worth 0.5 to each side. It is not a win rate, and the tie rate is not given. The objective (§4.4, Eq. 8) is Diffusion-DPO’s (LIT-774), “based on the DiffusionDPO method … with slight modifications, extending it to the Flow Matching framework”. It keeps the pair loss against a frozen reference and shares the initial noise and timestep across each pair. Eq. 8 is printed in DPO’s likelihood-ratio form, so the flow-matching version of the per-sample term is not shown. Diffusion-DPO’s defaults (β = 5000, lr 1e-8) converge too slowly, by the paper’s gradient argument (Eq. 9), and a smaller β with a higher learning rate fixes that. Flow-DPO (LIT-773) is a concrete case of the same direction: β = 500 and learning rate 5·10⁻⁶ with LoRA rank 64 for one epoch on a 1B video flow model, where a constant β beat the exact β(1 − t)² form. The β values are not on one scale, because Diffusion-DPO folds the number of timesteps into its β. Gains saturate on off-policy pairs, which motivates a reward model.
- Systems. It reaches 32% MFU with TP8 + SP + ZeRO-1, and pipeline parallelism was rejected for robustness. Hardware restarts run at 0.037 per 1k GPUs per day, against Llama 3.1’s 0.422 (Table 5).
- Where it ranks. HunyuanVideo (LIT-620) is its main open baseline. Against it on Movie Gen Bench it wins 615, ties 313, loses 361, and the three annotators on its own benchmark favour Step-Video too (e.g. 59-22-47; Table 7). Frame count and resolution are not matched. Against Movie Gen (LIT-626), on Movie Gen’s own benchmark, it wins 485, ties 315, loses 489, i.e. parity (Table 14). The paper’s reasons for not doing better include a third as many high-resolution pre-training videos (27.3M against 73.8M), less human-labelled SFT data, and Movie Gen’s 720P against its 540P. It loses to T2VTopA (Table 9).
Where the hedges are
Per DP-010:
- “State-of-the-art … compared with both open-source and commercial engines” (abstract) is contradicted by §9.3, which ranks “T2VTopA > Step-Video-T2V > T2VTopB”.
- “State-of-the-art” reconstruction is contradicted by Table 15, where HunyuanVideo’s PSNR is higher. An outside measurement (LIT-634, Table 1) puts the gap much wider: PSNR 28.719 against 30.240.
- Architecture comparisons are early and unquantified. DiT against MMDiT rests on “similar” early curves, and MMDiT was “not train[ed] … for an extended period”. Full 3D against factorized attention was run at 4B with no numbers.
- The DPO β and learning rate are unreported, and the effect is a 55/45 preference score on 300 prompts, with ties split and their rate not given.
- The sample counts disagree. §9.3 says 25.3M samples at 540P, and Table 6 totals 27.3M.
Standing in the anthology
Step-Video is the line’s evidence on how far a video latent can be compressed, and the one report that applies preference optimization to a video model. It is in this record for the VAE result and the DPO recipe, not for the “state-of-the-art” in its abstract.
Its limitations section (§§10–11) is unusually candid, and it names the same failures as Wan’s and Movie Gen’s: complex actions, rare compositions, physics. It attributes them to “inherent limitations of diffusion models”, which is asserted, not shown.
Read in full as NOTE-338.