Topics — The twenty-eight kinds of claim this anthology files against — the primary axis of both the practice registry and the reading list, and the same twenty-eight for each (ADR-026). The filing rule is the point: take a domain topic when the claim is ABOUT the domain; a claim merely discovered in one still takes its kind. The axis organizes the record, it does not bound it: content the list cannot place is evidence the list is short (ADR-059).
Generative modeling — diffusion, samplers, text-to-image, conditioning and control.
61 of 449 SOTA documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| SOTA-157 v3 | Train the language model as a masked diffusion model rather than autoregressively | Nie et al. (2025), LIT-217 — hold the paradigm fixed (pretrain, then SFT) and swap only the factorization: a forward masking process and a reverse process predicting masked tokens, optimizing a likelihood lower bound. Competitive with LLaMA3 8B on in-context learning; past GPT-4o on reversal poem completion. Filed Proposed — one group, one model, 8B. | Proposed |
| SOTA-187 v5 | Train the generative model in a learned compressed latent, not at full resolution | Active | |
| SOTA-188 v5 | Parametrize the network so its prediction target has unit variance at every noise level, and sample training noise from a log-normal | Active | |
| SOTA-195 v2 | Predict v rather than the noise when the model will be evaluated at low signal-to-noise | Active | |
| SOTA-202 v2 | Clamp the prediction to the training range at every step when sampling from a model’s own output | Active | |
| SOTA-203 v3 | Sample a diffusion model with a higher-order ODE solver on the weights you already trained | Lu et al. (2022), LIT-076 — the diffusion ODE is semi-linear, so solve the linear part exactly and approximate only the neural integral. 10-20 function evaluations, no retraining. Rests on LIT-038, which established that the sampler is not fixed by the training objective. | Active |
| SOTA-204 v2 | Anneal a discretisation from coarse to fine over training rather than fixing it | Song et al. (2023), LIT-093. Where a training loss approximates a continuous target through a step count, that count is a bias/variance dial: few steps give a biased but low-variance target early, many steps a faithful but noisy one later. | Proposed |
| SOTA-206 v3 | Keep a multi-step sampling option in a few-step generative model | Song et al. (2023), LIT-093. A one-step model that cannot spend more compute has no quality dial: whatever it produces is what you get. Consistency models trade compute for quality at inference without retraining, and that property is worth preserving by design. | Active |
| SOTA-207 | Use the deterministic sampler when the noise input has to mean something | Song et al. (2020), LIT-038. A stochastic sampler injects fresh noise at every step, so nothing about the starting point survives to the output. Setting the family’s stochasticity to zero makes the initial noise a latent code you can interpolate in and invert to. | Active |
| SOTA-232 | Convert an autoregressive model into a diffusion model by continual pretraining rather than training one from scratch | Gong et al. (2024), LIT-381 — ARXIV-2410.17891. Continual pretraining on under 200B tokens turns an existing autoregressive model into a diffusion language model — GPT2 and LLaMA across 127M-7B — competitive with the AR original. The reason to want one is infilling without prompt re-ordering, not perplexity. | Proposed |
| SOTA-254 v2 | Train the language model as a masked diffusion model when the corpus is fixed and the compute is not | Prabhudesai et al. (2025), LIT-442 — masked diffusion keeps extracting signal from a repeated corpus for roughly 512 epochs where autoregressive training stops at 32. Below a critical compute threshold, given in closed form from the unique-token count, autoregressive training is better and by a wide margin; above it, diffusion is better. The recommendation is the condition, not the objective. | Proposed |
| SOTA-262 v2 | Give each modality its own weights and let the streams attend jointly | Esser et al. (2024), LIT-449 — MMDiT gives the text and image streams separate projections and MLPs, then runs attention over the concatenated sequence so information flows both ways. It beats UViT and DiT at matched budget. Liang et al. (2024), LIT-483 — the same shape reached independently for autoregressive multi-modal LLMs, matching a dense baseline at 55.8% of the FLOPs, and ablated component by component: the feed-forward carries most of the benefit, the attention projections less, the layer norms none. | Active |
| SOTA-263 v4 | Shift the timestep schedule when the resolution changes, because more pixels need more noise | Hoogeboom et al. (2023), LIT-660, and Chen (2023), LIT-659. A timestep is not a fixed amount of corruption. Destroying the signal in an image with more pixels takes more noise, so a schedule set at one resolution under-corrupts at a higher one. Shifting log-SNR down with resolution takes ImageNet 256 FID from 7.65 to 3.76 in pixel space. The direction is measured. The magnitude is tuned in every source. | Active |
| SOTA-264 v2 | Fix the noise schedule’s endpoints against the bound and choose its shape to minimise the loss estimator’s variance | Kingma et al. (2021), LIT-446 — in continuous time the bound depends on the schedule only through its endpoints, so the shape is free. Spend it: optimise the endpoints against the VLB and the shape against the variance of the loss estimator. Same bound, faster optimisation, and a low-discrepancy sampler for the time variable cuts the variance further. | Proposed |
| SOTA-265 v3 | Tune the stochastic sampler’s diffusion coefficient after training; it is not fixed by the forward process | Albergo, Boffi and Vanden-Eijnden (2023), LIT-645, state it and Ma et al. (2024), LIT-447, measure it — score-based diffusion conventionally takes the reverse SDE’s diffusion coefficient from the forward process, and presents the two as intrinsically tied. They are not: the coefficient affects neither the velocity nor the score, only the integration. So it is a sampler hyperparameter, tunable on a frozen model, and tuning it tightens the KL divergence to the target. | Active |
| SOTA-266 v5 | Connect data and noise on a straight line, and sample the training timesteps from a logit-normal rather than uniformly | Esser et al. (2024), LIT-449, and Ma et al. (2024), LIT-447 — the straight-line path between data and noise beats the curved variance-preserving one at fixed architecture and compute, and the advantage is largest at few sampling steps. Movie Gen (LIT-626) confirms it for video at 5B. In SD3’s latent text-to-image sweep, the timestep distribution decides the ranking: uniform does not beat tuned ε-prediction and logit-normal does. | Active |
| SOTA-289 v2 | When the sampling budget is small, prefer uniform-state discrete diffusion with consistency distillation — masked diffusion cannot revise what it has already emitted | Proposed | |
| SOTA-301 | Apply the objective gradient before the denoiser, not after it, when guiding diffusion toward a task objective | Zhang et al. (2026), LIT-490 — the usual recipe adds −η∇f after the denoising step, which walks the sample off the geometry the model learned with nothing left to pull it back. Apply the gradient to the noisy iterate first and let the denoiser follow: it acts as an approximate projection, and the sampler becomes an inexact projected-gradient method. Same cost, one line, and in trajectory planning it is the difference between a plan that looks good and one that survives execution. | Proposed |
| SOTA-302 v2 | Steer a distilled generator by modulating its input noise, not by fine-tuning its weights | Eyring et al. (2025), LIT-491 — train a LoRA hypernetwork to predict an improved initial noise for a frozen step-distilled generator. GenEval on SANA-Sprint goes 0.70 → 0.75 for 0.1 s of added latency, recovering about half of what 30-second test-time optimization buys. Reward fine-tuning the same model instead takes it 0.73 → 0.62: the anchoring KL term is intractable in weight space and tractable in noise space. | Proposed |
| SOTA-306 v2 | Factorize the codebook: look up in a low-dimensional normalized space, embed in a high-dimensional one, and report utilization alongside reconstruction quality | Yu et al. (2021), LIT-500 — project the encoder output down to a low-dimensional lookup space, ℓ₂-normalize it and the codebook, match there, and project the winner back up to a wide embedding. Codebook usage goes 4% → 95% and FID 3.68 → 1.50 at unchanged throughput; removing the normalization alone drops usage to 2%. Confirmed independently by LIT-497 and LIT-494, and in all three the curve turns back up, so there is an optimum to find rather than a direction to follow. | Active |
| SOTA-307 v5 | Report generative FID as an error bar over several training seeds, and treat any gap below about 2% of the mean as inconclusive | Dufour, Efros and Pérez (2026), LIT-501 — on several hundred SiT networks, retraining the same recipe moves FID 3.2× more than resampling from a fixed model, and the coefficient of variation holds at 1–2% across four model sizes and every compute budget to 2M steps. Ten times the sampling budget shrinks the small term by √10 and leaves the large one untouched. A lucky training seed is worth up to 2× the compute. | Active |
| SOTA-333 v4 | For autoregressive generation of continuous sequences, train with an independent noise level per token and condition the rollout on slightly noised history | Chen et al. (2024), LIT-554 — train a causal model to denoise tokens that each carry an independent noise level. At rollout, treat the generated history as slightly noisy, so accumulated errors look like training noise. The same model then samples autoregressively, plans with horizon-wide guidance, and keeps near-future tokens cleaner than far ones. Long-rollout stability is shown on video qualitatively, with a small RNN. Planning and robot gains are measured. | Proposed |
| SOTA-334 | In a style-modulated generator, demodulate the convolution weights instead of instance-normalizing the activations | Karras et al. (2019), LIT-560 — AdaIN’s instance normalization discards relative feature magnitudes, so the generator smuggles scale past it with a localized spike, the “water droplet” in every StyleGAN image. Scale each output channel’s modulated weights by 1/√(Σ w’² + ε) instead. It gives the same expected unit variance and style mixing still works. The artifact disappears and FID is unchanged (4.40 → 4.39). | Proposed |
| SOTA-335 | When generated content must move continuously, as in video or animation, build the generator alias-free: treat features as continuous signals and low-pass filter around every nonlinearity | Karras et al. (2021), LIT-559 — fine detail in GAN output sticks to pixel coordinates because the generator aliases. Apply each nonlinearity at 2× temporary resolution and low-pass filter back, start from Fourier features rather than a learned constant, remove per-pixel noise, and handle boundaries properly. On FFHQ-U, translation equivariance goes to 63 dB (and rotation to 40 dB for the -R variant) at StyleGAN2’s FID. | Proposed |
| SOTA-336 | Train high-resolution GANs at a fixed topology with output skips in the generator and a residual discriminator, not by progressive growing | Karras et al. (2019), LIT-560 — progressive growing makes each resolution briefly the output, so it learns output-level detail and leaves features stuck to pixel positions. A fixed generator that sums upsampled RGB outputs from every resolution, with a residual discriminator, keeps the coarse-to-fine emphasis without changing topology. FFHQ FID 4.34 → 3.31, and the phase artifacts are gone. | Proposed |
| SOTA-337 v4 | When an ImageNet classifier is anywhere in the pipeline — training the generator or steering its sampling — confirm FID gains with a Fréchet distance in a non-ImageNet feature space | Kynkäänniemi et al. (2022), LIT-563 — FID’s features are nearly ImageNet class logits, so matching ImageNet-class statistics lowers it without improving images. Resampling a fixed StyleGAN2’s outputs cuts FFHQ FID 5.30 → 1.78, while CLIP-space FD moves 2.76 → 2.64. A model whose discriminator uses ImageNet features exploited this by accident, and the authors suspect data filters and samplers can too. Report a CLIP or SwAV Fréchet distance beside FID whenever that is the case. Classifier guidance (LIT-699) is the sampling-loop instance and the least checked: it maximises an ImageNet classifier’s confidence on every step, and its paper denies producing adversarial examples in one clause without running a test. | Active |
| SOTA-338 v2 | Train the GAN discriminator on frozen multi-scale pretrained features with fixed random channel and scale mixing | Sauer et al. (2021), LIT-562 — project real and generated images through a frozen EfficientNet, mix channels and scales with fixed random convolutions, and train one small discriminator per scale. It reaches StyleGAN2’s best LSUN-Church FID after 1.1M images instead of 88M. The speed-up is measured in FID, and at equal FID on FFHQ a CLIP-space distance and human raters prefer StyleGAN2 (LIT-563). | Proposed |
| SOTA-339 | For unpaired image-to-image translation of appearance, constrain both mappings with a cycle-consistency loss | Zhu et al. (2017), LIT-564 — learn X → Y and Y → X together, and penalize ‖F(G(x)) − x‖₁ and ‖G(F(y)) − y‖₁ alongside the two adversarial losses. With GAN losses alone the mapping may ignore its input. With the cycle it keeps content and changes appearance. It is scoped to colour and texture changes, and the authors report failure on geometric ones. | Proposed |
| SOTA-341 | Condition an image generator on each training image’s original size instead of discarding or upsampling small images | Podell et al. (2023), LIT-566 — embed each training image’s original height and width, like the timestep, and give it to the model. Then keep the small images that a minimum-resolution filter would drop (39% of SDXL’s pretraining data below 256²) without teaching the model their blur. At inference, set the size you want. On class-conditional ImageNet 512²: FID-5k 43.84 discarding, 39.76 keeping unconditioned, 36.53 keeping with size conditioning. | Proposed |
| SOTA-346 | Retune the training timestep shift for each autoencoder latent space, and never rank latent spaces for generation under one shared schedule | Black Forest Labs (2025), LIT-572 — four autoencoders, 30 timestep configurations each. The training shift alone moves FID by 61–86%. An RAE latent that wins when both are tuned loses to a tuned FLUX.2 latent when left unshifted. When you swap the autoencoder under a diffusion or flow model, sweep the shift again, using a logit-normal training distribution. When you compare autoencoders, compare each at its own optimum. | Proposed |
| SOTA-352 | When a structure predictor is made generative, add a regression model’s predictions to the training data so disordered regions are learned as disorder rather than invented structure | Abramson et al. (2024), LIT-584 — a diffusion head generates plausible compact structure even where the protein has none. AlphaFold- Multimer, a regression model, renders the same regions as extended loops. Mixing its predictions into training teaches the generative model that convention, and “greatly reduced” hallucination. Remaining hallucinations are flagged by low confidence but do not look disordered. | Proposed |
| SOTA-383 | Fix the sample count before comparing FID values, and settle a close comparison with an unbiased estimator rather than a tighter error bar | Bińkowski et al. (2018), LIT-615 — the FID estimator is biased, the bias depends on the distribution being measured, and no unbiased estimator exists. So two FID numbers are comparable only at equal n, and a small standard deviation is not evidence of a converged estimate: at d=2048 and 50,000 samples their construction reversed a true ordering in 100 trials out of 100, with standard deviations of 0.2 and 0.5. | Active |
| SOTA-386 v4 | Show a video diffusion model images before and alongside video | Proposed | |
| SOTA-389 v3 | Caption training video with a model that watches the video, not with captions of its frames | Proposed | |
| SOTA-390 v3 | In a video diffusion transformer, attend over space and time jointly rather than factorizing, and budget for the cost | Proposed | |
| SOTA-392 v3 | To get a one-step sampler from a rectified flow, distil it; treat reflow as an optional extra pass, and keep the pre-reflow model for many steps | Liu, Gong and Liu (2022), LIT-636. On CIFAR-10, one-step FID is 378 from the base model, 6.18 after distillation alone, 12.21 after one reflow, and 4.85 after reflow and distillation (Table 1a). Distillation does most of the work. Reflow adds a further gain at the cost of a second training pass, and worsens many-step quality from 2.58 to 3.36. InstaFlow (LIT-796) contests it at text-to-image scale, starting from a diffusion model rather than a rectified flow. | Proposed |
| SOTA-393 | Freeze the image model’s spatial layers when adding time if checkpoint compatibility matters; do not expect a quality gain from it | Girdhar et al. (2023), LIT-635, and Guo et al. (2023), LIT-633. Freezing a text-to-image model’s spatial layers and training only temporal ones keeps the model a drop-in for checkpoints fine-tuned from the same base. Emu Video’s one controlled test finds freezing costs nothing and gains little (55.0 / 58.1 against fine-tuning). The reason to freeze is compatibility, not quality. | Proposed |
| SOTA-394 v3 | When distilling a causal few-step video generator, take the teacher bidirectional, not causal | Yin et al. (2024), LIT-631. A block-causal 4-step student distilled from a bidirectional teacher beats one distilled from a causal teacher, at the same initialization, data and step count (Table 4). The causal teacher is also the weaker model before distillation, so the result shows the arrangement works and does not isolate why. | Proposed |
| SOTA-395 | Post-train an autoregressive video generator on its own rollouts, and score the whole rollout | Huang et al. (2025), LIT-629. Training a causal video generator on context it generated itself, with a distribution-level loss over the whole clip, beats teacher-forced and diffusion-forced training under every objective tried (Table 2). It holds quality as the number of autoregressive steps grows, where they degrade. Evidence is VBench on 5s clips, and the exposure-bias mechanism is inferred, not measured. | Proposed |
| SOTA-396 | When a latent video model reuses an image autoencoder or upsampler, fine-tune them on video so they see time | Blattmann et al. (2023), LIT-621. An image autoencoder decodes each frame independently and flickers. Fine-tuning its decoder on video, with a video discriminator, cuts reconstruction FVD on three datasets: 390.88 to 32.94, 35.82 to 18.66, and 73.78 to 25.55. Making the upsampler temporal cuts FVD from 165.98 to 45.39. Reconstruction FID worsens on one dataset and improves on two. | Proposed |
| SOTA-397 v2 | To condition an unconditionally trained diffusion model on known frames, use reconstruction guidance, not replacement | Ho, Salimans et al. (2022), LIT-627. Extending a video autoregressively from a model trained without conditioning, reconstruction guidance (steering each denoising step toward consistency with the known frames) gives FVD 136 against 451 for replacing the known frames’ noisy latents (Table 6). The model and guidance weight are held fixed. | Proposed |
| SOTA-410 v2 | For guided sampling, use a second-order multistep solver on the data-prediction parameterization, not a higher-order solver on the noise prediction | Lu et al. (2022), LIT-676. SOTA-203’s higher-order solver inverts under guidance: at scale 8.0 and 10 evaluations, FID is 13.04 for first-order DDIM against 114.62 at order 2 and 164.74 at order 3. Sample instead with a second-order multistep solver parameterized on the data prediction — which is also what makes SOTA-202’s per-step clamping available at all, since you cannot clamp x̂₀ if the solver’s state is ε̂. 20 evaluations then beat DDIM at 250. | Active |
| SOTA-412 | When the target is likelihood, learn a per-dimension noise schedule conditioned on a learned latent; a scalar or unconditioned one buys nothing | Sahoo et al. (NeurIPS 2024), LIT-677. For density estimation with a pixel-space diffusion model, replace the scalar learned schedule with a per-dimension one conditioned on a small discrete latent that an encoder infers from the image. Keep the endpoints fixed. VDM’s bound is reached in 5× fewer steps on CIFAR-10 and improved at equal steps. Each component alone does nothing: a scalar input-conditioned schedule and a time-only multivariate schedule both match VDM. | Proposed |
| SOTA-416 v2 | When sampling a diffusion model in few steps, start at t = T: use trailing, not leading, timestep spacing | Martin Garcia et al. (WACV 2025), LIT-687. DDIM’s “leading” spacing never visits the final timestep. So at one step, a pure-noise input is labelled with a timestep that claims an almost clean sample. With the same weights and only the spacing changed, Marigold’s single-step depth goes from noise to NYUv2 AbsRel 5.7. The mismatch vanishes as the step count approaches T, which is why it goes unnoticed at 50 steps. | Active |
| SOTA-422 | Rescale the noise schedule so the last timestep is pure noise, and train with v-prediction so the model can learn there | Lin et al. (WACV 2024), LIT-689, with Emu Video’s comparison, LIT-635. Common VP schedules leave signal at t = T. Stable Diffusion’s leaves √ᾱ_T = 0.068. The model learns to keep the leaked channel mean, and pure-noise inference then limits brightness. Rescale √ᾱ_t so it reaches 0 at T. Then train with v-prediction, because at zero SNR ε-prediction is trivial and v_T = x₀. Sample in the x₀ form, because converting v to ε at zero SNR divides by zero. | Proposed |
| SOTA-423 | When classifier-free guidance over-exposes a latent diffusion model, rescale the guided prediction toward the conditional prediction’s standard deviation | Lin et al. (WACV 2024), LIT-689. Compute x_cfg as usual, rescale it by std(x_pos)/std(x_cfg), and blend with weight φ back toward x_cfg. The source uses φ ≈ 0.7 at w = 7.5. It is the latent-space counterpart of Imagen’s dynamic thresholding, which needs a known pixel range. The evidence is qualitative, and part of the effect may simply be lower effective guidance. | Proposed |
| SOTA-424 v4 | Train one network for both conditional and unconditional scores by dropping the condition on 10% of examples, then pick the guidance weight by which metric you are willing to lose | Ho and Salimans (2022), LIT-693. Replace the conditioning with a null token on a fraction of training examples — 0.1 is the measured best, with 0.2 and 0.5 worse — so one network learns both scores, then extrapolate away from the unconditional estimate at sampling. The weight is not a quality dial: on ImageNet 64×64, w from 0.1 to 4.0 takes FID from 1.55 to 26.22 and IS from 66.11 to 260.2. Choose it by which of those you are prepared to sacrifice, and never compare two models at different weights. | Active |
| SOTA-425 v2 | Report precision and recall beside FID whenever the generator has a fidelity-diversity knob, because FID’s best value sits in the interior of that trade | Dhariwal and Nichol (2021), LIT-699. FID mixes fidelity and diversity, so on any knob that trades one for the other its optimum is interior and a single FID cannot say which side of the trade you are on. Their own Table 5 is the demonstration: ADM-G beats BigGAN-deep on FID at ImageNet 128, 256 and 512 while losing precision at all three (0.78/0.86, 0.82/0.87, 0.87/0.88) and winning recall by 0.24, 0.24 and 0.13. Report precision and recall, state the knob’s value, and never read a single FID as “better samples”. | Active |
| SOTA-428 | Choose the EMA length after training: store two power-function averages at snapshots and reconstruct any length by least squares | Karras et al. (2023), LIT-714 — EDM2. Track two power-function averages of the weights (σ_rel 0.05 and 0.10), save both every few thousand steps, and synthesize any EMA length after the run from a small linear solve. Then choose it per configuration, per guidance weight and per metric, because on ImageNet-512 the best length moves with all three — at guidance 1.4, FID wants 2% and FD_DINOv2 wants 14%. | Proposed |
| SOTA-431 | Make every layer of a diffusion denoiser preserve activation magnitude, and force each weight vector back to fixed norm after every step | Karras et al. (2023), LIT-714 — EDM2. Normalize each output channel’s weight vector on use, re-normalize the stored weights after every step, scale every fixed operation to preserve unit magnitude, and drop the group norms. On the ADM U-Net at ImageNet-512 this takes FID from 6.96 to 2.56 at equal compute, with on-use weight normalization alone worth 6.96 → 3.75. One architecture family; untested on transformers. | Proposed |
| SOTA-432 | Maintain two power-function weight averages during training and snapshot them, so the averaging length is a post-hoc sweep instead of a guess you have to make before the run | Karras et al. (2023), LIT-720. Keep two power-function averages of the weights during training (σ_rel 0.05 and 0.10) and store both in each periodic snapshot; afterwards, reconstruct the average for any averaging length by a least-squares fit over the stored profiles, with error falling as O(1/n⁴) in the snapshot count. The averaging length stops being a pre-training commitment. Worth it because that commitment is expensive in a way that does not show up in its own column: with the length pinned at 13%, the learning-rate decay moves FID by up to 72% across t_ref ∈ [30k, 160k]; swept post hoc, the same bracket is within 10% of the optimum. | Superseded — by SOTA-428; A duplicate of SOTA-428, which was filed from LIT-714 hours earlier and is the same recommendation from the same paper. SOTA-428 is also the better document: it carries the architecture, learning-rate, guidance and metric dependences, where this one had only the first two. The Active/unreplicated pairing this note argued for is a judgement SOTA-428 declined, at Proposed, and that is the one that stands. |
| SOTA-433 | Guide a diffusion model with a smaller, less-trained copy of itself rather than an unconditional model, and make sure the two models are degraded in the same way | Karras et al. (2024), LIT-721. Replace classifier-free guidance’s unconditional reference with a degraded copy of the same conditional model — smaller and under-trained, same task, same conditioning, same data. The quality gain survives and the diversity loss does not: ImageNet-512 FID 2.56 → 1.34 (EDM2-S), 1.25 (XXL), 1.01 at ImageNet-64, and unconditional EDM2-S 11.67 → 3.86, which CFG cannot do at all. The binding condition is compatibility: degrade the two models differently and the method stops working entirely. | Proposed |
| SOTA-440 | Supervise a feed-forward 3D reconstructor with a frozen camera-controlled video model’s renderings instead of captured multi-view data | Bahmani et al. (2025), LIT-743 — Lyra trains a 3D Gaussian Splatting decoder only on a frozen video model’s RGB decodings along six camera paths per image, with no captured multi-view data. In its one controlled arm, training on real RealEstate10K + DL3DV instead scores PSNR 19.08 against 24.77, and adding the real data to the generated data gives 24.74, no better. Both are scored against the teacher’s own renderings on its own prompts, which is the distribution the generated supervision was drawn from — so the arm shows the generated data is sufficient there, not that it beats captured data on real scenes. | Proposed |
| SOTA-442 | To RL-tune a flow-matching generator, roll it out as a same-marginal SDE built from its own velocity, and anchor each step with the closed-form KL to the reference | Liu et al. (2025), LIT-779. A flow model’s ODE sampler is not a stochastic policy. Swapping it for an SDE with the same marginals, built from the trained velocity alone, makes each step an isotropic Gaussian, so GRPO’s ratio and the KL to the reference model are both closed form. On SD3.5-M, GenEval goes 0.63 → 0.95 either way. Without the KL term, DrawBench aesthetic falls to 4.93 from a base of 5.39, against 5.25 with it (Table 2). Quality is judged by reward models only, one run each. | Proposed |
| SOTA-443 | Down-weight large residuals in consistency and MeanFlow training: use a Pseudo-Huber loss or an adaptive inverse-error weight, not plain squared L2 | Song and Dhariwal (2023), LIT-786, and Geng et al. (2025), LIT-784. When the regression target is the model’s own output under stop-gradient, squared L2 trains worse than a loss that limits the pull of large errors. iCT’s Pseudo-Huber beats squared L2 on CIFAR-10 (Fig. 2, curves). MeanFlow’s adaptive weight 1/(‖Δ‖² + c)^p takes one-step ImageNet-256 FID from 79.75 at p = 0 to 61.06 at p = 1 (Table 1e). Two groups, small settings, one run per arm. | Proposed |
| SOTA-444 | When a flow will be sampled with a coarse ODE solver, pair noise and data within each minibatch by exact optimal transport | Pooladian et al. (2023), LIT-789, and Tong et al. (2023), LIT-798. Re-pair each minibatch’s noise and data by exact discrete OT before the flow-matching regression. The marginals stay exact, the learned paths straighten, and on ImageNet-64 reaching FID 20 with Euler takes 12 steps instead of 29, at equal likelihood and 4% more time per iteration. It lowers the step count of an ODE sampler. It does not make one: four-step FID stays near 39 on ImageNet-32. Unconditional, 64 px and below. | Proposed |
| SOTA-445 | When the base text-to-image diffusion model is better than the models that generated the preference pairs, train on the pairs with Diffusion-DPO rather than fine-tuning on the preferred images | Wallace et al. (2023), LIT-774, §5.5 and Supp. S6. Pick-a-Pic’s images were generated by SDXL-beta and Dreamlike, and SDXL-1.0 is better than both. Supervised fine-tuning on the preferred images helped SD1.5 (55.5% win rate over its base) and made SDXL worse at every learning rate tried. Diffusion-DPO on the same pairs improved SDXL, preferred to the untuned base 70.0% of the time on PartiPrompts. The SDXL SFT arm has no number, there is one run per setting, and no one else has repeated it. | Proposed |
| SOTA-446 | In consistency training, take the target from the current weights under stop-gradient, not from an EMA teacher | Song and Dhariwal (2023), LIT-786, §3.2. Consistency Models trained the student against an EMA of itself. iCT shows that on a point-mass data distribution the limiting objective with any teacher other than the student carries no information about the data (Prop. 1), and that setting the teacher’s EMA rate to zero improves CIFAR-10 FID with both LPIPS and squared L2 (Fig. 2a, curves). Keep an EMA of the weights for evaluation. Do not use it for the target. | Proposed |
| SOTA-447 | When distilling a large image or video model by continuous-time consistency, add a small DMD distribution-matching term | Zheng et al. (2025), LIT-783. Pure sCM distillation of 2B–14B text-to-image and text-to-video teachers loses small text, texture and temporal coherence. Adding DMD’s loss at weight 0.01, on the student’s own 1–4-step rollouts with a fake-score network updated 5–10 times per step, fixes it in samples. Four-step Wan2.1-1.3B VBench is 84.43 against a re-implemented DMD2’s 84.56. sCM’s own Fig. 7 (LIT-790) is why the consistency term stays primary: distribution matching alone loses recall. | Proposed |
| SOTA-448 | Without a pretrained teacher, train a one-step generator as a MeanFlow average-velocity model rather than by consistency training or a shortcut model | Geng et al. (2025), LIT-784. Train a network for the average velocity over an interval, regressed onto v − (t − r)·du/dt under stop-gradient, with du/dt from a Jacobian-vector product. From scratch on ImageNet-256 with DiT-XL/2, one-step FID is 3.43 after 240 epochs, against 10.60 for a shortcut model and 34.24 for iCT as Inductive Moment Matching re-ran it (Table 2). Those rows are copied, at other budgets. No arm compares distilling a teacher. | Proposed |
| SOTA-449 | When reward-tuning a classifier-free-guided diffusion model over several rounds of sampling, train the guided prediction at the fixed sampling guidance weight, not the conditional prediction alone | Black et al. (2023), LIT-776, App. E.1 and Fig. 10. Reward fine-tuning should not train the unconditional branch, since the reward depends on the prompt. Training the conditional prediction alone made performance deteriorate after the first round of fine-tuning, which the authors attribute to the guidance weight becoming miscalibrated. Fixing the weight (5 in their runs) and training through the guided prediction used for sampling fixed it. One curve per arm, on reward-weighted regression and JPEG compressibility. | Proposed |