nucleation

LIT-1529: Multi-LoRA Composition for Image Generation

1 min read

Properties5
StatusActive — read 2026-10-25 (NOTE-1160) in full, from arXiv v2, the TMLR version; worth reading for two ways of letting separately trained low-rank operators act on one denoising trajectory without merging them, by turns (LoRA Switch) or by averaging their guided predictions (LoRA Composite), for GPT-4V and human judgements preferring both to weight merging as LoRAs are added (THEORY-955), and for its ablation in which, at equal shares of steps, the switching interval and the LoRA acting first change the result (THEORY-949). Read it knowing that the merging baseline's summed update grows with the number of LoRAs, which the design does not separate from the methods' idea; that the two methods are never compared head to head by the judge; that the judge has a positional bias of about ten points and correlates 0.45 with two annotators; and that §5.1.1 attributes LoRA Switch's scores to LoRA Composite.
Version2
Tagscompositionality, anthology-candidate
Cited as “paper” byNOTE-1160 — Zhong et al. — Multi-LoRA composition for image generation
Cited as “source” byTHEORY-949 — When several LoRAs take turns over one denoising trajectory, each given the same share of steps, the result depends on the schedule and not only on the shares: switching at every step scored worst (a composition win rate against weight merging near 38%, against about 59% when each LoRA kept five consecutive steps), shortening the turns late in sampling scored worse than keeping them fixed, and which LoRA acted first mattered more than the order of the rest, the character first scoring best and the style first slightly below a random order, THEORY-955 — On 480 sets of two to five public LoRAs on two Stable Diffusion 1.5 checkpoints, GPT-4V's paired judgements, averaged over both presentation orders and three seeds, scored giving each LoRA its own denoising steps in rotation (LoRA Switch) and averaging every LoRA's guided noise prediction at every step (LoRA Composite) above merging the LoRAs' weights at 0.8 each, at every number of LoRAs, the composition gap growing from 0.04–0.11 points of 10 at two LoRAs to 0.92–1.32 at five as merging fell from about 8.7 to 5.2; two annotators rating 120 images agreed in direction; the design does not separate composing at sampling time from the merged update's scale growing with the number of LoRAs

Graph View

Zhong, M., Shen, Y., Wang, S., Lu, Y., Jiao, Y., Ouyang, S., Yu, D., Han, J. & Chen, W. (2024). Multi-LoRA composition for image generation. Transactions on Machine Learning Research. arXiv:2402.16843.

Read in full on 2026-10-25 (NOTE-1160). The main comparison is held in THEORY-955 and the switching ablation in THEORY-949. LoRA Composite turns out to be MultiDiffusion’s per-step average (LIT-1522, THEORY-958) with every mask full and LoRA-modified models in place of prompts.

Why this is here

Filed as the work behind the genre U46 named. At U46 the reply gave its perspectives separately learned denoising capabilities (LoRA adapters) and combined their contributions step by step; LoRA Switch and LoRA Composite are per-step versions of that combination, without the learned allocation of influence that the exchange wanted to add. The record holds none of the LoRA literature. It bears on QUESTION-273, a population of denoisers being one of the mechanisms of influence it asks about, and on the audited-governance study CASE-158, whose arms decide which specialist acts. Filed for that question, not as an instruction for practice.

Backlinks