compositionality.
33 of 1538 NOTE documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| NOTE-245 | Mathematical Foundations for a Compositional Distributional Model of Meaning | The paper builds DisCoCat as a product category FVect × P of vector spaces and a free pregroup. A sentence’s meaning is f(w₁⊗…⊗wₙ), where f is the linear map got by putting vector spaces in place of the pregroup types in the reduction p₁…pₙ ≤ s. The ε maps become inner products and the η maps become Σᵢ eᵢ⊗eᵢ. So “John likes Mary” is Σ_ijk c_ijk⟨v|vᵢ⟩⟨w_k|w⟩ sⱼ ∈ S, and “not” is the 2×2 swap matrix placed on the sentence wire. There is no theorem and no experiment. The sentence space S is shown only as 1- or 2-dimensional truth values, over a toy model with one basis vector per individual, and the paper says how neither S nor the verb tensors are to be built from data. Its headline similarity numbers (3/4, 1/4, 3/8) are unnormalised inner products. Under its own Definition 5.1 they are 0.949, 0.316 and 0.6. | Read |
| NOTE-246 v2 | Reasoning about Meaning in Natural Language with Compact Closed Categories and Frobenius Algebras | The paper sets the sentence space equal to the noun space (F(s)=F(n)=W). It then uses the basis-copying map σ: v_i ↦ v_i⊗v_i of the Frobenius algebra fixed by W’s basis to lift corpus-built verb matrices Σ_i sbj_i⊗obj_i into rank-3 tensors. So a transitive sentence becomes sbj⊙(verb·obj) (“copy-subject”) or obj⊙(verbᵀ·sbj) (“copy-object”), a vector in W. On the Grefenstette–Sadrzadeh verb-disambiguation set, copy-object gets Spearman ρ=0.172, against 0.168 (Kron), 0.163 (Multp), 0.143 (copy-subject), 0.050 (additive) and an inter-annotator bound of 0.620. On a 112-term definition-matching task the best F1 is 0.21 (nouns) and 0.27 (verbs). No significance test is reported. The disambiguation numbers are identical to the authors’ COLING 2012 paper, and the definition numbers are the same data re-reported. | Read |
| NOTE-525 | Does Data Scaling Lead to Visual Compositional Generalization? | (n, k) framework: two concepts with n values, k seen combinations per value. From-scratch ResNet-50s: near-100% ID, large OOD drops; 4× ID data leaves 60–80% drops; more values or combinations help. Features go from spurious, to decodable but entangled, to linearly factored (R² > 0.8, cosine < 0.1–0.2). Proposition 4.1: with linearly factored embeddings spanning 2n − 1 dimensions, k = 2 suffices for a linear classifier to be correct on all (n − k)·n unseen pairs. DINO, DINOv2, CLIP: partly factored, above chance, not perfect. | Read |
| NOTE-592 | T2I-CompBench | 6,000 compositional prompts (attribute binding by colour, shape and texture; spatial and non-spatial relations; complex), each sub-category split 700/300, with a metric per category: question-per-object BLIP-VQA for binding (Kendall τ 0.63 with humans on colour against 0.19 for CLIPScore), a UniDet box rule for space, and a 3-in-1 average for complex prompts. Spatial relations are hardest. GORS, reward-weighted LoRA finetuning of SD v2 on its own selected samples, is rated best by humans in every sub-category, but its binding score falls sharply on pairs unseen in finetuning. | Read |
| NOTE-593 | GenEval | An automated text-to-image benchmark of 553 templated prompts in six tasks, scored per image as correct or not by a COCO-trained Mask2Former (presence, count, position) and masked-crop zero-shot CLIP (colour). 83% agreement with human annotators against 88% between annotators, and 91% on unanimous images; beats a per-task tuned CLIPScore on counting, position and binding. Best 2023 open model (IF-XL) scores 0.61 overall, at most 0.15 on position and 0.35 on attribute binding. | Read |
| NOTE-596 | Cumulative cultural evolution in the laboratory | Four chains of ten adults each learned a 27-item artificial language from 14 items and passed their test output on. Transmission error fell and string–meaning structure rose. Unfiltered (Experiment 1), the languages collapsed to 2–5 words that systematically underspecified meanings (colour lost in every chain). With homonyms filtered from training (Experiment 2), they became compositional, with substrings for colour, shape and motion. Participants believed they were copying. | Skimmed |
| NOTE-597 | Compositional Visual Generation with Composable Diffusion Models | Treats a conditional diffusion model’s noise prediction as the gradient of an energy, so concepts compose at sampling time by adding scores: AND is ε(x,t) + Σ wᵢ(ε(x,t|cᵢ) − ε(x,t)), the product p(x)Πp(x|cᵢ)/p(x) under conditional independence of the concepts given x; NOT divides by the negated concept’s likelihood. No retraining. On CLEVR object positions it beats an EBM baseline (31.36% against 7.34% at three components); on relations it does not (2.80% against 4.26%); on FFHQ attributes it trails LACE in accuracy at three components but has the best FID. Text composition with GLIDE and Stable Diffusion is qualitative only. | Read |
| NOTE-599 | Language Evolution by Iterated Learning With Bayesian Agents | Iterated learning is a Markov chain on hypotheses. If every learner shares a prior and samples from its posterior, the chain is a Gibbs sampler on P(d|h)P(h), so the probability a learner holds language h converges to the prior P(h), whatever the amount of data passed on. If learners take the MAP hypothesis, it is stochastic EM with no observations: the outcome centres on the prior’s mode, depends on data amount and noise, and in a two-language case is the same for every prior strength below a threshold. An unbounded population with equal fitness has the same equilibrium. | Read |
| NOTE-601 | Seven Sketches in Compositionality | From Chapter 1 and Section 3.4: an observation is a monotone map, and a generative effect is its failure to preserve joins, Φ(a) ∨ Φ(b) < Φ(a ∨ b); left adjoints preserve joins, and out of a preorder with all joins a monotone map preserves joins iff it is a left adjoint. Database schemas are categories, instances functors to Set, and a schema map induces migrations Σ ⊣ Δ ⊣ Π. The other five chapters were seen only at their openings and summaries. | Skimmed |
| NOTE-603 | Classifier-Free Diffusion Guidance | Shows that guidance in diffusion models needs no classifier: one network trained with randomly dropped conditioning supplies conditional and unconditional scores, and the extrapolation (1 + w)ε(z, c) − wε(z) trades FID for Inception score on ImageNet as classifier guidance does. The paper’s own argument shows the guided field is in general not the score of any density, so the Bayes-tilted target that motivates it is not what is shown to be sampled. | Read |
| NOTE-610 | T2I-CompBench++ | Extends T2I-CompBench to eight sub-categories (adding numeracy and 3D spatial relations), tests multimodal LLMs as judges, and benchmarks eleven models. GPT-4V agrees with human rankings best on interactions and complex prompts, where CLIPScore and the conference version’s 3-in-1 metric were weak; specialised VQA and detector metrics stay best elsewhere. Human ratings cover only six 2023-era models, and the interaction category remains near ceiling for humans. | Read |
| NOTE-615 v2 | Морфология сказки | Finds that in a hundred Russian wondertales the stable units are characters’ functions, defined by their consequence for the action: 31 of them, in one order, over seven spheres of action, so every wondertale is a selection from one sequence and the variety lies in omissions, realisations and the casting of characters. Plot and variant cannot be separated, tales can be generated from the scheme, and the wondertale is conjectured to be morphologically a myth. | Read |
| NOTE-625 | Backprop as Functor | Learners (P, I, U, r), with a request function r : P × A × B → A, compose and form a symmetric monoidal category Learn. For step size ε and an error e with ∂e/∂x(x₀, −) invertible, sending a differentiable parametrised function to (P, I, p − ε∇ₚE, f_a(∇ₐE)) is a faithful strong symmetric monoidal functor Para → Learn: gradient descent on a composite is the composite of local updates exchanging requests, which is backpropagation, and the proof is the chain rule. | Read |
| NOTE-628 v2 | Larger communities create more systematic languages | Varies community size alone in a laboratory language-creation game: groups of eight built more systematic, compositional languages than groups of four, faster and more uniformly across groups, while reaching the same communicative success and convergence. Input variability, greater in larger groups, predicted gains in structure; small groups differed more from each other on every measure. | Read |
| NOTE-639 | Attend-and-Excite | Identifies catastrophic neglect in Stable Diffusion, where a named subject never appears because no image patch attends to its token, and corrects it at inference by gradient steps on the latent that raise the smoothed peak attention of the most neglected subject token during the early steps. Measured gains are in CLIP and caption similarity and human preference on two-subject prompts; attribute binding improves only in the figures, and relations are not addressed. | Read |
| NOTE-642 | Morphology of the Folktale | In chs. II–III of the 1968 English text, Propp’s four theses and his 31 functions appear as in the Russian. The functions are named by nouns of action under capitalised headings, with Greek signs for the seven preparatory functions and Latin ones after. “Dramatis personae” and “characters” alternate for the same word, and the class studied is “fairy tales”, under a title Dundes calls misleading. The second edition’s additions are visible on the page: Wagner’s notes correct about forty of Propp’s tale references in two chapters and renumber the tales to the later Afanas’ev numbering. | Skimmed |
| NOTE-644 | Semantic Unification | Puts basic DRT into sheaf form: a DRS is a section of a presheaf of consistent literals, an anaphoric resolution is a cover by variable maps, and the discourse’s meaning is the gluing, unique if it exists (Proposition 1). All the interpretive work is in choosing the cover; the only obstruction to gluing shown is inconsistency, so the paper models context dependence and ambiguity, not contextuality in the Abramsky–Brandenburger sense. Its probabilistic section ranks covers by summed corpus counts, not by gluing distributions. | Read |
| NOTE-645 v2 | On the Quantum-like Contextuality of Ambiguous Phrases | Models meaning selection in two-word ambiguous phrases as a Bell-type measurement scenario. Hand-set supports give one possibilistically contextual (Hardy-type) model; every corpus-estimated model is signalling, so only Contextuality-by-Default applies, under which two noun–verb pairs read in both grammatical orders are contextual (1/30 and 7/30). The paper’s own bootstrap leaves both plausibly noncontextual (probabilities above .56 and .08), on very small counts. | Read |
| NOTE-665 | On the Emergence of Linear Analogies in Word Embeddings | Derives the parallelogram analogies of word embeddings from a generative model of co-occurrence in which words are bundles of independent binary attributes: the co-occurrence ratio is then a Kronecker product, and its logarithm, the PMI, is a bilinear form of rank at most d + 1 whose eigenvectors are affine in the attributes, so a spectral PMI embedding is a linear image of the attribute hypercube. Robustness to noise, pruning and deletion of a relation’s own pairs is argued from eigenvalue bounds and simulated; Wikipedia agrees in shape. Every embedding is an eigendecomposition, none is trained. | Read |
| NOTE-677 | The Random Hierarchy Model | Introduces the Random Hierarchy Model, a random context-free grammar in which each symbol has m synonymic expansions, and shows by experiment that deep feature-learning networks learn it from about n_c m^L examples, polynomial in the input dimension, while shallow and lazy networks need an exponential number. That size coincides with the emergence of synonym-invariant hidden representations and, by a scaling argument, with the size at which the class statistics of input patches, equal for synonyms, exceed sampling noise; removing those statistics makes deep networks fail as well. | Read |
| NOTE-684 | The Sparse Random Hierarchy Model | Adds sparsity to the Random Hierarchy Model, so that labels are unchanged by discrete displacements of the informative features, and shows by experiment that deep networks learn it from a number of examples polynomial in the input dimension, (s0 + 1)^L n_c m^L up to a factor s^(L/2) without weight sharing and (s0 + 1)^2 n_c m^L with it, and that hidden layers become insensitive to synonym swaps and to feature displacements at about the same training-set size as test error falls. That coincidence is offered as the reason deformation stability tracks performance on images, which is not tested on images. | Read |
| NOTE-685 | A Phase Transition in Diffusion Models | For the Random Hierarchy Model with known rules, exact denoising is belief propagation; an annealed mean-field map, checked against exact BP, shows that as depth grows the class is recovered below a threshold noise and lost above it, provided the grammar is sparse (s f < 1), while low-level features degrade smoothly and persist past the threshold. The same top-versus-bottom crossing appears in classifier layers under forward–backward runs of an ImageNet diffusion model, and not for a Gaussian mixture. | Read |
| NOTE-687 | How the structure of language is acquired | Extends the Random Hierarchy Model to predicting a masked last token. The correlation between two tokens shrinks by about a factor m per level of their common ancestor, a sample of P sequences resolves it only above a (v²P)^(−1/2) noise floor, and deep transformers and CNNs show loss steps at P_ℓ ≈ v m^(2ℓ−1), where their layers come to encode the level-ℓ hidden symbol. On character-level text, the correlation decay exponent predicts where the loss for a given context length levels off. The route from correlations to learned hidden variables is argued, not proved. | Read |
| NOTE-691 | Probing Latent Hierarchy via Diffusion | In the Random Hierarchy Model with known rules, the tokens changed by noising and Bayes-optimal denoising form blocks set by the depth of the latent that changed; an annealed mean-field calculation, checked against exact belief propagation at one parameter set, gives a correlation length diverging as |ε − ε*|^(−ν), ν = log s / log F′(p*), at the class threshold, and a susceptibility peak there, which a Gaussian field with power-law spectrum lacks. Masked text diffusion and ImageNet diffusion show a susceptibility peak at an intermediate noise, over about one decade of distance and with no non-hierarchical control. | Read |
| NOTE-813 | Fu et al. — Evaluating Compositional Scene Understanding in Multimodal Generative Models | Human raters judged DALL-E 3 images. Agreement with single familiar relations rose well above DALL-E 2’s. It fell when subject and object of an asymmetric relation were swapped, and again when two relations were combined; low-agreement reversed images mostly had the wrong relation or swapped roles. Five VLMs learned relational concepts few-shot below human level, and four were at chance with six objects. | Read |
| NOTE-1158 | Bar-Tal, Yariv, Lipman and Dekel — MultiDiffusion | A training-free sampler that runs one pre-trained diffusion model on several crops or masked prompts of one canvas and, at every step, replaces their disagreeing proposals with the least-squares reconciliation, which for pixel-sampling maps is a per-pixel weighted average. It makes seamless panoramas (crop FID 10.3 against 18.4 and 45.5 for outpainting baselines) and follows rough region layouts (COCO IoU 0.26 against 0.16–0.17), most of the layout gain coming from an early masked bootstrapping phase. Its guarantee that each crop follows the reference model’s path holds only at zero loss, which disagreeing overlaps do not reach. | Read |
| NOTE-1160 | Zhong et al. — Multi-LoRA composition for image generation | Two training-free ways of using several LoRAs in one diffusion sampler without merging their weights: LoRA Switch gives each LoRA its own denoising steps in rotation, and LoRA Composite averages every LoRA’s guided noise prediction at every step. On 480 sets of two to five public LoRAs, GPT-4V’s paired judgements (and two annotators on 120 images) prefer both to weight merging, increasingly as LoRAs are added. In Switch, the switching interval and which LoRA acts first change the result at equal step shares. The baseline’s merged adapter grows with the number of LoRAs, which the design does not separate from the decoding-time idea. | Read |
| NOTE-1189 | Elhage, Nanda, Olsson et al. — A mathematical framework for transformer circuits | An exact reformulation of attention-only transformers, and a reading of toy models with it. Heads are independent and additive, each a QK bilinear form that sets the attention pattern and an OV map that sets what moves; with patterns frozen, the logits are a sum of path terms (direct, one per head, one per composed pair of heads). Zero-layer models are bigram tables, one-layer models bigram and skip-trigram tables dominated by copying heads, and in a two-layer model the only substantial composition builds induction heads from a previous-token head by K-composition. MLP layers, two-thirds of a standard transformer’s parameters, are left open. | Read |
| NOTE-1223 | Baez & Fong — A compositional framework for passive linear networks | Open passive linear circuits (resistors, inductors and capacitors, the latter two through the Laplace transform over the field R(s)) are the morphisms of a hypergraph category Circ of decorated cospans, and black-boxing, which keeps only the Lagrangian relation a circuit imposes between its terminals’ potentials and currents, is a hypergraph functor Circ → LagRel (Thm 1.1, Thm 7.7). The relation is the graph of the differential of the power functional, the Dirichlet form left when the interior nodes are eliminated by the principle of minimum power; series resistors are the worked example. Dirichlet forms alone compose by that elimination but have no identities, so the target is Lagrangian relations. | Read |
| NOTE-1225 | Spivak — Compositional dynamics in learning and mechanics | A category-theoretic syntax, the operad ArrSm of “smooth adaptive arrangements” (a responsive parameter space, a lens of output and input maps between manifolds, and a real potential), with two functorial semantics into polynomial coalgebras, i.e. discrete-time input-output machines. They differ only in the integrator: Φconf stores position and takes a descent step, Φphase stores position and momentum and takes a symplectic-Euler step. Φconf turns a parameterized map plus a loss into gradient descent with backpropagation, and a closed system with inverse-Hessian response into Newton’s method. One arrangement of harmonic particles on a finite directed graph gives the discrete wave equation under Φphase and the discrete heat equation under Φconf, both governed by the symmetrized graph Laplacian. The main technical result is lens internalization, a lax symmetric monoidal functor Lens_C → C. | Read |
| NOTE-1326 | Sarkar, Mukherjee and Sato — Relational composition as parallel transport | The companion paper’s premise, stated by its authors: per-entity frame freedom is “a chosen inductive bias, not a consequence of fresh names”. Under it, scores go through ⟨xᵢ, Tᵢⱼxⱼ⟩ and transports must be supplied. Its parsed-benchmark controls show the premise’s orthogonality helping on StepGame and costing 64.6 points on CLUTRR. | Skimmed |
| NOTE-1331 | Sarkar, Mukherjee and Sato — Attention as lattice gauge theory | Under per-entity O(d) frame freedom, transports carry invariant information only around cycles, a triangle’s invariants are class functions of its holonomy, and an equivariant edge update is Ψ(H)Tᵢⱼ with Ψ conjugation-equivariant: a sound conditional theorem, standard in lattice gauge theory and synchronization, whose “unique form” is a family. The untrained S4 experiments show composition doing the work and Wilson-loop routing adding 0.6–7.8 points; the 0.99 AUC is a ceiling the data’s construction sets, as the paper itself computes. | Read |
| NOTE-1349 | Gentner — Structure-mapping | Gentner proposes that an analogy maps the relations of a base domain onto a target and drops the objects’ attributes, and that which relations go across is decided by systematicity: a relation belonging to a system tied together by higher-order relations is more likely to be imported than an isolated one. The rules look only at the form of the representation (arity and order of predicates), which lets literal similarity, analogy, abstraction and mere appearance be told apart by what kinds of predicate overlap. The framework is argued from examples; the empirical support is one study summarized in a paragraph. | Read |