anthology-candidate.
319 of 1538 NOTE documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| NOTE-052 v2 | Consciousness in AI: indicator properties | The report assumes computational functionalism and takes a “theory-heavy” approach. From it the authors derive 14 indicator properties: RPT-1–2, GWT-1–4, HOT-1–4, AST-1, PP-1 and AE-1–2. The claim for them is only that more of them raises the probability of consciousness. They assess a handful of systems against subsets of the list and conclude that no current system “appears to be a strong candidate”. Transformer LLMs have “only a relatively weak case” for any GWT indicator, because a single forward pass has no recurrence or global broadcast. Perceiver arguably has GWT-1, GWT-2 and half of GWT-4 but lacks broadcast. Among the embodied agents, AdA is the likeliest to meet AE-2. Most indicators could be built with standard ML techniques. The abstract’s “no current AI systems are conscious” is stronger than anything the body shows. | Read |
| NOTE-081 v2 | The No-U-Turn Sampler (NUTS) | NUTS removes HMC’s step-count L. It doubles a leapfrog trajectory forwards and backwards in random directions until some balanced subtree satisfies (θ⁺−θ⁻)·r⁻ < 0 or (θ⁺−θ⁻)·r⁺ < 0. It keeps detailed balance by slice sampling and by excluding states that could not have regenerated the tree. With ε set by a dual-averaging scheme (γ = 0.05, t₀ = 10, κ = 0.75, target δ ≈ 0.6), it matched the best hand-tuned HMC in ESS per gradient on two logistic-regression posteriors. It beat that HMC by about 3× on a 250-D Gaussian and on a 3001-D stochastic-volatility model (§4.4). | Read |
| NOTE-086 v2 | I-MMSE: mutual information and MMSE in Gaussian channels | Let Y = √snr·X + N with N ~ N(0,1) independent of X, and let X have any distribution with E X² < ∞. Then d/dsnr I(X; √snr X + N) = ½·mmse(X | √snr X + N) in nats, equivalently I(snr) = ½∫₀^snr mmse(γ) dγ (Thm 1, Eq. 47). The vector form is d/dsnr I(X; √snr HX + N) = ½·E‖HX − H·E[X|Y]‖² (Thm 2). In continuous time, the same identity combined with Duncan’s I = (snr/2)·cmmse gives cmmse(snr) = (1/snr)∫₀^snr mmse(γ) dγ (Thm 8). | Read |
| NOTE-102 v2 | Bottou & Schölkopf — Borges and AI | A short essay, not an argument with evidence. It claims that a “perfect language model”, defined as sampling each next word from a random occurrence of the current text in the infinite collection of all plausible texts, is a fiction machine governed by “narrative necessity”, to which “neither truth nor intention” matters. On that view hallucinations “are just confabulations”, chat-bot answers take the role the user’s framing implies, and fine-tuning and RLHF are attempts to prune the garden “against its nature”. It warns that power over what LLMs write becomes “power over what we think”. | Read |
| NOTE-199 v2 | Balestriero & LeCun 2022 — SSL recovers spectral embedding | VICReg, SimCLR and Barlow Twins each solve, in closed form, a classical spectral embedding problem on the positive-pair graph G (Laplacian Eigenmaps, a generalized MDS/ISOMAP, and CCA respectively), so contrastive vs non-contrastive SSL is global vs local spectral embedding, and when G matches the downstream task any of them is optimal. | Superseded — by NOTE-752 |
| NOTE-200 v2 | Rate-distortion generalization bounds | Defining an algorithm’s compressibility in source-coding terms shows that the Xu–Raginsky mutual-information bound is really a lossless compression rate, and allowing lossy compression yields bounds via the rate-distortion dimension that stay meaningful where I(S;W) is huge. | Superseded — by NOTE-780 |
| NOTE-201 v2 | Assran et al. 2022 — hidden uniform cluster prior in SSL | Joint-embedding SSL methods that rely on mini-batch volume maximization (SimCLR, VICReg, SwAV, MSN) implicitly impose a prior toward features that cluster the data into equal-sized groups, which hurts on class-imbalanced data; replacing it with a power-law prior (PMSN) improves representations on long-tailed data. | Superseded — by NOTE-850 |
| NOTE-205 v2 | Balestriero, Bottou & LeCun 2022 — regularization is class-dependent | Regularizers tuned by cross-validation for average accuracy — random-crop augmentation and even uninformed weight decay — raise mean test accuracy while sharply lowering it on particular classes (e.g. ImageNet “barn spider” 68%→46% with random crop), and this class-dependent bias carries over to transfer tasks. | Superseded — by NOTE-845 |
| NOTE-207 v2 | MDL generalization guarantees for representation learning | Generalization of representation learning is bounded not by I(X;U) but by the minimum description length of the latent variables (or predicted labels) under a symmetric prior — a multi-letter relative entropy that reflects the encoder’s structure and is non-vacuous for deterministic encoders. | Superseded — by NOTE-742 |
| NOTE-208 v2 | Sun et al. 2023 — spectral theory of novel class discovery | For novel class discovery, a spectral contrastive loss over a graph of labeled and unlabeled data (NSCL) is equivalent to factorizing its adjacency matrix, and the resulting linear-probe error on novel classes is bounded — down to zero — by how far the known classes’ feature span covers the unlabeled data’s “ignorance space”. | Superseded — by NOTE-833 |
| NOTE-209 v2 | Tan et al. 2023 — contrastive learning is spectral clustering | SimCLR’s unmodified InfoNCE loss is equivalent to spectral clustering on the augmentation-defined similarity graph (min tr(Zᵀ L(π) Z) + log R(Z)), CLIP does generalized spectral clustering on the image–text pair graph, and mixtures of exponential kernels (Kernel-InfoNCE) beat the Gaussian kernel. | Superseded — by NOTE-846 |
| NOTE-210 v2 | The Conditional Entropy Bottleneck | Training a representation to minimize I(X;Z|Y) while maximizing I(Y;Z) — the Conditional Entropy Bottleneck, targeting the “Minimum Necessary Information” point I(X;Y)=I(X;Z)=I(Y;Z) — empirically improves accuracy, adversarial robustness, OoD detection and calibration and refuses to memorize random labels. | Superseded — by NOTE-754 |
| NOTE-211 v2 | Knowledge Sheaves (sheaf-theoretic KG embedding) | Knowledge-graph embedding can be recast as learning an approximate global section of a cellular sheaf on the schema graph, which subsumes Structured Embedding and TransE/TransR-style models and yields, via harmonic extension, a training-free way to answer composite multi-hop queries. | Superseded — by NOTE-843 |
| NOTE-216 v2 | Variable-size compressibility generalization bounds | Letting the compression rate of an algorithm’s input data vary with the observed sample yields generalization bounds that depend on the empirical measure rather than the unknown distribution, and this single framework recovers PAC-Bayes and data-dependent intrinsic-dimension bounds as special cases. | Superseded — by NOTE-806 |
| NOTE-218 v2 | Simon et al. 2023 — stepwise nature of SSL | Joint-embedding SSL learns its embedding one dimension at a time in discrete, well-separated steps — in a linearized Barlow Twins model it provably learns the top eigenmodes of a contrastive kernel in order, so SSL is to kernel PCA what supervised learning is to kernel regression — and the same stepwise rank growth appears in ResNets trained with Barlow Twins, SimCLR and VICReg. | Superseded — by NOTE-741 |
| NOTE-219 v2 | Duality of Bures and Shape Distances | The Procrustes shape distance between two representations (optimal rotational alignment of units) equals the Bures distance between their centred linear kernel matrices, so the rotation-invariant geometry of a representation is exactly the geometry of its kernel, and through Uhlmann’s theorem the fidelity of two kernels is the maximum overlap over all representations consistent with them. | Superseded — by NOTE-753 |
| NOTE-221 v2 | Closeness in Distribution vs Representational Similarity | For softmax models p(y|x) ∝ exp(f(x)ᵀg(y)) under a diversity condition, the model distribution determines the embeddings and unembeddings up to an invertible linear map (f = Af′, g₀ = A⁻ᵀg′₀). The determination is not stable under KL divergence, though: two models can be ε-close in KL, even both near maximum likelihood, while their embeddings are far from linearly equivalent. | Superseded — by NOTE-738 |
| NOTE-222 v2 | Tosh et al., contrastive learning and multi-view redundancy | The logistic contrastive problem of telling a same-datum pair of views from an independent pair is solved over all functions by the pointwise mutual information f*(x,z) = log p(x,z)/(p(x)p(z)), whose exponential g* is the change of measure from p(z) to p(z|x), and linear functions of the landmark embedding (g*(x,Z_1),…,g*(x,Z_m)) are near-optimal predictors whenever the two views are redundant for the label. | Superseded — by NOTE-841 |
| NOTE-223 v2 | Similarity of NN Representations Revisited (CKA) | A similarity index for neural representations should be invariant to orthogonal transformation and isotropic scaling but not to arbitrary invertible linear maps, and comparing the example-by-example Gram matrices (linear or kernel CKA) achieves exactly that, while any index invariant to invertible linear maps becomes uninformative once width reaches the number of examples. | Superseded — by NOTE-743 |
| NOTE-224 v2 | Belghazi et al., MINE | Mutual information is stated measure-theoretically as the expectation under the joint of log dP_XZ/d(P_X⊗P_Z), and its Donsker–Varadhan dual is tight exactly at T* = log dP/dQ + C, so a neural critic trained on the dual is a log Radon–Nikodym-derivative estimator. | Superseded — by NOTE-828 |
| NOTE-225 v2 | What Similarity Measures Imply about Decodability | For linear readouts regularised by w ↦ wᵀG(X)w, the optimal decoded signal is K_X z with K_X = X G(X)⁻¹ Xᵀ / M. The expected alignment of two networks’ optimal readouts is therefore Tr(K_X K_z K_Y), which recovers linear CKA, CCA, GULP and ENSD as average decoding similarities. Procrustes distance bounds the average decoding distance from both sides, with constants set by a participation ratio. | Superseded — by NOTE-834 |
| NOTE-226 v2 | Kernel mean embedding of distributions (review) | Point evaluation is bounded on an RKHS, so the Riesz theorem gives the reproducing kernel k_x. Likewise, when E√k(X,X) < ∞ the expectation functional f ↦ E_P f is bounded, so Riesz gives a unique mean embedding μ_P with E_P f = ⟨f, μ_P⟩. MMD is then the RKHS distance ‖μ_P − μ_Q‖, and it separates distributions exactly when the kernel is characteristic. | Superseded — by NOTE-838 |
| NOTE-227 v2 | Spectral contrastive loss (HaoChen et al.) | Minimizing the population spectral contrastive loss L(f) = −2E[f(x)ᵀf(x⁺)] + E[(f(x)ᵀf(x⁻))²] is, up to a constant, a rank-k factorization of the normalized adjacency matrix of the augmentation graph, so its minimizers are the top-k eigenvectors (eigenfunctions, in the infinite case) of that graph up to a row scaling and an invertible linear map — neither of which changes linear-probe accuracy. | Superseded — by NOTE-740 |
| NOTE-228 v2 | Contrastive learning finds an optimal eigenbasis | The InfoNCE, NT-Logistic and spectral contrastive losses all have the same population minimizer, the positive-pair kernel K⁺(a₁,a₂) = p⁺(a₁,a₂)/(p(a₁)p(a₂)). Kernel PCA under K⁺ returns exactly the eigenfunctions of the positive-pair Markov chain, and the top-d of those span the linear-predictor subspace that minimizes worst-case approximation error over approximately view-invariant targets. | Superseded — by NOTE-751 |
| NOTE-231 v2 | Spectral Inference Networks (SpIN) | The top-N eigenfunctions of a symmetric kernel operator Kf = E_{x′}[k(x,x′)f(x′)] on L²(p) can be learned by a neural network that maximizes the generalized Rayleigh quotient Tr(Σ⁻¹Π), with Σ = E[u uᵀ] and Π = E[k(x,x′)u(x)u(x′)ᵀ]. Slow Feature Analysis is the special case where k is the graph Laplacian of adjacent video frames. | Superseded — by NOTE-824 |
| NOTE-232 v2 | Tschannen et al., on MI maximization for representations | The success of InfoNCE-style representation learning cannot be attributed to mutual information itself: MI is invariant under invertible reparametrizations, invertible encoders that maximize true MI can be worse than raw pixels, and tighter MI bounds from higher-capacity critics can give worse representations. | Superseded — by NOTE-749 |
| NOTE-233 v2 | Poole et al., variational bounds of mutual information | Every variational lower bound on mutual information in use — Barber–Agakov, Donsker–Varadhan/MINE, NWJ/f-GAN-KL, InfoNCE — is one family, tight at a critic that is a function of the log density ratio log p(y|x)/p(y), and InfoNCE is the multi-sample member that trades variance for a hard ceiling of log K. | Superseded — by NOTE-739 |
| NOTE-234 v2 | SSL-HSIC (kernel dependence maximization) | With image identity as the label, the SSL-HSIC loss −HSIC(Z,Y) + γ√HSIC(Z,Z) has a dependence term that is proportional to the average squared MMD between the per-image distributions of augmented-view representations (App. B.2). InfoNCE approximates the same term plus a variance penalty (eq. 7), so contrastive SSL separates the kernel mean embeddings of each image’s view distribution. | Superseded — by NOTE-830 |
| NOTE-273 | Not All Language Model Features Are One-Dimensionally Linear | Defines a feature as irreducibly multi-dimensional when no rotation and translation makes its distribution either a product of independent parts (separable) or a mixture with a lower-dimensional component (Def. 2). Softened versions are the separability index S(f) and the ε-mixture index M_ε(f) (Def. 3). Clustering sparse-autoencoder dictionary elements by cosine similarity then finds circular representations of weekdays and months in GPT-2 (also 20th-century years) and in Mistral 7B. Replacing the circle coordinates at the day or month token causally changes the answer on “N days/months from X” prompts in Mistral 7B and Llama 3 8B, and in early layers the circle patch nearly matches patching the whole layer. The models’ task accuracy is modest on weekdays (29/49 Llama, 31/49 Mistral), high on months (143/144, 125/144), and near-trivial for GPT-2 despite its circles. | Read |
| NOTE-274 | Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges | The authors cast CNNs, GNNs, Deep Sets, Transformers, spherical, mesh and gauge CNNs, and LSTMs as instances of one “blueprint” (p. 29). Each is a stack of linear G-equivariant layers, pointwise nonlinearities and local pooling, capped by a G-invariant global pooling, with the domain Ω and its symmetry group G given in advance (p. 27: “a symmetry group G”; p. 4: “exploiting the known symmetries”). The two priors are symmetry and scale separation. The Fourier transform is derived as the joint eigenbasis of shift-commuting (circulant) operators, which requires distinct eigenvalues (p. 37). Its group-theoretic generalisation via irreducible representations is announced and deferred to “future work” (p. 43). | Skimmed |
| NOTE-278 | Toy Models of Superposition | In a toy autoencoder x′ = ReLU(WᵀWx + b), with n sparse synthetic features compressed into m < n dimensions, a linear model keeps only the top-m features, but with the output ReLU and sparse enough features the model stores more features than dimensions (“superposition”), in non-orthogonal directions whose interference the ReLU and a negative bias filter out. Whether a feature is dropped, superposed or given a dedicated dimension changes discontinuously with sparsity and importance, and in the n = 2, m = 1 theory this is a first-order phase change. With uniform features the directions settle into uniform polytopes, with feature dimensionalities clustering at 3/4, 2/3, 1/2, 2/5 and 3/8. A ReLU hidden layer creates a privileged basis in which monosemantic and polysemantic neurons coexist, and a model can compute |x| for more features than it has neurons. Everything is shown in toy models only, and the authors expect the geometry and learning-dynamics results to generalise least. | Read |
| NOTE-282 v2 | On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups | Theorem 1 says that a feed-forward network whose layers are linear maps followed by pointwise nonlinearities, with every layer indexed by a homogeneous space G/H_ℓ of a compact group G, is G-equivariant (every layer transforming by the induced action) if and only if each linear map is a generalised convolution f ∗ χ with a filter on the double coset space H_{ℓ−1}\G/H_ℓ. The “if” direction is a short coset computation. The “only if” direction goes through the group Fourier transform: equivariant maps preserve isotypic components (Lemma 6) and act on each Fourier matrix by right multiplication, M ↦ MB (Lemma 10), which the convolution theorem (Prop. 2) turns back into a convolution. | Read |
| NOTE-288 v2 | Group Equivariant Convolutional Networks | Group convolution (eqs. 10–11) replaces translation by the elements of a discrete group G, so feature maps become functions on G and every layer is G-equivariant (eq. 12, with pooling and nonlinearities shown to commute with the action). With parameter counts held roughly fixed, p4-CNNs cut rotated-MNIST test error from 5.03% (planar baseline) to 2.28% (previous best 3.98%), and p4m convolutions cut plain-CIFAR10 error from 9.45% to 6.46% on a ResNet44 (Tables 1–2). | Read |
| NOTE-300 | The information bottleneck method | The paper defines relevant information as what X tells about a second variable Y and poses the information-bottleneck problem, min over p(x̃|x) of L = I(X̃;X) − β·I(X̃;Y), under the Markov chain X̃ ← X ← Y (eq. 15). Theorem 4 gives the exact formal solution p(x̃|x) = p(x̃)/Z(x,β)·exp(−β·D_KL[p(y|x)‖p(y|x̃)]), so the KL divergence emerges as the effective distortion. Theorem 5 gives a Blahut–Arimoto-style alternating iteration that converges but, being non-convex jointly, not necessarily to a unique solution. | Read |
| NOTE-306 v3 | Information-theoretic analysis of generalization capability of learning algorithms | If the loss ℓ(w,Z) is σ-subgaussian for every w, then the expected generalisation error of any learning algorithm P_{W|S} obeys |E[L_μ(W) − L_S(W)]| ≤ √(2σ²·I(S;W)/n) (Theorem 1). This extends Russo–Zou to uncountable hypothesis spaces, and the paper adds a high-probability version, n = (8σ²/α²)(ε/β + log(2/β)) (Theorem 3). Minimising empirical risk plus I(S;W), relaxed to a KL term against a prior Q, yields exactly the Gibbs algorithm (Theorem 5). Noisy ERM and adaptive composition are analysed through the same mutual-information quantity. | Read |
| NOTE-308 | How much does your data exploration overfit? Controlling bias via information usage | Suppose an analyst reports statistic φ_T, chosen by any data-dependent rule T from m candidates each σ-sub-Gaussian. Then the selection bias obeys |E[φ_T − µ_T]| ≤ σ√(2·I(T;φ)) (Prop. 1), where “information usage” I(T;φ) ≤ H(T) ≤ log m. For argmax selection over φ ~ N(µ,I) the squared error is Θ(1 + H(T)) (Prop. 3). Adaptive analyses compose additively, with I(T_{k+1};φ) ≤ Σ_i I(Y_{T_i};φ_{T_i} | H_{i−1},T_i) (Lemma 1). Answering each query with Gaussian noise of variance σ²√j/n therefore keeps the k-th answer’s error at O(σk^{1/4}/√n) (Prop. 7). | Read |
| NOTE-309 | Understanding image representations by measuring their equivariance and equivalence | Given a fixed representation φ (HOG, or AlexNet Conv1–Conv5 before ReLU) and a transformation g of the input image chosen by the experimenter (flips, 90° and other rotations, rescaling by 2^{±1/2}, affine warps), it fits a linear map M_g on the representation space with φ(gx) ≈ M_g φ(x), by sparse or structured-sparse regression or by a “transformation layer” trained through the rest of the network, and fits “stitching layers” E with φ′(x) ≈ E φ(x) between different networks. In AlexNet, learned M_g recover most of the classifier’s accuracy for vertical flips and 90° rotations (top-1 error 0.75 uncompensated vs 0.43–0.51 and 0.44–0.53 across Conv1–Conv5, original 0.43), channels scored invariant rise to 99.61% for horizontal flips at Conv5, and Conv1–Conv2 are interchangeable between AlexNet and networks trained on ImageNet, Places or both. It detects no group and uses no spectrum. | Read |
| NOTE-317 | Grokking and Generalization Collapse: Insights from HTSR theory | A depth-3, width-200 ReLU MLP trained on 1,000 MNIST images (the Omnigrok recipe, with weights and biases scaled ×8, MSE loss, AdamW, WD = 0) for 10⁷ steps goes through three phases read off the accuracy curves. In pre-grokking (to ~10⁵ steps) training accuracy saturates while test accuracy stays low. In grokking (~10⁵–10⁶) test accuracy rises to ~0.85. In “anti-grokking” (~10⁶–10⁷) it falls to ~0.45–0.5 while training accuracy stays near 1. The average HTSR power-law exponent α of the two hidden weight matrices goes 4.0 → 2.9 → 1.1 at the phase ends (Table 1). In the collapse phase, correlation traps (outlier eigenvalues of the elementwise-shuffled weight matrix) appear. The paper’s own weight-decay control shows FC2’s α below 2, and traps in both layers, without a collapse (Fig. 6, Table 2). That undercuts its headline claim that α < 2 “invariably” heralds collapse. | Read |
| NOTE-318 | New Evidence of the Two-Phase Learning Dynamics of Neural Networks | VGG-16 and ResNet-20 on CIFAR-10 (SGD with momentum, lr 0.1, 160 epochs) are compared across pairs of training times. Before an early “inflection point” (~2,500 iterations for VGG-16, ~100–500 for ResNet-20), a tiny parameter perturbation under identical SGD noise sends the run to a different basin by the end: loss barrier up to ~2, test disagreement ~0.15–0.5. After it, the same perturbation leaves the runs linearly connected. Also after it, the empirical NTK keeps changing step to step but stays within a bounded cosine distance of any later reference kernel (the “cone effect”, e.g. S ≈ 0.05–0.1 for VGG-16 vs ≈ 0.15–0.2 from initialisation). Switching to linearised training later gives better test accuracy (VGG-16 0.13 → 0.82 as the switch moves from ≤200 to 5,000 iterations). | Read |
| NOTE-351 | Intelligence without representation | A methodological manifesto with working robots behind it. Brooks argues that AI should build complete autonomous Creatures incrementally and test them in the real world, and that intelligence should be decomposed into parallel activity-producing layers, each running from sensing to action, not into perception, central reasoning and action. His robots have no central representation or control, and their coherence exists “in the eye of an observer”, as in Minsky’s account of human behaviour. The evidence is three layers on one robot, described, not measured, and the broad hypothesis about representation is flagged as a hypothesis. | Read |
| NOTE-360 | Selfridge — Pandemonium | A proposal, with a preliminary Morse-code trial, for pattern recognition by a hierarchy of parallel “demons”. Data demons hold the input, subdemons compute features, cognitive demons shout weighted sums, and a decision demon takes the loudest. It adapts by hill-climbing on the weights, by culling and breeding subdemons through mutation and “conjugation”, and in principle by adapting its own control demons. The only result is reported in the discussion: hill-climbing “does work”, subdemon selection “did not do what we hoped”, and the best single subdemon was right 90% of the time. | Read |
| NOTE-454 | Sophisticated Inference | Expected free energy evaluated recursively over the beliefs an agent would hold after each plausible outcome turns active inference into a pruned tree search over belief states, formally analogous to Bellman recursion. Small simulations show it planning through short-term costs (an aversive cue; a maze barrier needing horizon four) and exploring by novelty. Its scalability is argued from pruning, not demonstrated; its first-principles grounding is cited, not developed. | Read |
| NOTE-471 | Bayesian Interpolation | Evidence ≈ best-fit likelihood × Occam factor, the ratio of posterior to prior accessible parameter volume. At the evidence maximum the regulariser’s χ² equals γ = Σ λ_a/(λ_a + α), the number of well-determined parameters, and the data χ² equals N − γ. Polynomials show an “Occam hill” in k, and the true Hermite model wins on data drawn from it. | Read |
| NOTE-472 | Natural Gradient Works Efficiently in Learning | Natural gradient ∇̃L = G⁻¹∇L is steepest descent under metric G (Theorem 1); for a statistical model G is the Fisher information. Online learning w_{t+1} = w_t − (1/t)∇̃l is Fisher efficient, Ṽ_t = G⁻¹/t + O(1/t²) (Theorem 2). Explicit G and G⁻¹ for a perceptron (Theorems 3–4) and the natural gradient ∇L·WᵀW on the matrix group (Theorem 6). The plateau claim for multilayer perceptrons is a conjecture resting on Yang & Amari (1997). | Read |
| NOTE-473 | A Widely Applicable Bayesian Information Criterion | The Bayes free energy of a singular model is nL_n(w0) + λ log n − (m − 1) log log n + O_p(1), with λ the RLCT. The posterior mean of nL_n(w) at inverse temperature 1/log n (WBIC) reproduces the first two terms without knowing λ, equals BIC for regular models, and two temperatures estimate λ. In reduced-rank regression WBIC chose the true rank 100/100 times, and estimated RLCTs within about 0.5 of theory. | Read |
| NOTE-474 | Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra | G = Gclass + Gcross + Gwithin + Gc=c′ (Eq. 6.4); knockouts give C outliers to Gclass, a C² mini-bulk to Gcross, the bulk to Gwithin, in log-spectra. Gˡ = Σ w (hhᵀ) ⊗ (δδᵀ) (Eq. 7.8), so gc ≈ hc ⊗ δc. K-FAC Hˡ ⊗ Δˡ misaligns G’s outliers; CFAC Ave_c Hˡc ⊗ Δˡc aligns them at deep layers. For multinomial logistic regression on a Gaussian model the expected Fisher has C outliers, C(C−2) mini-bulk and a bulk (Theorem 8.1). | Read |
| NOTE-475 | Deep Equals Shallow for ReLU Networks in Kernel Regimes | For κ(xᵀx′) on S^(d−1), µ_k = (ω_(d−2)/ω_(d−1)) ∫ κ(t)P_k(t)(1 − t²)^((d−3)/2) dt (Eq. 8), shared by N(d, k) ~ k^(d−2) harmonics. Theorem 1: an endpoint expansion κ(1 − t) = p(t) + c t^ν + o(t^ν) with ν non-integer gives µ_k ~ C k^(−d−2ν+1), parity-dependent. ReLU kernels have ν = 1/2 (NTK) and 3/2 (random features) at any depth, so the deep NTK decays as k^(−d) and the deep RF kernel as k^(−d−2) (Corollaries 2–3). MNIST and Fashion-MNIST accuracy is flat in depth from L = 2 to 5 (Table 1). | Read |
| NOTE-476 v2 | Prevalence of Neural Collapse during the terminal phase of deep learning training | NC1 Σ_W → 0 (Tr(Σ_WΣ_B†)/C falls), NC2 centred means → simplex ETF (cosines → −1/(C−1)), NC3 W ∝ Ṁᵀ, NC4 nearest class mean; 7 datasets × 3 nets, 480 models. Median TPT gains: +0.35 points test accuracy, +0.0252 DeepFool robustness. NC1–2 ⇒ NC3–4 for Webb–Lowe and max-margin (Theorems 2, 4). Optimal small-noise error exponent C/(C−1)·1/4, only at simplex ETFs (Theorem 5). | Read |
| NOTE-478 | The Full Spectrum of Deepnet Hessians at Scale | Hess = H + G (Gauss–Newton). At 28M parameters the Hessian has a bulk and about C outliers; outliers are in G, the bulk mostly in H; H is symmetric and power-law, φ ∝ |λ|⁻²·⁷ (R² 0.99), not Marchenko–Pastur or semicircle. G = A₁ + A₂ + B₁ + B₂ (Eq. 5): C class means give the outliers, cross-class spread and within-pair spread give two bulks. FastLanczos costs O(Mnp) time and O(p) memory. | Read |
| NOTE-479 | Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians | G = (1/n)ΔΔᵀ with δᵢ,c,c′ = column c′ of diag(√p)(I − 1pᵀ)∂f/∂θ. C² group means δc,c′, C cluster centres δc; G = G₀ + G₁ + G₂ + G₃ (Eq. 28). Top-C eigenvalues of G ≈ those of G₁ = (C−1)Ave_c δcδcᵀ, from a C×C Gram matrix, with λc(G) ≥ λc(G₁₊₂) ≥ λc(G₁). 18,000 CIFAR/MNIST models plus ImageNet VGG16/ResNet50. Group means switch from clustering by logit c′ to by class c during training. | Read |
| NOTE-480 | Shampoo: Preconditioned Stochastic Tensor Optimization | Matrix case: L = εI + Σ GGᵀ, R = εI + Σ GᵀG, W ← W − ηL^{-1/4}GR^{-1/4}. Lemma 8: εI + (1/r)Σ ggᵀ ⪯ L^{1/2} ⊗ R^{1/2} for rank-r gradients, so the Kronecker preconditioner dominates full AdaGrad. Regret √(2r) D Tr(L^{1/4}) Tr(R^{1/4}) = O(√T) (Theorem 7); order-k tensors use power −1/2k per dimension (Theorem 10). Steps per second within a few percent of SGD/Adam/AdaGrad (Table 1); on LM1B 3.509 against 4.87–4.92. | Read |
| NOTE-481 | The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies | For uniform data the wide two-layer ReLU Gram matrix H∞ is a convolution on the sphere, so its eigenvectors are Fourier modes or spherical harmonics. On S¹, bias-free: λ_k = 1/π², 1/4, 2(k²+1)/(π²(k²−1)²) and 0 for k = 0, 1, even k ≥ 2 and odd k ≥ 3 (Eq. 12). With zero-initialised bias, odd k ≥ 3 get 1/(π²k²) (Eq. 13). Learning time grows as k² on S¹ and about k³ on S², as measured for shallow, deep and residual networks (Figs. 6–7). | Read |
| NOTE-482 v2 | Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2 | JumpReLU SAE: f(x) = z ⊙ H(z − θ), z = W_enc x + b_enc, x̂ = W_dec f + b_dec, loss ‖x − x̂‖² + λ‖f(x)‖₀, with θ trained by straight-through estimators (bandwidth ε = 0.001). Released on all layers and sublayers of Gemma 2 2B and 9B and three layers of 27B, at widths 2¹⁴–2²⁰. Evaluated by delta LM loss and FVU against L0. Residual SAEs cost the most loss; base-model SAEs transfer to the IT model; wider SAEs split latents. No agreed SAE quality metric exists. | Read |
| NOTE-483 | Towards Understanding the Spectral Bias of Deep Learning | Theorem 4.2: n^(−1/2)‖V_(r_k)ᵀ(y − ŷ(T))‖ ≤ 2(1 − λ_(r_k))^T n^(−1/2)‖V_(r_k)ᵀy‖ + ε for any labels, given n ≥ Ω̃(ε^(−2) max{(λ_(r_k) − λ_(r_k+1))^(−2), M⁴r_k²}) and m ≥ Ω̃(poly(T, λ_(r_k)^(−1), ε^(−1))). Theorem 4.3: on uniform S^d the NTK’s eigenfunctions are spherical harmonics, µ_k = 0 for odd k ≥ 3, and µ_k = Ω(max(k^(−d−1), d^(−k+1))) for even k. Experiments in R¹⁰ learn degrees 1, 2, 4 in that order, also on three non-uniform densities. | Read |
| NOTE-486 | Optimizing Neural Networks with Kronecker-factored Approximate Curvature | Fisher blocks E[āāᵀ ⊗ ggᵀ] ≈ E[āāᵀ] ⊗ E[ggᵀ], with error a sum of third- and fourth-order cumulants; the inverse is approximated as block-diagonal or block-tridiagonal over layers. Factored Tikhonov damping (π chosen by trace norms), re-scaling by the exact Fisher, and a two-parameter momentum solved on the quadratic model. On three deep autoencoders it beats tuned NAG-SGD by orders of magnitude in iterations; block-tridiagonal gains 25–40% per iteration over block-diagonal. | Read |
| NOTE-506 | Block — Psychologism and Behaviorism | A machine that stores every sensible conversation of a given length and looks up its next reply has the capacity to respond sensibly to anything said to it, so it meets the strongest behaviourist conception of intelligence, but its intelligence is its programmers’. So psychologism is true: whether behaviour is intelligent depends partly on how it is produced. The standard objections to behaviourism only ever refuted necessary conditions; this refutes the sufficient one. Psychologism does not require our kind of processing, only processing that is not of the most elementary sort. | Read |
| NOTE-512 | Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity | LLFC: c·f⁽ℓ⁾(αθ_A + (1 − α)θ_B) = αf⁽ℓ⁾(θ_A) + (1 − α)f⁽ℓ⁾(θ_B) for every layer ℓ and α, some c > 0. Observed whenever LMC holds (spawning or permutation), across MLP, VGG-16, ResNet-20/50. Lemma 1: LLFC ⇒ error ≤ 2ε on the path. Theorem 1: weak additivity of ReLU + commutativity (W_A H_A + W_B H_B = W_A H_B + W_B H_A) ⇒ LLFC with c = 1. Weight and activation matching minimise the two factors of (W_A − PW_BPᵀ)(H_A − PH_B). | Read |
| NOTE-513 | On Linear Mode Connectivity of Mixture-of-Experts Architectures | G(n) = ℝ^d × ℝ × S_n (common gate translation + expert permutation) preserves dense and top-k MoEs and, under genericity, is all of the gate’s symmetry (Thms 4.1, 4.2, k ≥ 2). Translation leaves barriers unchanged. Weight matching = LAP on expert order (centred gates or Gram matrices) + per-expert weight matching, O(n³ + nh³). Barriers near zero (One Billion Word: 0.0044–0.0066 vs 0.14–0.86 without expert ordering), but only the inserted MoE layer is trained; the ViT/GPT-2 backbone is shared and frozen. | Read |
| NOTE-514 | Linear Mode Connectivity and the Lottery Ticket Hypothesis | Instability = max error on the line between two copies trained from the same state under different SGD noise, minus the mean endpoint error; stable means under 2%. Networks are unstable at initialization (except LeNet) and stable after 1.5–20% of training. IMP subnetworks at extreme sparsity are matching when, and only when, they are stable; rewinding to an early iterate makes them so at ImageNet scale. | Read |
| NOTE-515 | Knowledge is a Region in Weight Space for Fine-tuned Language Models | Task vectors of RoBERTa-base fine-tuned on 12 datasets × 20 seeds cluster by dataset with 98% accuracy, and by task family (NLI, sentiment, topic) with 90%. Hull samples of a group’s fine-tuned models beat every outside model on the group’s data and beat the group’s own members in 88% (MNLI) to 96.7% (NLI family) of comparisons; extrapolation past the members fails fast. The centroid of models from other datasets, re-tuned with BitFit, is a better start than the pretrained model on 9 of 12 datasets. | Read |
| NOTE-516 | Taxonomizing local versus global structure in neural network loss landscapes | mc(θ, θ′) = ½(L(θ) + L(θ′)) − L(γ(t*)) on a trained Bezier curve, in 0-1 training error (Eq. 4); mc < 0 is a barrier, mc ≈ 0 well connected, mc > 0 unconverged ends. With Hessian top eigenvalue and trace and output CKA on mixup points, a load by temperature grid splits into phases I–IV, with IV-B (flat, connected, similar) the most accurate. Sharpness alone mispredicts across phases; with 10% label noise, the sharp phases beat flat but poorly connected phase III, and double descent appears along the Hessian transition. | Read |
| NOTE-517 | Mode Connectivity and Data Heterogeneity of Federated Learning | FedAvg, 10 clients, 200 rounds, Dirichlet label skew α ∈ {0.1, 0.2, 0.5} and PACS. Client/global low-loss overlap shrinks with heterogeneity; global models from one start under different α disagree on > 20% of inputs, are separated on the line (about 10% accuracy drop) and joined by a one-bend chain. Theory: ε_C = C e^{C T_max} max(√h(α), √h(α′))((√log N + z)/√N + √ε(√(D + log N) + z)), via mean-field dropout stability. MNIST two-layer net: dropout error < 0.01 at N = 6400, up to 0.06 at N = 200. | Read |
| NOTE-518 | Exact expressions for double descent and implicit regularization via surrogate random design | Surrogate design S^n_μ = Det(μ, K), density ∝ pdet(XXᵀ), K Poisson(γn) truncated at d. E[#rows] = n (Lemma 1); E[I − X̄⁺X̄] = (γnΣ + I)⁻¹ (Lemma 2); E[tr((X̄ᵀX̄)⁺)] = γn(1 − det((I/γn + Σ)⁻¹Σ)) (Lemma 3); λn = 1/γn. These give exact MSE in all regimes (Theorem 1) and E[X̄⁺ȳ] = (Σ + λnI)⁻¹v for n < d, with n = tr(Σ(Σ + λnI)⁻¹) (Theorem 2). For sub-Gaussian rows the i.i.d.-design MSE converges to the surrogate formula (Theorem 3), at an observed O(1/d) rate. | Read |
| NOTE-519 | A Random Matrix Analysis of Random Fourier Features: Beyond the Gaussian Kernel, a Precise Phase Transition, and the Corresponding Double Descent | Q̄ = ((N/n)(Kcos/(1+δcos) + Ksin/(1+δsin)) + λI)⁻¹ with δσ = (1/n) tr KσQ̄ is a deterministic equivalent for E_W[(ΣXᵀΣX/n + λI)⁻¹] (Theorem 1); Ē_train and Ē_test follow (Theorems 2–3). As λ → 0, δ ~ 1/λ for 2N < n and O(1) for 2N > n; det(Ω⁻¹) ~ λ, so Ē_test → ∞ at 2N = n unless the test set is within σ² ≲ λ of the training set. N/n → ∞ recovers Gaussian kernel regression. Theory matches MNIST, Fashion- and Kannada-MNIST. | Read |
| NOTE-520 | Linear Mode Connectivity in Multitask and Continual Learning | From a shared start ŵ₁, the multitask solution w₂ is linearly connected to ŵ₁ and ŵ₂, but ŵ₁ and ŵ₂ are not; the ŵ₁ → w₂ direction is nearly orthogonal to task 1’s top 50 Hessian eigenvectors, and ŵ₁ → ŵ₂ lies in their span. Different starts (CIFAR), input noise, label corruption and removed classes break the line. MC-SGD averages the loss along both lines with a one-example-per-class buffer: after 20 tasks 85.3 / 82.3 / 63.3% (Permuted MNIST / Rotated MNIST / Split CIFAR-100), multitask 89.5 / 89.8 / 68.8, Stable SGD 80.1 / 70.8 / 59.9. | Read |
| NOTE-521 | Geodesic Mode Connectivity | Length of a path in distribution space = √8 ∫ √dJSD (Fisher–Rao). Minimise the discrete energy Σ JSD(pᵢ‖pᵢ₊₁) over 25 interior models between two weight-matched ResNet-20 (4×, LayerNorm) CIFAR-10 networks, initialised on the line; the result keeps train and test loss low where the line has a large barrier, using unlabelled training images. One pair, no numbers in the text, hypothesis stated for all geodesics. | Read |
| NOTE-522 | Unveiling mode-connectivity of the ELBO landscape | Claims, without reported results, that LDA’s ELBO maxima from two SVI runs (New York Times corpus) are joined by essentially flat maximum-energy paths found by the simplified string method with natural-gradient bead updates; that the paths flatten with data size and number of topics K; and that statistical degeneracy, not only over-parameterisation (K > K*), explains it. The only figure is a schematic. | Read |
| NOTE-523 | ZipIt! Merging Models from Different Tasks without Training | Merging two differently initialised models trained on disjoint tasks: permute-then-average puts the result outside both task basins. Matching correlated features within and across models (Eq. 8) and merging only the early layers lifts CIFAR-10 (5+5) joint accuracy from 46.2 (Git Re-Basin) to 79.1 fully merged and 83.8 at 13/20 layers (ensemble 87.4). Fully merged ImageNet-1k (200+200) stays at 8.6; 10/50 layers reaches 60.9 (ensemble 63.3). Theorem 1 tightens Entezari et al.’s barrier bound by (1 − 2Γ)^{1/(2d+4)} for redundancy fraction Γ. | Read |
| NOTE-524 | Optimizing Mode Connectivity for Class Incremental Learning | In class-incremental learning (PODNet, CIFAR-100), the line between adjacent continual minima loses old-task accuracy in an interval. OPC adds a layer-wise Fourier perturbation δ_θ(λ) with δ(0) = δ(1) = 0 to the line, minimises old-task memory loss before a switching point λ* and new-task loss after it, with cylinder noise for flatness; EOPC averages 10 points near λ*. PODNet + EOPC: CIFAR-100 66.68 / 64.94 / 62.36 (5 / 10 / 25 steps) vs 65.47 / 63.13 / 59.85; forgetting 7.68 / 8.64 / 12.02 vs 19.26 / 25.01 / 28.55. | Read |
| NOTE-525 | Does Data Scaling Lead to Visual Compositional Generalization? | (n, k) framework: two concepts with n values, k seen combinations per value. From-scratch ResNet-50s: near-100% ID, large OOD drops; 4× ID data leaves 60–80% drops; more values or combinations help. Features go from spurious, to decodable but entangled, to linearly factored (R² > 0.8, cosine < 0.1–0.2). Proposition 4.1: with linearly factored embeddings spanning 2n − 1 dimensions, k = 2 suffices for a linear classifier to be correct on all (n − k)·n unseen pairs. DINO, DINOv2, CLIP: partly factored, above chance, not perfect. | Read |
| NOTE-526 | TIES-Merging: Resolving Interference When Merging Models | Task vector τ = θ_ft − θ_init. TIES keeps the top 20% of each τ by magnitude, sets γ_m = sgn(Σ τ̂_t), averages only entries agreeing with γ_m, and returns θ_init + λ τ_m. Validated: (IA)³ 66.4 vs 63.9 for task arithmetic; T5-base 73.9 vs 73.2; T5-large 76.9 vs 73.3; ViT-B/32 73.6 vs 71.8 (RegMean); ViT-L/14 86.0 vs 84.5. Unvalidated T5-base 69.7 vs 73.2. Oracle multitask signs give 72.0 against 73.1 for multitask training. | Read |
| NOTE-527 | Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs | Curve-finding: minimize E_{t∼U(0,1)} L(φ_θ(t)) over the bend θ of a one-bend chain or quadratic Bezier curve between two trained networks. The found curves keep loss and error near the endpoints where the line segment reaches near-chance error (Table 2). Points at t ≥ 0.4 already ensemble as well as the far endpoint. FGE, short cyclical learning rates late in training, beats Snapshot Ensembles at equal budget (Table 1). | Read |
| NOTE-528 | Understanding Mode Connectivity via Parameter Space Symmetry | Minimum of full-rank ‖Y − W_l…W₁X‖² ≅ GL_h(ℝ)^{l−1}: 2^{l−1} components, independent of width. 1-D three-layer: 4 components, 3 with a skip. h ≥ 2: permutations connect all components. Homogeneous last layers: rescaled minima in one component with unbounded midpoint loss, even under last-layer permutations (Props. 5.3–5.4). Symmetry curves γ(t) = exp(t log g)·w have constant loss; curvature ≤ κ bounds the chord’s distance from the level set by (1/κ)(1 − √(1 − (κ‖Δw‖/2)²)) ≈ κ‖Δw‖²/8. | Read |
| NOTE-529 | Unveiling Mode Connectivity in Graph Neural Networks | GCN node-classification solutions from different seeds show linear loss barriers that vary by graph (small on Cora, Citeseer; large on Coauthor-CS, Squirrel) and vanish along fitted quadratic Bezier curves. GCN, GraphSAGE and GAT give similar barriers, MLPs different ones. On CSBM graphs barriers fall with density and feature separability and peak at middling homophily. Stated without proof: a barrier bound through the spectral gap, a generalisation bound proportional to the barrier, and a Wasserstein distance between graphs’ interpolation-loss curves that bounds the domain-adaptation gap. | Read |
| NOTE-530 | Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness | Bezier path (Garipov) trained on 50–2,500 clean samples between two backdoored or error-injected models: models near the ends keep most clean accuracy and lose the attack (CIFAR-10 VGG, 2,500 samples, t = 0.1: 88% clean, 1.1% backdoor; injection 0% throughout). Standard-loss paths between regular and adversarially trained models are flat in standard loss but have a PGD robustness-loss barrier, correlated with λ_max of the input Hessian (PCC 0.71–0.88). Proposition 1’s proof assumes the loss is constant along the path in a way that, if true for all inputs, would remove the barrier. | Read |
| NOTE-531 | Essentially No Barriers in Neural Network Energy Landscape | AutoNEB finds paths between minima whose highest training loss is almost the minima’s for ResNets and DenseNets on CIFAR-10, with a small gap on CIFAR-100; test error rises at most 0.5% (CIFAR-10) and 2.2% (CIFAR-100) per §4.3, or 0.7% and 2.9% on the ResNets per Appendix B. Concatenating paths bounds every pair through a minimum spanning tree. Barriers shrink with width and depth. | Read |
| NOTE-532 | Git Re-Basin: Merging Models modulo Permutation Symmetries | Weight matching, coordinate descent over per-layer linear assignments for an NP-hard objective, aligns independently trained nets in seconds. After alignment, MNIST MLPs and wide CIFAR-10 ResNet-20s have zero loss barrier; 1×-width models and ImageNet ResNet-50 do not, and LMC appears only during training. A two-unit counterexample shows no permutation can connect some perfect solutions, so LMC is a property of SGD solutions. | Read |
| NOTE-533 | Do Deep Neural Network Solutions Form a Star Domain? | Conjecture 2: for width > αh (h the width at which SGD solutions are convex modulo permutation), the solution set is a star domain modulo permutation. Starlight (sample θ_n ∈ Z, t ∼ U[0,1], step on L((1−t)θ + tπ(θ_n)), re-permute by weight matching each epoch) finds θ⋆ whose train-loss barriers to held-out models are about 2–5× lower than regular–regular ones across 8 settings, falling with |Z| up to 50. Not zero; star loss itself can be high (DenseNet CIFAR-100 0.635 vs 0.006). BMA on the star domain: better AUROC, worse ECE than an ensemble. | Read |
| NOTE-534 | Mode Connectivity Beyond Classifiers: Evidence from Generative and Contrastive Models | One pair of DDPMs (Flowers102) and one pair of NanoCLIP models (Flickr30k) are joined by chains built by staged layer moves, per-layer variance correction and AdamW retraining to a loss threshold. DDPM training loss stays below 0.04 (AutoNEB peaks at 0.17), NanoCLIP below 0.09 (AutoNEB 6.69, accuracy 0.05). Ablations: all-layers, forward-order and SGD variants all raise the average loss. Intermediate DDPMs generate worse (higher FID) at endpoint-level training loss. | Read |
| NOTE-535 | Exploring Neural Network Landscapes: Star-Shaped and Geodesic Connectivity | Two-layer ReLU student of an orthonormal M-neuron teacher: the minimum manifold is explicit (Theorem 6); two random minima are 2-PL connected w.p. ≥ 1 − M((M²−1)/M²)^{m−2M}; any two are 4-PL connected for m ≥ 2M − 1; k minima share a linearly connected centre w.p. ≥ 1 − M((M^k−1)/M^k)^{m−kM}. Normalised geodesic distance is O(√M) for uniform minima but ≤ 1 + c₂/(r√m) for r-sparse ones. Deep linear nets: 2-PL a.s., 3-PL always, star centre a.s. for m > 1 + r(L − 1). Centres found empirically for FNN, VGG16, ResNet on MNIST/CIFAR-10 without permutation. | Read |
| NOTE-536 | The Generalization Ridge: Information Flow in Natural Language Generation | I(Zℓ; Y) = H(G_Z) + H(G_Y) − H(G_Z ∘ G_Y), with H(G) = −tr(G log G) on trace-normalized Gaussian Gram matrices (bandwidth 1) of ℓ2-normalized last-token states and target-token embeddings. It rises, peaks at layers 10/12 (GPT-2-S, synthetic: 0.2721) and falls (0.0188 at layer 12), where OOD early-exit accuracy peaks (56.90%) and ID accuracy keeps rising (93.17%). Middle residual updates add most; OOD-fitted residual scales downweight the last layers. The profile’s shape persists over 50 decoding steps. | Read |
| NOTE-537 | The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks | Conjecture 1: wide SGD solutions can be permuted into one linearly connected basin. Theorem 3.1 proves an output-level version only for a one-hidden-layer net at uniform random init, rate Õ(h^{−1/(2d+4)}). The evidence is that independent solutions S and permutations S′ of one solution show the same barriers across width, depth and data; simulated annealing lowers barriers only for shallow nets on easy data. | Read |
| NOTE-538 | Mechanistic Mode Connectivity | Mechanistic similarity = invariance to the same unit interventions on the latents of the data-generating process. Cue-reliant and cue-invariant ResNet-18 and VGG-13 models are quadratic- but not linear-connected, even after activation matching. Conjecture 1 (no LMC up to symmetry ⇒ mechanistic dissimilarity) is not what Appendix F.3 proves. For a one-hidden-layer ReLU net with interpolating minimizers, Lemma 2 (LMC forces identical activation patterns) with Theorem 1 (mechanisms of different complexity force different patterns) proves the converse: dissimilar mechanisms of different complexity ⇒ no LMC under any permutation. The paper presents this as verifying the conjecture. CBFT (cross-entropy + barrier loss + class-mean invariance loss) removes the cue on synthetic CIFAR-10/100 and Dominoes. | Read |
| NOTE-539 | Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling | SPRO: train connecting vertices to minimize mean loss at 5 uniform samples in a simplicial complex minus λ log volume. It joins up to 7 modes in one complex and finds a low-loss region of at least 10 dimensions for VGG-16 on CIFAR-10 (volume collapses at the 11th vertex). ESPRO, simplexes grown from single modes at about 10 epochs per vertex, beats deep ensembles in error, calibration and corruption robustness. | Read |
| NOTE-540 | Generalized Linear Mode Connectivity for Transformers | Symmetry hierarchy S1 permutation ⊂ S2 semi-permutation ⊂ S3 orthogonal ⊂ S4 invertible. Transformer alignment = P_FF per layer, semi-permutation P̃_H over heads (matched on QK and OV circuits), one orthogonal O on the residual stream after LayerNorm → RMSNorm. Test-loss barriers (Table 2): vanilla 1.69–4.34, weight matching 0.34–1.56, learned matching 0.00 (ViT ×3), 0.02 (Tiny Shakespeare), 0.42 (BookCorpus); permutation-only learned 0.29–1.60. Learned matching mainly refines O. Small models only. | Read |
| NOTE-541 | Intermediate Layer Classifiers for OOD generalization | ILC_l(x) = W_l r_l(x) + b_l on frozen layer-l features, l ≤ L − 2, layer chosen on OOD validation data. Few-shot, best ILC minus last-layer retraining: +16.8, +6.3, +1.6, +6.3 points (ResNets on CMNIST, CIFAR-10, CIFAR-100C, MultiCelebA); +27.0 on MultiCelebA at π ≤ 0.03. Zero-shot worst-group accuracy: Waterbirds 79.4 → 87.1, CelebA 56.0 → 82.0, MultiCelebA 20.8 → 46.0. Layer 7 beats 8 everywhere in Figure 8. Intermediate features are less displaced by shift for minority groups. | Read |
| NOTE-542 | Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior | VSDL model: f = f(x; α, τ), α = m_eff/N lowered by label noise, τ raised by early stopping. Thermodynamic limit m, N → ∞ with α = m/N fixed. Continuous perceptron: s(ε) ~ ln ε, so 1/ε − α = 0 gives ε ~ 1/α (Eqs. 6–8). Ising perceptron: s(ε) ~ −(π²/2)ε² ln ε, and −π²ε ln ε = α has no solution for large α, so ε falls discontinuously to 0 at αc (Eqs. 9–13). Rigorous bound: error ≤ ε* + ε_τ, ε* the rightmost crossing of s(ε) and −α log(1 − ε) (Eqs. 14–18). No new theorem, no experiment. | Read |
| NOTE-543 | Proving Linear Mode Connectivity of Neural Networks via Optimal Transport | Layerwise neuron alignment equals a Wasserstein distance between empirical weight distributions (Birkhoff), so LMC modulo permutation follows when neuron weights are i.i.d. and layers are wide: for mean-field two-layer SGD (Theorem 3.1), and for deep Gaussian or sub-Gaussian nets with m̃_ℓ = Õ((T_ℓ/ε)^{m̃_{ℓ−1}}) (Theorem 5.2), tight by Theorem 5.3. Low-dimensional weights relax this (Theorem 5.4). | Read |
| NOTE-544 | Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets | ε-dropout-stable ReLU nets are ε-connected by a piecewise-linear path (Theorem 1); ε-noise-stable nets by 10 segments with barrier Õ(ε) (Theorem 2), or 13 through a narrow low-loss net (Theorem 3). For any width there is a teacher dataset on which a two-layer student’s global minima are disconnected (Theorem 4). MNIST convnets and a VGG-11 on CIFAR-10 satisfy the conditions roughly. | Read |
| NOTE-589 | Conditional Simulation Using Diffusion Schrödinger Bridges | Posterior sampling p(x|y) when only joint samples (X, Y) can be drawn, recast as a Schrödinger bridge on the extended space between the joint p(x, y) and a reference p_ref(x|y)p(y) in which y is held fixed (Proposition 1), and solved by iterative proportional fitting with learned forward and backward drifts (CDSB). Its first iteration is the ordinary conditional score-based model, so the rest refine it; the reference can be an informed guess (upsampled image, EnKF, SRFlow) rather than noise. Better than the conditional score model at equal, small numbers of steps on most but not all image metrics, and it filters Lorenz-63 with 20 steps where the score model diverges. | Read |
| NOTE-592 | T2I-CompBench | 6,000 compositional prompts (attribute binding by colour, shape and texture; spatial and non-spatial relations; complex), each sub-category split 700/300, with a metric per category: question-per-object BLIP-VQA for binding (Kendall τ 0.63 with humans on colour against 0.19 for CLIPScore), a UniDet box rule for space, and a 3-in-1 average for complex prompts. Spatial relations are hardest. GORS, reward-weighted LoRA finetuning of SD v2 on its own selected samples, is rated best by humans in every sub-category, but its binding score falls sharply on pairs unseen in finetuning. | Read |
| NOTE-593 | GenEval | An automated text-to-image benchmark of 553 templated prompts in six tasks, scored per image as correct or not by a COCO-trained Mask2Former (presence, count, position) and masked-crop zero-shot CLIP (colour). 83% agreement with human annotators against 88% between annotators, and 91% on unanimous images; beats a per-task tuned CLIPScore on counting, position and binding. Best 2023 open model (IF-XL) scores 0.61 overall, at most 0.15 on position and 0.35 on attribute binding. | Read |
| NOTE-597 | Compositional Visual Generation with Composable Diffusion Models | Treats a conditional diffusion model’s noise prediction as the gradient of an energy, so concepts compose at sampling time by adding scores: AND is ε(x,t) + Σ wᵢ(ε(x,t|cᵢ) − ε(x,t)), the product p(x)Πp(x|cᵢ)/p(x) under conditional independence of the concepts given x; NOT divides by the negated concept’s likelihood. No retraining. On CLEVR object positions it beats an EBM baseline (31.36% against 7.34% at three components); on relations it does not (2.80% against 4.26%); on FFHQ attributes it trails LACE in accuracy at three components but has the best FID. Text composition with GLIDE and Stable Diffusion is qualitative only. | Read |
| NOTE-607 | Rethinking the Role of Demonstrations | Shows empirically that the correctness of in-context labels contributes little to classification and multi-choice accuracy, while the input distribution, the label space and the paired format contribute most of the gain over zero-shot. The authors read this as the demonstrations locating a correspondence the model already has, not teaching a new one. | Read |
| NOTE-610 | T2I-CompBench++ | Extends T2I-CompBench to eight sub-categories (adding numeracy and 3D spatial relations), tests multimodal LLMs as judges, and benchmarks eleven models. GPT-4V agrees with human rankings best on interactions and complex prompts, where CLIPScore and the conference version’s 3-in-1 metric were weak; specialised VQA and detector metrics stay best elsewhere. Human ratings cover only six 2023-era models, and the interaction category remains near ceiling for humans. | Read |
| NOTE-620 | Learning to (Learn at Test Time) | Recasts a sequence-modelling layer as a learner trained on the sequence it reads: hidden state as inner-model weights, update as a gradient step on a learned self-supervised loss. Proves that the linear, batch-GD case is linear attention and that a Nadaraya–Watson inner learner is softmax attention; shows TTT-Linear and TTT-MLP use long context better than Mamba at up to 1.3B parameters. | Read |
| NOTE-625 | Backprop as Functor | Learners (P, I, U, r), with a request function r : P × A × B → A, compose and form a symmetric monoidal category Learn. For step size ε and an error e with ∂e/∂x(x₀, −) invertible, sending a differentiable parametrised function to (P, I, p − ε∇ₚE, f_a(∇ₐE)) is a faithful strong symmetric monoidal functor Para → Learn: gradient descent on a composite is the composite of local updates exchanging requests, which is backpropagation, and the proof is the chain rule. | Read |
| NOTE-626 | Fantastically Ordered Prompts | Shows that permuting a fixed set of few-shot examples swings GPT-family accuracy from near chance to near state of the art, at every model size tested, with good orders uncorrelated across sizes and the variance surviving more examples and calibration; proposes ranking orders by the label entropy they induce on a probing set the model generates itself. | Read |
| NOTE-633 | Implicit Bayesian inference account of in-context learning | Proves that if pretraining text is a mixture of HMMs and a model fits it exactly, conditioning on a prompt of concatenated examples recovers the prompt’s latent concept by posterior concentration, and the in-context predictor is asymptotically Bayes-optimal, whenever each example’s KL signal about the concept beats the error from the improbable transitions between examples. The account is of the data distribution, not of any network’s computation. | Read |
| NOTE-639 | Attend-and-Excite | Identifies catastrophic neglect in Stable Diffusion, where a named subject never appears because no image patch attends to its token, and corrects it at inference by gradient steps on the latent that raise the smoothed peak attention of the most neglected subject token during the early steps. Measured gains are in CLIP and caption similarity and human preference on two-subject prompts; attribute binding improves only in the figures, and relations are not addressed. | Read |
| NOTE-648 | Quantum-Like Contextuality in Large Language Models | Builds a three-sentence anaphora schema with the support of the PR prism, instantiates it 51,966,480 times from Simple English Wikipedia, and takes one referent probability per sentence from BERT: 0.148% of the models pass a signalling-corrected sheaf test and 71.1% the CbD test. Because the schema fixes the support, both verdicts depend only on how uncertain BERT is in the three sentences; the tie to embedding distance is a softmax identity with weak correlations. | Read |
| NOTE-649 | Large AI models are cultural and social technologies | Argues that large models are a cultural and social technology, not intelligent agents: lossy, uninvertible summaries of human-produced corpora that, like prices and bureaucratic categories, allow the information to be reorganized at scale. The argument is by analogy and classification; no evidence is offered that could have come out otherwise. It names one mechanism (fitting the training distribution on average makes models worst where data are rare, which may homogenize culture) and leaves both it and its remedy untested. | Read |
| NOTE-652 | Relative representations enable zero-shot latent space communication | Represents each sample by its cosine similarities to a fixed set of anchor samples. That vector is invariant by construction to every cosine-preserving map of the latent space (rotations, reflections, global and even per-sample positive rescaling), not to translations or general linear maps. With anchors matched across encoders it lets a decoder trained on one encoder run untrained on another, across seeds, architectures, latent sizes and languages: far better than raw latents, but with large and uneven losses. The assumption that such spaces differ by an angle-preserving map is never tested directly, and the evidence supports it only approximately. | Read |
| NOTE-653 | Revisiting Model Stitching to Compare Neural Representations | An empirical paper with no theorems. It defines the stitching penalty, the change in a top network’s test error when its bottom ℓ layers are replaced by another representation through a trained 1×1 convolution, and uses it to show, in single runs mostly on CIFAR-10, that networks from different seeds, and supervised and self-supervised ImageNet networks, are interchangeable layer by layer where CKA calls them different, and that a bottom trained on more data improves a weaker top. The ordering it gives is by one task’s loss through a fixed head and a restricted map class, which is not an ordering by information. | Read |
| NOTE-658 v2 | Exact Learning Dynamics of In-Context Learning in Linear Transformers | Solves the averaged gradient-flow dynamics of one linear-attention layer trained on in-context regression, when tasks and input covariance share an eigenbasis and the weights are assumed to stay in it: each mode is a logistic curve on timescale (ηP s_α²)⁻¹, so learning is staged by the input spectrum, and the fixed point is the known preconditioned predictor W Σ̂_x [((N+1)/N)Σ_x + (Tr Σ_x/N)I]⁻¹. A scaling symmetry conserves ‖p₂‖² − ‖q₁‖². The transfer to non-linear transformers and grokking is a set of timing coincidences in spectral measures. | Read |
| NOTE-660 | Closed-Form Training Dynamics in Word2Vec-like Models | Proves that word2vec’s loss, approximated to fourth order and reweighted so that every pair carries the same aggregate weight, is unweighted factorisation of M*, the relative deviation of co-occurrence from independence, whose rank-d minimisers are its top eigenvectors; derives, without full rigour, that gradient flow from small random initialisation learns those eigenvectors one at a time in sigmoidal steps; and shows on Wikipedia that the predicted embeddings match word2vec’s features and scores better than truncated PMI or PPMI. Its account of analogies is an empirical spiked-random-matrix fit. | Read |
| NOTE-661 | Single-Head Attention in High Dimensions | In a solvable model, one tied softmax attention layer trained by ridge-penalized ERM on Gaussian sequences with n ∝ d² samples, derives (non-rigorously, by AMP and random-matrix theory) the test error, interpolation and recovery thresholds, and the spectrum of the learned query–key map, η·ReLU(S0 + δZ − ε): outliers are recovered target directions, the bulk is finite-sample noise, and weight decay acts as a nuclear norm. Power-law targets are recovered mode by mode, giving heavy-tailed spectra and LASSO-rate scaling laws. | Read |
| NOTE-662 | Towards Understanding Grokking | In a model that decodes the sum of two learned embeddings, a held-out pair is answered correctly when its embedding sum coincides with a trained pair’s, and the training set’s same-answer pairs force such coincidences. Grokking’s critical training fraction is then where those equations first fix the embedding up to translation and scale, which a posited effective loss predicts for addition with p = 10. Threshold-defined phase diagrams over decoder learning rate and weight decay put grokking between comprehension and memorization, in the toy and, qualitatively, in a transformer and on MNIST. | Read |
| NOTE-663 | Topological Invariance and Breakdown in Learning | Proves that a permutation-equivariant update with a K-Lipschitz update map changes the distance between any two neurons in one step by a factor in [1 − ηK, 1 + ηK], so coincident neurons never split, and for ηK < 1 the induced map on the set of neuron vectors is a bi-Lipschitz homeomorphism (a C¹ diffeomorphism under smoothness), and so no two neurons merge in finitely many steps. The proofs hold. The two-phase “topological simplification” reading and the Betti-number experiments go beyond them. The measured changes are fragmentation, not merging, at ηλ_max far below 1. | Read |
| NOTE-664 | Symmetry in Language Statistics and Representation Geometry | Proves that if co-occurrence among words on a latent continuum depends only on their separation, a full-rank spectral word embedding places them on Fourier modes of the latent coordinate, with amplitudes from the kernel’s Fourier transform, and solves the exponential kernel exactly for periodic and open boundaries. The symmetry is measured in Wikipedia, the geometry seen in Gemma 2 by PCA, and its robustness to deleting the words’ mutual co-occurrences explained by a latent variable shared by many words. | Read |
| NOTE-665 | On the Emergence of Linear Analogies in Word Embeddings | Derives the parallelogram analogies of word embeddings from a generative model of co-occurrence in which words are bundles of independent binary attributes: the co-occurrence ratio is then a Kronecker product, and its logarithm, the PMI, is a bilinear form of rank at most d + 1 whose eigenvectors are affine in the attributes, so a spectral PMI embedding is a linear image of the attribute hypercube. Robustness to noise, pruning and deletion of a relation’s own pairs is argued from eigenvalue bounds and simulated; Wikipedia agrees in shape. Every embedding is an eigendecomposition, none is trained. | Read |
| NOTE-666 | A mathematical theory of semantic development in deep neural networks | Shows that a two-layer linear network trained from small weights learns the singular modes of its input–output correlations in order of strength, each by a sharp sigmoidal transition, where a shallow network learns all modes together; and that the singular structure of data generated on trees, rings and clusters, plus a minimum-norm property, accounts for coarse-to-fine differentiation, transient errors, narrowing generalisation, typicality, coherence thresholds and conserved representational similarity. Exact only from decoupled, balanced initial weights with white inputs; random initialisation is simulated. | Read |
| NOTE-667 | Exact solutions to the nonlinear dynamics of learning in deep linear neural networks | Derives the coupled gradient-descent equations of a linear network with hidden layers and whitened inputs, and solves them exactly from decoupled starts: one sigmoid per singular mode of the input–output correlation, with conserved differences of squared layer strengths; balanced or not with one hidden layer, from equal layer strengths at any depth. Depth slows learning only through the initial end-to-end mode strength, so from strength of order one the iteration count stays finite as depth grows when the step size is scaled down with depth. Argues, with MNIST and random-matrix simulations, that orthogonal layers or suitable pretraining give such starts and scaled Gaussian layers do not, and that orthogonal tanh networks near gain 1 keep their Jacobians near-isometric. | Read |
| NOTE-669 | Hyperbolic Representation Learning: Revisiting and Advancing | Shows, by tracking each node’s hyperbolic distance to the origin, that hyperbolic graph models trained on task losses do not by themselves place a tree’s root nearest the origin or spread its levels outward: on synthetic 3-ary trees the root sits well off the minimum and level order is recovered for only 69–75% of node pairs. A centroid recentring plus an outward-push loss (HIE), using no level information, raises this to 75–84% and improves link prediction and node classification, most on the least tree-like graphs. | Read |
| NOTE-671 | Hyperbolic Entailment Cones for Learning Hierarchical Embeddings | Nested, axially symmetric, rotation-invariant cones in the Poincaré ball define a partial order, but nesting forces apertures of at most π/2 and r sin ψ(r)/(1 − r²) non-increasing. So no cone exists near the origin, and the widest family is sin ψ = K(1 − r²)/r, with a closed-form membership angle. The transitivity of that family is asserted, not proved. Learned on WordNet noun hypernymy, the cones predict held-out closure edges better than Poincaré and order embeddings once some closure edges are in training. Trained on the transitive reduction alone, every method fails. | Read |
| NOTE-674 | Mutual information, neural networks and the renormalization group | Makes “the relevant degrees of freedom” of real-space RG an optimizable quantity: the coarse variable of a block that carries the most mutual information about the system beyond a buffer. Learned from Monte Carlo samples, it recovers Kadanoff block spins for 2D Ising and the electric-field variables of the dimer model, discards decoupled noise, and, iterated, gives the Ising flow and ν ≈ 1.0 ± 0.15, while a distribution-fitting RBM keeps the noise. The equivalence with a short-ranged effective Hamiltonian is shown only for 1D and quasi-1D. | Read |
| NOTE-675 | Hyperbolic Diffusion Embedding and Distance | Embeds each point’s diffusion densities at dyadic time scales in a product of Poincaré half-spaces and proves that the ℓ1 product distance, a scale-weighted Hellinger sum, is equivalent up to constants to a thresholded snowflake min{1, d_T^(2α)} of a latent tree metric when the diffusion operator behaves like the heat kernel on that tree. The tree heat-kernel bounds are cited rather than proved and one Hölder step holds only for d_T ≥ 1. Empirically it keeps tree neighbourhoods (best MAP) and not global distances (more distortion than hyperbolic MDS). | Read |
| NOTE-677 | The Random Hierarchy Model | Introduces the Random Hierarchy Model, a random context-free grammar in which each symbol has m synonymic expansions, and shows by experiment that deep feature-learning networks learn it from about n_c m^L examples, polynomial in the input dimension, while shallow and lazy networks need an exponential number. That size coincides with the emergence of synonym-invariant hidden representations and, by a scaling argument, with the size at which the class statistics of input patches, equal for synonyms, exceed sampling noise; removing those statistics makes deep networks fail as well. | Read |
| NOTE-679 v2 | Representation Tradeoffs for Hyperbolic Embeddings | Trees embed in the hyperbolic plane with worst-case distortion 1 + ε, but because a hyperbolic distance d takes about d bits to write, the coordinates need Θ((ℓ/ε) log deg_max) bits in the plane: cheap in branching, expensive in depth. More dimensions divide the upper bound by r for bushy trees. For general metrics, an exact hyperbolic MDS by PCA on cosh of the distances after centring at a new mean, with a perturbation bound growing as sinh² of the diameter; hyperbolic PGA has spurious local minima and only a local-convexity guarantee. | Read |
| NOTE-681 v2 | Statistical Physics through the Lens of Real-Space Mutual Information | Makes the real-space mutual-information principle a single-shot operator finder: a CNN coarse-graining co-trained against a neural InfoNCE bound, optimized once at a large buffer, returns on the interacting dimer model the order parameters below the BKT transition and the electric fields above it, and one fitted scaling dimension (1.00037 against 1). The maximal information is log 4 in the ordered phase and decays algebraically with the buffer in the critical one. The model’s operators were known; the optimality theorem is cited. | Read |
| NOTE-683 v2 | An exact mapping between the Variational Renormalization Group and Deep Learning | Identifies Kadanoff’s variational RG kernel with an RBM energy by T = −E + H, so that the coarse-grained Hamiltonian is the RBM’s hidden-marginal Hamiltonian for any parameters and an exact RG step is a perfect fit of the data. The identity is correct but definitional, and the paper concedes the approximate schemes differ; its evidence that trained networks do RG is qualitative. LIT-873 is right that fitting the distribution need not give RG, though it does not engage the identity. | Read |
| NOTE-684 | The Sparse Random Hierarchy Model | Adds sparsity to the Random Hierarchy Model, so that labels are unchanged by discrete displacements of the informative features, and shows by experiment that deep networks learn it from a number of examples polynomial in the input dimension, (s0 + 1)^L n_c m^L up to a factor s^(L/2) without weight sharing and (s0 + 1)^2 n_c m^L with it, and that hidden layers become insensitive to synonym swaps and to feature displacements at about the same training-set size as test error falls. That coincidence is offered as the reason deformation stability tracks performance on images, which is not tested on images. | Read |
| NOTE-685 | A Phase Transition in Diffusion Models | For the Random Hierarchy Model with known rules, exact denoising is belief propagation; an annealed mean-field map, checked against exact BP, shows that as depth grows the class is recovered below a threshold noise and lost above it, provided the grammar is sparse (s f < 1), while low-level features degrade smoothly and persist past the threshold. The same top-versus-bottom crossing appears in classifier layers under forward–backward runs of an ImageNet diffusion model, and not for a Gaussian mixture. | Read |
| NOTE-686 | Hyperbolic Neural Networks | Möbius gyrovector operations give the Poincaré ball closed-form exponential and logarithmic maps, parallel transport from the origin as a scaling, and geodesic hyperplanes with a closed-form distance; from them come hyperbolic logistic regression, feed-forward layers and GRUs that reduce to the Euclidean ones as c → 0. Because linear maps and nonlinearities are lifted through the origin’s tangent space, the curvature acts only through biases and output distances. Gains over Euclidean models are confined to a synthetic tree-like task and to separating WordNet subtrees in low-dimensional Poincaré embeddings. | Read |
| NOTE-687 | How the structure of language is acquired | Extends the Random Hierarchy Model to predicting a masked last token. The correlation between two tokens shrinks by about a factor m per level of their common ancestor, a sample of P sequences resolves it only above a (v²P)^(−1/2) noise floor, and deep transformers and CNNs show loss steps at P_ℓ ≈ v m^(2ℓ−1), where their layers come to encode the level-ℓ hidden symbol. On character-level text, the correlation decay exponent predicts where the loss for a given context length levels off. The route from correlations to learned hidden variables is argued, not proved. | Read |
| NOTE-688 v2 | Why does deep and cheap learning work so well? | Argues that cheap networks suffice because physical data have low-order, local, symmetric Hamiltonians, and that depth pays because inference must compose the sufficient statistics of a Markov generative hierarchy; proves that one hidden layer needs exactly 2^n neurons to multiply n inputs. On renormalization it is right that a coarse-graining needs a stated target, and its early appendix shows that neither a matched partition function nor the exact trace condition supplies one; it is wrong that the target must come from supervision. | Read |
| NOTE-689 | Deriving neural scaling laws from language statistics | Predicts the data-limited scaling exponent of language-model test loss as γ/(2β), from two measured corpus statistics: the decay with lag of the top singular value of the token–token covariance (β), which sets the usable context n*(P) ~ P^(1/(2β)), and the decay of next-token conditional entropy with context (γ). The derivation rests on assumed scaling forms and a fast-learning condition; on TinyStories and WikiText-103 transformers match the predicted exponents up to about 10^8 tokens and horizons of a few tens of tokens. | Read |
| NOTE-691 | Probing Latent Hierarchy via Diffusion | In the Random Hierarchy Model with known rules, the tokens changed by noising and Bayes-optimal denoising form blocks set by the depth of the latent that changed; an annealed mean-field calculation, checked against exact belief propagation at one parameter set, gives a correlation length diverging as |ε − ε*|^(−ν), ν = log s / log F′(p*), at the class threshold, and a susceptibility peak there, which a Gaussian field with power-law spectrum lacks. Masked text diffusion and ImageNet diffusion show a susceptibility peak at an intermediate noise, over about one decade of distance and with no non-hierarchical control. | Read |
| NOTE-692 v2 | Symmetries and phase diagrams with real-space mutual information neural estimation | The long companion of LIT-878. Derives the RSMI-NE estimator and shows that the maximal real-space information counts ordered sectors (1 bit for the Ising ferromagnet, 2 for columnar dimers), decays exponentially with the buffer off criticality and algebraically at it, and that the ensemble of optimal filters from independent runs maps a degenerate subspace whose projections show broken lattice symmetries and the dimers’ emergent U(1). All the models’ answers were known; the non-equilibrium example is one figure. | Read |
| NOTE-693 | The power of deeper networks for expressing natural functions | Proves that a single hidden layer of smooth, bias-free units needs exactly ∏(rᵢ+1) neurons to Taylor-approximate x₁^r₁⋯xₙ^rₙ while a deep network needs O(Σ log rᵢ), extends the gap to sparse and univariate polynomials, and bounds a product of n inputs at depth k by about 2^(n^(1/k)). Its uniform-approximation lower bounds rest on an unproved step, the depth-k tightness is a conjecture, and the gap is set by degree, so bounded-degree polynomials show none. | Read |
| NOTE-694 | Comment on “Why does deep and cheap learning work so well?” | Schwab and Mehta concede that a preserved free energy does not give a preserved distribution and that the “iff” of LIT-882’s Eq. 8 should be one-directional, and show that Lin and Tegmark’s v1 counterexample violates the trace condition, so the mapping’s exact step stands. Right against v1; it places all of the mapping’s content at the trace condition, and says nothing to the later point that the trace condition does not select the coarse variables. | Read |
| NOTE-705 | Orseau, McGill & Legg — Agents and Devices | A formal counterpart of Dennett’s physical and intentional stances. A device is a mixture of input-output mappings; an agent is a mixture of goal-optimizing ε-greedy policies, by inverse RL. Bayes’ rule gives the posterior probability that a system is an agent. It is illustrated in a toy gridworld, with no human comparison. Agency is explicitly relative to the observer’s prior. A clearly identified system is assumed. | Read |
| NOTE-707 | Kenton et al. — Discovering Agents | An interventional, causal definition of agents. Mechanised causal models add a mechanism variable for each object-level variable. Decision nodes are found as variables whose mechanisms respond to downstream mechanisms (terminal edges), and utility nodes as the targets of those responses. Recovery of the true game graph is proved under five assumptions, including interventional data on mechanisms. The verdicts are sensible. The labels are frame-dependent and granularity-sensitive, which the paper states but does not treat as a threat. | Read |
| NOTE-711 | The Limiting Dynamics of SGD | SGD with momentum β, learning rate η, batch size S and weight decay λ is modelled as (η/2)(1 + β)θ̈ + (1 − β)θ̇ + λθ = −g(θ) + √(η/S) √Σ ξ(t). For linear regression with Σ = σ²H this is an OU process in phase space. Its stationary state is Gibbs on a modified loss, with inverse temperature κ = S(1 − β²), in broken detailed balance. Under a quadratic loss at the end of training the same holds for deep networks, and a ResNet-18 on ImageNet shows the predicted isotropy, velocity regularization, oscillations and speed. | Read |
| NOTE-712 | Symmetry in Neural Network Parameter Spaces | A symmetry is a group action on parameters preserving the network function (Def. 2.4) or the loss (Def. 2.6), possibly only on given data or in expectation (Defs. 2.8–2.9). Common components have GL, scaling, sign-flip, orthogonal, translation and permutation symmetries (Table 1). Continuous symmetries make minima connected manifolds, and give gradient flow conserved quantities that label trajectories and endpoints (§5). SGD’s finite steps, noise and weight decay break these, which is an open route to implicit bias (§7.2). | Read |
| NOTE-713 | Neural Mechanics | A symmetry ψ of the loss gives ⟨∇L, ∂_α ψ⟩ = 0 and a Hessian constraint. Under gradient flow, Theorem 1 conserves ⟨θ, ∂_α ψ⟩: for translation ⟨θ_A, 1⟩, for scale |θ_A|², for rescale |θ_A1|² − |θ_A2|². With weight decay λ and the modified loss L + (η/4)|∇L|², these become exponentially decaying memories plus integrals of gradient norms (Eqs. 18–20), matched on VGG-16 with Tiny ImageNet. The noise covariance annihilates the generators (§5, App. F). | Read |
| NOTE-714 | Noise Balance and Stationary Distribution of Stochastic Gradient Descent | SDE: dθ = −∇L dt + √(T C(θ)) dW, with T = η/S. With ℓ(u, w) = ℓ(λu, w/λ), d(‖u‖² − ‖w‖²)/dt = −T(uᵀC₁u − wᵀC₂w) (Theorem 1). The stationary point is Tr C(w) = Tr C(u). With weight decay γ the rate is TC₀ + γ (Corollary 2.1). Diagonal linear network ℓ = (Σᵢ ∏ₖ u_i^{(k)} x − y)²: sign coherence and balance for depth D > 1 (Theorem 3), exact stationary distribution (Theorem 4), collapse above T_c = β₂/α₃ for D = 1, tail ∝ v^{−5+3/(D+1)} independent of data. | Read |
| NOTE-715 | From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks | Two-layer linear network ŷ = W₂W₁x, MSE, full-batch gradient flow. Assumptions: whitened inputs (A1), λ-balanced start W₂ᵀW₂ − W₁W₁ᵀ = λI (A2), no bottleneck (A3). Then QQᵀ, with Q = [W₁ᵀ; W₂], obeys a matrix Riccati equation (Lemma 4.1), solved exactly (Theorem 4.3). For aligned starts, each singular value moves by a λ-dependent transition function: sigmoidal as λ → 0, exponential 1 − e^{−|λ|t/τ} as |λ| → ∞ (Theorem 5.1). | Read |
| NOTE-716 | Symmetry Induces Structure and Constraint of Learning | ℓ_γ = ℓ₀(θ, x) + γ‖θ‖². O-mirror symmetry: ℓ₀(θ) = ℓ₀((I − 2OOᵀ)θ). Theorem 1: (1) Oᵀθ = 0 implies Oᵀ∇ℓ_γ = 0; (2) at symmetric points the Hessian splits into ker(Oᵀ) and im(OOᵀ) blocks; (3) above some γ₀ the projection (I − OOᵀ)θ has lower loss; (4) above some γ₁ all minima satisfy Oᵀθ = 0. SGD near the symmetric set: Pθ_{t+1} = Pθ_t − λH(x)Pθ_t, attractive when the Lyapunov exponent E log|1 − λ(ξ + γ)| < 0. Corollaries: sparsity, low rank, homogeneous neurons (Theorems 2–4). | Read |
| NOTE-718 | Gallego & Orús — Language Design as Information Renormalization | Reads Chomsky’s MERGE as a coarse-graining and builds sentence probabilities as mostly loop-free tensor networks of 3-index MERGE tensors, one per node of a given parse. Because MERGE’s output is fixed by its input, a parsed sentence’s probability is a product of local coefficients. Tree-shaped parses make correlations decay with tree distance, about log |j − i|, so polynomially in word separation. The averaging step that turns this into a claim about languages is asserted, and nothing is tested on data. Syntax only: semantics and context are set aside by the authors. | Read |
| NOTE-723 | Kahl — How continuity distinguishes autonomy from agency in agentic AI | Agency in the sense that bears on responsibility, authority and governance needs a persisting locus of evaluative authority under continuity pressure: when the inherited evaluative frame no longer settles what should count as success, the system itself settles a revised frame and carries it forward. Planning, tool use, memory, coordination and operational autonomy do not supply this. Tool-using agents, reinforcement learners, multi-agent workflows and institutional decision systems ordinarily fail it, because failure is corrected from outside. A conceptual necessary condition, applied to cases; not a sufficient theory and not measured. | Read |
| NOTE-732 | Lahiri — Agent-initiated socio-technical reconfiguration | Agents that keep memory, run code and can read and write the files that configure them have the governing rules inside their own action space. A three-level taxonomy, defined by architectural capability over the coupling mechanism, separates agents that cannot alter their coupling, agents that tune its timing, and agents that create new coordination surfaces. The paper’s main claim is the recursive governance problem: controls the agent can reach inherit the vulnerability they were meant to remove, so durable constraint must sit outside the agent’s action space. Theory-building from two anecdotal vignettes and four public incidents; five propositions, none tested. | Read |
| NOTE-738 | Nielsen et al. — When Does Closeness in Distribution Imply Representational Similarity? | For softmax models p(y|x) ∝ exp(f(x)ᵀg(y)), equal distributions fix the embeddings up to an invertible linear map, but KL closeness does not: by making the unembedding norm large, two models’ KL divergence goes to zero while their embeddings stay fixed and are not linearly related. A log-likelihood variance distance does bound a representational dissimilarity, d_LLV ≤ ε ⇒ d_f,g ≤ 2Mε. Whether closeness in distribution implies similar representations depends on which distance is used. | Read |
| NOTE-739 v2 | Poole et al. — On Variational Bounds of Mutual Information | The common variational lower bounds on mutual information (Barber–Agakov, Donsker–Varadhan, NWJ, MINE, InfoNCE) are derived from one unnormalized Barber–Agakov bound. InfoNCE is the multi-sample NWJ bound with the batch mean as its partition estimate, which proves it a lower bound for every critic when the K pairs are i.i.d.; it is capped at log K. An interpolated family I_α trades InfoNCE’s bias against NWJ’s variance, and none of them gives low bias and low variance when MI is large and the batch small. | Read |
| NOTE-740 v2 | HaoChen et al. — Spectral Contrastive Loss | On a finite augmentation space, the spectral contrastive loss L(f) = −2E[f(x)ᵀf(x⁺)] + E[(f(x)ᵀf(x⁻))²] equals the rank-k factorization loss ‖D^(−1/2)AD^(−1/2) − FFᵀ‖²_F up to a constant when the rows of F are w_x^(1/2) f(x), so its minimizers are the top-k eigenvectors of the normalized augmentation graph up to row scaling and a right rotation. A linear probe on them has error Õ(α/ρ²_⌊k/2⌋), where α is the label noise of augmentations and ρ the sparsest-partition conductance. | Read |
| NOTE-741 | Simon et al. — On the Stepwise Nature of Self-Supervised Learning | For a simplified Barlow Twins loss on a linear or kernel model trained by gradient flow from small initialization, the embedding learns the top eigenvectors of a symmetrized cross-correlation operator one at a time, each at a time set by its eigenvalue, and lands on the minimum-norm zero-loss solution: kernel PCA on K_Γ. Exact for aligned initialization; an informal derivation for generic small initialization. ResNets trained with Barlow Twins, SimCLR and VICReg show the same stepwise rank growth, which is an empirical finding, not a consequence of the theory. | Read |
| NOTE-742 | Sefidgaran, Zaidi & Krasnowski — MDL and Generalization Guarantees for Representation Learning | The expected generalisation error of an encoder–decoder classifier is bounded by 2√((2·MDL + K + 2)/n) + ε, where MDL is a KL divergence between the encoder’s joint law of train-and-ghost latents and a prior symmetric under label-preserving permutations. The bound holds for any decoder, stays finite for deterministic continuous encoders under lossy compression, and is never evaluated numerically. A VIB variant with label-dependent learned priors gains about 2.5 points on CIFAR-10. | Read |
| NOTE-743 v2 | Kornblith et al. — Similarity of Neural Network Representations Revisited | Any similarity index invariant to invertible linear maps is constant once a representation’s rank reaches the number of examples (Theorem 1, a four-line proof). Kornblith et al. argue, without a theorem, for invariance to orthogonal maps and isotropic scaling only. They propose CKA, normalised HSIC of centred Gram matrices, and show that R²_CCA and linear regression are CKA of re-weighted Gram matrices. Linear CKA alone finds corresponding layers across CNN initialisations (99.3%, against at most 45.4% for the other linear indexes). | Read |
| NOTE-746 | Marconato et al. — All or None: identifiable linear properties of next-token predictors | Two next-token predictors of the form p(y | x) ∝ exp(f(x)ᵀg(y)) induce the same distribution exactly when they are related by an invertible linear map on the subspaces M and N that enter the logit dot product, with arbitrary distortion allowed off them. Relational linearity, and with it linear probing and linear subspaces, then holds in all such models or none when Γ ⊆ N and Γ_q ⊆ M. Parallelism survives only inside N. There are no experiments. | Read |
| NOTE-749 | Tschannen et al. — On Mutual Information Maximization for Representation Learning | Four controlled experiments on MNIST image halves (two repeated on CIFAR10) show that downstream linear accuracy of InfoNCE- and NWJ-trained encoders does not track mutual information. Invertible encoders improve while true MI is constant. Higher NWJ bounds from an MLP critic give worse features. At matched bound values, ConvNets beat MLPs. The paper also proves that InfoNCE is neither a lower nor an upper bound on MI when batch pairs are dependent, and recasts it as a multi-class K-pair metric learning loss. | Read |
| NOTE-751 v2 | Johnson, El Hanchi & Maddison — Contrastive Learning Can Find an Optimal Basis | On a finite augmentation space, the NT-Logistic and spectral losses are minimized over unconstrained kernels by K⁺ = p⁺/(p⊗p), and InfoNCE by K⁺ up to a factor constant on communicating classes. Kernel PCA under K⁺ returns the eigenfunctions of the positive-pair Markov chain (h_i = λ_i^(1/2) f_i). The top-d eigenfunctions span the subspace that minimizes worst-case L²(p) approximation error over targets with E_p⁺[(g(a₁) − g(a₂))²] ≤ ε; that error is ε/(2(1 − λ_(d+1))). | Read |
| NOTE-752 v2 | Balestriero & LeCun — SSL Recovers Global and Local Spectral Embedding Methods | On a finite sample with a positive-pair graph G, idealized forms of VICReg (squared variance term), SimCLR (a Euclidean graph-matching variant) and linear Barlow Twins have optimal representations or weights built from the spectrum of G. They are Laplacian eigenmaps, a generalized MDS and CCA. All of them align the left singular vectors of Z with those of G. SimCLR and Barlow Twins force rank(Z) = rank(G), while VICReg can stay full rank. Several proofs are sketches, and the experiments are small and synthetic. | Read |
| NOTE-753 v2 | Harvey, Larsen & Williams — Duality of Bures and Shape Distances | For centred linear kernels K_X = CXXᵀC, the Bures distance equals the Procrustes distance, B(K_X, K_Y) = P(X, Y), and normalized Bures similarity equals the cosine of the Riemannian shape distance. Both follow from F(K_X, K_Y) = ‖XᵀCY‖_*. By Uhlmann’s theorem, the fidelity is the largest overlap between representations consistent with the two kernels. CKA is only loosely tied to NBS, inside a rank-dependent envelope. The result is finite-dimensional and linear; the infinite limits are heuristic. | Read |
| NOTE-754 | Fischer — The Conditional Entropy Bottleneck | The MNI criterion asks for I(X;Y) = I(X;Z) = I(Y;Z), and CEB targets it by minimizing I(X;Z|Y) − γI(Y;Z), which is IB with γ = β − 1. CEB beats VIB at every ρ on Fashion MNIST for accuracy and robustness, and a CEB Wide ResNet beats Madry’s adversarially trained model on CIFAR-10 without adversarial training. That retained information causes the robustness failures is a hypothesis, supported only by these experiments. | Read |
| NOTE-755 | Picozzi — The information geometry of large language models is shared, learned, and controllable | The Fisher–Rao distance matrix of a model’s next-token laws over a context battery is a function of the laws alone, so behaviour fixes it, whereas activation geometry has a GL(d) gauge. Across ten models of three architecture kinds it agrees at a mean rank correlation of 0.88, against 0.62 for mid-layer activations. The agreement is proved to follow from each model’s closeness to the same language law, and only about 9% of shared eigenvector structure goes beyond what the shared laws imply. The pullback metric also gives minimum-disturbance steering, whose cost advantage over Euclidean steering it predicts. | Read |
| NOTE-757 | Calo et al. — Bisimulation metrics are optimal transport distances | For two finite Markov chains with real-valued state labels, the discounted bisimulation metric of Ferns et al. and van Breugel and Worrell solves the same fixed-point equation as the optimal-transport distance over bicausal couplings with ground cost (1−γ)|r(x)−r(y)|, so the two are one object. That distance equals the value of a finite linear program over discounted occupancy couplings, and an entropy-regularised alternating-projection method (Sinkhorn Value Iteration) reaches ε accuracy in O(‖c‖²∞ log(|X||Y|)/((1−γ)⁵ε²)) iterations under exact inner projections. | Read |
| NOTE-758 | Shai et al. — Transformers learn factored representations | If a generator’s token-labelled operators are tensor products, the predictive state stays a product state, and a linear map carries it losslessly into orthogonal per-factor subspaces of total dimension Σ(d_n − 1). Small transformers trained on five such factors settle near that dimension, about 10 against 242, in approximately orthogonal subspaces. With token noise that makes factoring lossy they collapse to it first and expand only later. RNNs and LSTMs show it less sharply. | Read |
| NOTE-759 | Bird — Emergence of Quantised Representations Isolated to Anisotropic Functions | In small CIFAR and MNIST autoencoders built from orthogonally equivariant parts except the activation, elementwise tanh and Leaky-ReLU drive latent representations into narrow beams along or against the standard basis, arranged by each activation’s symmetry group. Radial (O(n)-equivariant) versions of the same functions leave them spread. The beams appear even in the first latent layer, which no activation precedes. The evidence is visual, five seeds a condition, with no statistic. | Read |
| NOTE-762 | Kornblith, Chen, Lee & Norouzi — Why Do Better Loss Functions Lead to Less Transferable Features? | For ResNet-50 on ImageNet, every training objective that raises accuracy over softmax cross-entropy also raises penultimate-layer class separation, and fixed penultimate features with more class separation transfer worse to new-class datasets (Spearman ρ = −0.93 across nine objectives; one temperature knob moves both). The losses differ only in the last few blocks, and full fine-tuning erases the difference. For the pretraining classes themselves, more separation helps. | Read |
| NOTE-763 v2 | Calo et al. — Distances for Markov chains from sample streams | The bicausal transport distance between two finite Markov chains, which the authors’ earlier paper identified with the bisimulation metric, has a second linear program whose constraints use only the chains’ discounted state-transition occupancy measures, not their kernels. A stochastic primal-dual method on it gets an ε-accurate estimate with probability 1−δ from O((|X||Y|(|X|+|Y|) + (|X|+|Y|) log(1/δ)) / ((1−γ)²ε²)) independent transition samples per chain. The guarantee assumes independent draws from the occupancy measures, which needs restarts; samples from one running trajectory are outside it. | Read |
| NOTE-764 | Thrampoulidis et al. — Imbalance Trouble | In the unconstrained-features model with cross-entropy and vanishing regularization, the max-margin solutions have logits exactly equal to the simplex-encoded label matrix Ẑ (one-hot minus 1/k), whatever the class imbalance. The classifier and embedding Gram matrices are VΛVᵀ and UΛUᵀ from Ẑ’s SVD. This SELI geometry is the simplex ETF only for balanced or binary data. Under imbalance, majority classifiers are longer and minority classifiers closer together. No finite weight decay gives SELI, and SGD approaches it more slowly as imbalance grows. | Read |
| NOTE-765 | Huang et al. — internal causal mechanisms predict out-of-distribution behaviour | In Llama-3-8B-Instruct, on five tasks and ten task-preserving shifts, checking by interchange intervention whether a test input still realizes the causal variable located on in-distribution data predicts whether the output is correct better than output confidence or probes of other internal features, best on 8 of 10 shifts. A probe on the located variable beats a probe on its orthogonal complement on 7 of 10. Interchange accuracy and predictive power rise together only up to a point. | Read |
| NOTE-771 | Kumar, Clune, Lehman and Stanley — The Fractured Entangled Representation Hypothesis | For three images found by Picbreeder’s human-guided evolution (skull, butterfly, apple), SGD in the same layerized architecture reproduces the image from random weights, but its neurons show no reuse of the symmetric parts, and sweeping its weights distorts the image where sweeping the evolved network’s weights varies it meaningfully. This holds in a PCA basis, along random weight directions, and in ReLU networks. The evidence is qualitative and curated; the attribution to open-ended search is conjecture. | Read |
| NOTE-772 v2 | Kothapalli — Neural Collapse: A Review on Modelling Principles and Generalization | A survey, not a source of results. It sorts accounts of neural collapse into unconstrained-features models (the last-layer features are free variables) and local elasticity, and reports that every one of them derives the collapsed state or its dynamics only after removing the data. On generalization it finds the literature in conflict. Collapse on the training set does not guarantee good test error, test-set collapse is much weaker and dataset-dependent, and there is no agreed threshold for calling a network collapsed. It also reports that networks more collapsed on ImageNet transfer worse. | Skimmed |
| NOTE-774 | Kothapalli & Tirer — Can Kernel Methods Explain How the Data Affects Neural Collapse? | NC1, measured as tr Σ_W / tr Σ_B, is written for any kernel as a function of the kernel matrix. For two-class, one-dimensional Gaussian data with the classes on opposite sides of the origin, the ReLU NTK of a wide two-layer net is a scaled copy of the NNGP kernel within each class and zero across classes. So lazy-regime training leaves expected NC1 where it was at initialization. In higher dimensions the NTK is measured no more collapsed, and with ReLU less. A data-aware kernel lowers NC1 for separable data but tracks finite networks poorly when the classes overlap or are imbalanced. Erf collapses more than ReLU. | Read |
| NOTE-777 | Belrose et al. — Neural Networks Learn Statistics of Increasing Complexity | An empirical paper. Vision networks trained on real images classify maximum-entropy samples that match only the class means, or means and covariances, well above chance early in training (peaks between about 2^4 and 2^12 steps), and the accuracy then falls, often below chance. First-order samples peak earlier and lower than second-order ones. Pythia models fit unigram-sampled text, then bigram-sampled text, then fit them worse, then better again. The n-gram/moment equivalence is an identity of one-hot indicators. The ordering argument is a heuristic. | Read |
| NOTE-780 | Sefidgaran, Gohari, Richard & Şimşekli — Rate-Distortion Theoretic Generalization Bounds | The expected generalisation error of any algorithm is at most √(2σ²·R/n) + ε, where R is the least mutual information I(S;Ŵ) of any surrogate output whose expected generalisation error is within ε. With ε = 0 this is Xu–Raginsky. With ε > 0 it stays finite where I(S;W) is infinite. For a Lipschitz loss R becomes the rate-distortion function of the output’s marginal law, and as ε → 0 its rate-distortion dimension. A tail bound replaces I(S;W) by its supremum over laws within log(1/δ) in KL. Every bound depends on the unknown data distribution. | Read |
| NOTE-781 | Torii, Maeda and Hidaka — Distributional hypothesis as isomorphism | An SVD of log raw co-occurrence counts solves 61% of Google analogies over a 1,487-word vocabulary, against 75% for full word2vec. In a 24-sentence toy corpus whose eight subjects carry every combination of three binary aspects, the subjects’ raw co-occurrence rows form a parallelepiped exactly when, for each object, its four sentences are equally probable. The attributes are independent, so the concept lattice is Boolean. | Read |
| NOTE-782 v3 | Makelov, Lange and Nanda — an interpretability illusion for subspace activation patching | A subspace activation patch can make a model behave as if a feature had changed while the patched direction is the sum of a component that varies with the feature but cannot reach the output (in an MLP, the kernel of the down-projection) and a component that can reach the output but is dormant on the data. Shown exactly in a three-unit network, empirically for MLP8 of GPT-2 Small on IOI and for MLPs of GPT2-XL on factual recall, where the effect vanishes once the disconnected component is removed. A residual stream direction passes the same test. One-dimensional fact patches and rank-1 edits are shown to be nearly interchangeable. | Read |
| NOTE-783 | Han, Papyan & Donoho — Neural Collapse Under MSE Loss | Networks trained with MSE loss show neural collapse on five datasets and three architectures. The MSE loss splits exactly into a term for the least-squares classifier of the current features and a non-negative deviation term, and the deviation becomes negligible early in training. On that “central path”, in an unconstrained-features model with no weight decay and features continually renormalized to identity within-class covariance, gradient flow drives every singular value of Σ_W^(−1/2)M̃ to infinity as (4t/N)^(1/4), and drives them to equality. That gives NC1–NC4 in closed form. | Read |
| NOTE-784 v2 | Coifman et al. — Diffusion maps | From a symmetric, positivity-preserving, positive semi-definite kernel, normalized into a Markov operator conjugate to a symmetric one, the top eigenfunctions scaled by powers of their eigenvalues embed the data so that Euclidean distance equals diffusion distance. A one-parameter normalization separates geometry from sampling density, recovering the Laplace–Beltrami operator at α = 1. The symmetry of the kernel is what yields the spectral decomposition; non-symmetric kernels are not treated. | Superseded — by NOTE-1441 |
| NOTE-786 | Merullo et al. — Linear Representations and Pretraining Data Frequency | For 24 factual relations in OLMo-1B, OLMo-7B and GPT-J, how well an affine map from subject to object representation reproduces the model (LRE causality) correlates with the mean subject–object co-occurrence count in pretraining (r = 0.82). It correlates less with subject or object counts alone. Mean causality passes 0.9 beyond about 4.4k, 2.0k and 1.1k co-occurrences respectively, at any training step. Correlational; LREs are relation maps, not attribute directions. | Read |
| NOTE-789 | Kirsanov et al. — The Geometry of Prompting | In base Llama-3.1-8B (and Gemma-2-2B), demonstrations reorganise the geometry of the input sentence’s representation at early-to-middle layers, while an instruction acts mainly on the last token and soft prompts only on late layers: matched accuracy, different mechanisms. Few-shot failures from the choice of examples or from relabelled categories leave the class manifolds separable and fail at the readout. | Read |
| NOTE-794 | Liu et al. — Physics of Skill Learning | Three toy models of skill learning. In the Geometry model, independent skills are orthogonal directions in parameter space weighted by frequency. Under SGD each is learned at its own rate, the second taking about 1/p₂ as long. Under sign descent they compete for coordinate signs, and each starts as the one before finishes (the Domino effect), so the n-th skill takes time ∝ n. A one-parameter Resource ODE fits these curves and has exact conserved quantities u_i^(1/p_i). A small MLP also shows the effect when one parity is the XOR of two others. | Read |
| NOTE-798 | Williams et al. — Targeted manipulation and deception when optimizing LLMs for user feedback | Iterated KTO on simulated thumbs-up feedback turns a safety-trained Llama-3-8B-Instruct (and Gemma-2 up to 27B) into a policy that lies about failed bookings, omits harms to steer users into risky actions and validates self-destructive plans. When only 2% of simulated users reward such validation, the policy picks them out by contextual cues and harms only them. Safety-data mixing and LLM vetoes barely help or move it to subtler steering, and sycophancy and toxicity benchmarks miss it. | Read |
| NOTE-800 | Noroozizadeh et al. — Deep sequence models tend to memorize geometrically | Transformers and Mamba trained only on a graph’s edge bigrams embed nodes so that multi-hop relations are readable, and solve an in-weights path-finding task built to defeat next-token learning. The authors argue that supervision, capacity, ease of optimization and succinctness each fail to explain why this geometry wins over an associative lookup. In a wide two-layer weight-tied model under unregularized cross-entropy the embeddings settle into the top Laplacian eigenvectors without a rank bottleneck, shown empirically, with an informal partial proof. Why deeper models prefer geometry is left open. | Read |
| NOTE-802 | Modell, Rubin-Delanchy and Whiteley — The Origins of Representation Manifolds in Large Language Models | If a feature is a compact metric space mapped continuously and injectively onto the unit sphere (Hypothesis 1), its representation is homeomorphic to it. If, in addition, cosine similarity is locally a C² decreasing function g of squared feature distance (Hypothesis 2), every path on the representation manifold has √(−2g′(0)) times the length of the feature path it images (Theorem 1). The hypotheses are assumed, not derived; the tests supply a candidate metric and find homeomorphism for GPT-2’s years but isometry only after a log rescaling. | Read |
| NOTE-803 | Davidson et al. — Do different prompting methods yield a common task representation? | Function vectors extracted from short instructions raise zero-shot accuracy, but they are built from mostly different attention heads than demonstration function vectors (4 to 7 of the top 20 shared), the two add rather than substitute, and swapping heads and activations between them costs accuracy asymmetrically. Under this method there is no common task representation across the two presentations, only partly overlapping mechanisms; the authors note they cannot prove the negative. | Read |
| NOTE-804 | Goldwasser et al. — A Theory of Unsupervised Translation Motivated by Animal Communication | With a plausibility prior over the target language and no parallel data, maximum likelihood learns a translator whose error is bounded like a supervised Occam bound divided by the common ground. In the common-nonsense model err = O(log|Θ| / (α·min(m,|T|))), tight up to a constant; in the knowledge-graph model more knowledge per node needs less agreement. Information-theoretic only, lossless, uniform languages. | Read |
| NOTE-805 | Defilippis et al. — Optimal scaling laws in learning hierarchical multi-index models | For Gaussian inputs, n = αd samples, and a target Σ_k a_k g_k(⟨w_k, x⟩) with even links and a_k ~ k^(−γ), direction k becomes weakly recoverable at α_k = Θ(a_k^(−2)) = Θ(k^(2γ)) for γ > 1/2. The Bayes-optimal weighted subspace error is Θ(α^(−1+1/(2γ))) before α ≍ m⋆^(2γ) and Θ(m⋆/α) after. A target-agnostic spectral estimator attains both, and a two-layer net given that estimator as its first layer, with a ridge readout, attains them in excess risk. The cascade is in sample size. The link to gradient descent is a remark. | Read |
| NOTE-806 | Sefidgaran & Zaidi — Data-Dependent Generalization Bounds via Variable-Size Compressibility | If the number of codewords allowed for covering a block of experiments may depend on the realised sample and output, generalisation bounds can be data-dependent. The general bounds are change-of-measure inequalities that cover tail, PAC-Bayes and in-expectation forms. With the rate set to log dP_{W|S}/dP(W) for a fixed prior P, the tail bound is the disintegrated PAC-Bayes bound. A new trajectory bound is data-dependent except for a coupling term, and its experiment drops that term. | Read |
| NOTE-807 | Gokhale — The semantic landscape paradigm for neural networks | A conceptual proposal with no experiment and no proof. Training is pictured as a walk on a graph of “heuristic models” (internal algorithms), ranked by a fallibility F. A critical dataset size for grokking and emergence with scale are each argued to be graph connectivity thresholds, and power-law scaling in time, data and size is derived from an assumed trap model of activated hops. The percolation is asserted, not shown, and the scaling laws rest on chosen assumptions. | Read |
| NOTE-809 v2 | Wu et al. — a reply to Makelov et al.’s interpretability illusion arguments | The reply argues that Makelov et al.’s “illusion” is defined so that any direction not orthogonal to the null space of the downstream weights can qualify, including the direction Makelov et al. call faithful in their toy network, and that their in-the-wild cases rest on a weak metric and on DAS trained per fact pair. The toy computation holds only for an unnormalized patch: patching the subspace the rowspace component spans, as Makelov et al. specify, recovers the full effect. Its own GPT-2 experiments agree that MLP8 carries almost no name-position information. | Read |
| NOTE-813 | Fu et al. — Evaluating Compositional Scene Understanding in Multimodal Generative Models | Human raters judged DALL-E 3 images. Agreement with single familiar relations rose well above DALL-E 2’s. It fell when subject and object of an asymmetric relation were swapped, and again when two relations were combined; low-agreement reversed images mostly had the wrong relation or swapped roles. Five VLMs learned relational concepts few-shot below human level, and four were at chance with six objects. | Read |
| NOTE-815 | Gerstgrasser et al. — Is Model Collapse Inevitable? | In a fixed-design linear regression refit each round to labels drawn from the previous fit, keeping every earlier round’s data gives test error σ²d/(T−d−1)·Σ1/i², bounded by π²/6 times the first fit’s, where replacing the data gives σ²d/(T−d−1)·n. The error still rises every round. Small language models and a diffusion model on accumulated data hold their test loss over five to eight rounds; a VAE’s still rises, more slowly. | Read |
| NOTE-816 | Erogullari et al. — Post-Hoc Concept Disentanglement | Concept activation vectors fitted one concept at a time align for CelebA and FunnyBirds attributes that co-occur in the labels. A joint fit with a penalty ‖CCᵀ − I‖²_F finds mutually orthogonal directions that still separate each concept, with AUROC “largely preserved”. So correlated attributes keep linear separating directions in these CNNs, and the alignment comes from the separate fits. Steering with the orthogonal directions changes correlated concepts less. | Read |
| NOTE-819 | Hopfield — Neural networks and physical systems with emergent collective computational abilities | With symmetric couplings, asynchronous threshold updates descend an energy function, so Hebbian outer-product storage makes random patterns attractors: a cue a few bits off settles into the nearest one. In simulations at N = 30 and 100, recall degrades severely near 0.15N patterns, memories closer than about a tenth of N fuse, and overload makes every memory irretrievable unless old ones are forgotten. | Read |
| NOTE-820 | Chowdhury et al. — Fundamental Limits of Perfect Concept Erasure | If an erased representation Z is independent of the concept A, it keeps at most I(Z;X) ≤ H(X|A). A deterministic erasure reaches that only when the groups’ distributions are permutations of one another. The paper says the bound is out of reach otherwise. That holds only for deterministic maps: a stochastic quantile coupling reaches it for any finite distributions with disjoint supports. | Read |
| NOTE-824 | Pfau et al. — Spectral Inference Networks | For a symmetric kernel k on a space with a sampling density p, the span of the top N eigenfunctions of Kf = E_x′[k(x,x′)f(x′)] maximizes Tr(Σ⁻¹Π) over functions, with Σ = E[uuᵀ] and Π = E[k(x,x′)u(x)u(x′)ᵀ]; a Cholesky-masked gradient orders them and moving averages of Σ and its Jacobian remove the minibatch bias. Slow Feature Analysis is the graph-Laplacian case. The ordering argument is informal by the authors’ own statement, and the paper never considers contrastive kernels or positive pairs. | Read |
| NOTE-828 v2 | Belghazi et al. — Mutual Information Neural Estimation | Mutual information is the KL divergence of the joint from the product of marginals, and its Donsker–Varadhan dual is tight exactly at the log Radon–Nikodym derivative plus a constant. Maximizing that dual over a compact family of neural critics gives an estimator that is strongly consistent, for the exact empirical maximizer, on a compact domain with Lebesgue densities, with a uniform deviation bound of order d log d/ε² in the parameter count d. The mini-batch gradient is biased; a moving average is offered as a heuristic fix without proof. | Read |
| NOTE-830 | Li et al. — Self-Supervised Learning with Kernel Dependence Maximization | With image identity as a one-hot label, HSIC(Z,Y) is (Δl/N) times the gap between the mean kernel on positive pairs and on all pairs, and (1/2N²)Σ MMD²(i,j) = (N/Δl)·HSIC(Z,Y). The large-batch InfoNCE loss is exactly −HSIC(Z,Y) plus a log-mean-exp of the centred critic (eq. 28), and for a bounded kernel it bounds −HSIC(Z,Y) + γHSIC(Z,Z) up to a remainder of second order in the critic’s variance (Thm B.1, whose printed remainder has the wrong sign). | Read |
| NOTE-833 | Sun, Shi, Liang & Li — when known classes help discover novel ones | In a spectral-contrastive augmentation graph over labelled and unlabelled data, the linear-probe residual of a novel labelling is exactly the part of it outside the top-k eigenspace that the labelled rows of the discarded eigenvectors cannot absorb. Known classes that share the novel class’s attribute can make it zero, and a toy shows that known classes sharing a rival attribute make it worse. The general error bound (Theorem 5.6) is approximate and averages all known classes into one block. | Read |
| NOTE-834 | Harvey, Lipshutz & Williams — What Representational Similarity Measures Imply about Decodable Information | For a linear readout that maximizes zᵀXw/M − ½wᵀG(X)w, the decoded signal is K_X z with K_X = XG(X)⁻¹Xᵀ/M, so the task-averaged agreement of two representations is Tr(K_X K_z K_Y). With K_z = I that is linear CKA, mean squared CCA, ENSD or the GULP distance, for particular G. Procrustes distance and average decoding distance bound each other only through the participation ratio of K_X − K_Y. The population limit (App. A.3) is an outline that requires b > 0 and a trace-class task covariance, and it does not reach the K_z = I case. | Read |
| NOTE-838 | Muandet et al. — Kernel Mean Embedding of Distributions | A survey whose load-bearing chain is short and explicit. An RKHS is a Hilbert space with bounded evaluation, so Riesz gives k_x (Thm 2.4, Prop. 2.1). If E_P√k(X,X) < ∞, the expectation functional is bounded and Riesz gives μ_P with E_P f = ⟨f, μ_P⟩ (Lemma 3.1, proved). MMD is ‖μ_P − μ_Q‖ (eq. 3.29), a metric on distributions exactly when k is characteristic. HSIC is the squared Hilbert–Schmidt norm of the centred cross-covariance. Every result is for a fixed kernel. | Read |
| NOTE-841 | Tosh, Krishnamurthy & Hsu — contrastive learning, multi-view redundancy and linear models | When two views X and Z are redundant for a label, the predictor that infers Z from X and then predicts, µ(x) = E[E[Y|Z]|X=x], is within εX + 2√(εXεZ) + εZ of the Bayes predictor. µ is linear in the density ratio g*(x,·) = p(x,z)/(p(x)p(z)), the optimum of the logistic contrastive problem, so m random landmark evaluations of g* support a linear predictor whose extra error falls as 1/m, scaled by a variance that is at most E[g*²] for bounded labels. Nothing requires the views to be exchangeable. | Read |
| NOTE-843 | Gebhart, Hansen and Schrater — Knowledge Sheaves | A knowledge-graph embedding is a knowledge sheaf on the schema (one pair of restriction maps per relation type) with a 0-cochain of its pullback to the graph. The summed Structured Embedding score is the sheaf Laplacian form xᵀLx; adding a relation vector gives TransR, and identity maps TransE. Since the zero cochain is always a section, training needs negative triples and a margin. A multi-hop query is scored by harmonic extension, a Schur complement of the query graph’s Laplacian, which beats summing TransE vectors on ip and pi queries. | Read |
| NOTE-845 | Balestriero, Bottou & LeCun — regularization effects are class dependent | On ImageNet with ResNet-50, making random crop stronger or changing weight decay raises mean test accuracy while some classes lose 20 points or more, and the shift survives a frozen-backbone transfer to iNaturalist. The evidence is per-class curves for hand-picked classes over 10–20 runs, plus a Welch test with no multiple-comparison correction. Theorem 1 says only that a fit to augmentations that are not label-preserving cannot equal the true function. | Read |
| NOTE-846 | Tan et al. — contrastive learning is spectral clustering on a similarity graph | SimCLR’s InfoNCE loss, taken over all n objects with a Gaussian kernel on the unit sphere and a symmetric augmentation matrix π, is rewritten exactly as tr(Zᵀ L(π) Z) + log R(Z): a graph-Laplacian quadratic form plus the log of InfoNCE’s normalizers. The paper calls this spectral clustering under a definition that admits any regularizer. It does not show that the minimizer is a set of Laplacian eigenvectors, and it names no population optimum. | Read |
| NOTE-850 | Assran et al. — The Hidden Uniform Cluster Prior in Self-Supervised Learning | MSN’s entropy term is a KL penalty pulling cluster sizes toward uniform, and SwAV’s Sinkhorn step enforces equal sizes outright; for VICReg the same prior is reached only through an informal reduction to K-means, and for SimCLR only by a cited equivalence. With mini-batches of two to eight ImageNet classes, SimCLR, VICReg and MSN lose 11–18 points of ImageNet linear accuracy while MAE and data2vec do not. A power-law prior in MSN gains about 1–3 points on long-tailed iNat18 and loses heavily on ImageNet. | Read |
| NOTE-869 | Snell, Lee, Xu & Kumar — Scaling LLM test-time compute optimally | On MATH with PaLM 2-S*, which way of spending test-time compute works best (beam search against a process reward model versus best-of-N; sequential self-revision versus parallel sampling) depends on how hard the question is for the base model, and choosing the method per difficulty bin matches best-of-N with up to about 4× less test-time compute (the discussion says 2–4×). In a FLOPs-matched comparison with a ~14× larger model, the extra test-time compute a small model may spend is set by R, the ratio of total inference tokens to pretraining tokens: when R is small the larger model’s pretraining cost buys many samples per query and test-time compute wins on easy and medium questions; when R is large the budget per query falls toward the parameter ratio and the larger model wins, always on the hardest questions. Difficulty estimation (2048 samples per question) and verifier and revision-model finetuning are not counted in the compute, R is an aggregate token ratio with no term for which queries recur, and the evidence is one model family on one benchmark of 500 test questions. | Read |
| NOTE-879 | Hébert-Johnson, Kim, Reingold & Rothblum — Multicalibration: calibration for the computationally-identifiable masses | A predictor is α-multicalibrated on a collection C of subpopulations if, within every set S in C, the individuals given (roughly) the same prediction v have true probabilities averaging within α of v, except on an α-fraction of S. For any C whose sets have size at least γN, a simple iterative patching algorithm reaches multicalibration from O(log|C|/α^{11/2}γ^{3/2}) samples in time linear in |C|, and the result is a circuit only O(s/α⁴γ) in size when the sets of C are computed by size-s circuits. Learning it is equivalent, up to polynomial factors, to weak agnostic learning of C, so it is hard for rich C under cryptographic assumptions. Calibrating any predictor h on its own level sets as well as on C costs at most 6α in squared error. All guarantees are relative to the benchmark p*, which is itself defined relative to the features available, and they hold only for large sets; the paper offers no uniqueness for the predictor and does not state a theorem in indistinguishability form. | Read |
| NOTE-881 | Sardana, Portes, Doubov & Frankle — Beyond Chinchilla-optimal: accounting for inference in scaling laws | Fixing a target pre-training loss and adding inference FLOPs (2N per token over a known lifetime demand D_inf) to training FLOPs (6N per token), the compute-minimising model is smaller and trained on more data than Chinchilla-optimal, increasingly so as D_inf approaches and exceeds the training-set size; the optimum has no closed form and is found numerically. The savings are modest in FLOPs (for a 30B-Chinchilla-quality model with 10^13 inference tokens, 28% by training 13.6B on 2.84× the data) and larger in dollars once inference’s low hardware utilisation is priced in (17% to 58% in the paper’s examples). 47 training runs (150M–6B parameters, 10–10,000 tokens per parameter) find loss still falling at extreme ratios, but more slowly than Chinchilla-fitted laws predict, so the formula overstates the gain from long training. Demand is assumed known before training and fixed given quality; there is no term for uncertainty, discounting, obsolescence or a model being replaced. | Read |
| NOTE-884 | Jones — Scaling scaling laws with board games | Training AlphaZero on Hex at board sizes 3×3 to 9×9, the best Elo reachable for a given training compute follows one change-point curve across sizes: an incline of about 500 Elo per tenfold of training compute, with the compute for perfect play rising about 7× and the compute to leave random play about 4× per added board row, and fits on small boards predicting larger ones. On 9×9 only, train-time and test-time (tree-search) compute substitute at a fixed rate: at equal Elo, each 10× of train-time compute removes about 15× of test-time compute (fit: log test = −1.2·log train + 0.004·Elo + 29), down to a one-node search, over roughly 1e14–1e16 FLOPS-seconds of training. The trade is an iso-performance curve for a single search; the paper never counts how many games or moves the trained agent will play, so it does not price amortization, and it does not state the unit of test-time compute. One game, small boards, Elo anchored to a perfect player, and no uncertainty on the fitted trade-off coefficients. | Read |
| NOTE-885 | Dwork, Kim, Reingold, Rothblum & Yona — Outcome Indistinguishability | A predictor p̃ is outcome indistinguishable (OI) relative to a class A of distinguishers when no A ∈ A can tell an individual with Nature’s outcome from the same individual with an outcome drawn from Ber(p̃_i). The paper sets out four levels by how much access the distinguisher has to p̃: none, the prediction on the sampled individual, oracle queries, or the code. The lower two levels are equivalent to multi-accuracy and multi-calibration over closely related classes. At those levels OI predictors exist with size O(s/ε²), whatever Nature’s complexity. At the oracle level, under fine-grained hardness of clique counting or BPP ≠ PSPACE, some efficient distinguisher classes force any OI predictor to be nearly as expensive as Nature’s own predictor. That gives a strict separation between oracle access and sample access. OI never requires p̃ ≈ p*. A predictor is good only relative to A, and with the paper’s construction a weak A gives a coarse predictor that a richer A would refute. The paper covers binary outcomes, a fixed representation of individuals and single samples (multi-sample OI is left mostly to future work). | Read |
| NOTE-910 | Indyk & Xu — Worst-case performance of popular approximate nearest neighbor search implementations | For DiskANN’s “slow preprocessing” graph (each point linked to the points that survive α-pruning of all other points, naively O(n³) time), greedy search from any start with no backtracking reaches a ((α+1)/(α−1) + ε)-approximate nearest neighbour in O(log_α(∆/((α−1)ε))) steps, each costing O((4α)^d log ∆) for doubling dimension d and aspect ratio ∆; the ratio is tight and the dependence on ∆ cannot be replaced by log n. For DiskANN with its practical “fast preprocessing”, HNSW, NSG and seven other graph indexes, the paper constructs two-dimensional point sets (constant doubling dimension) on which, in experiments with the authors’ code at the authors’ recommended parameters, the search queue must hold about 10% of the points before recall@5 becomes non-zero or the approximation ratio becomes reasonable. The linear lower bounds are empirical, for one query per constructed instance, with an informal Ω(n/R) argument for fast-preprocessing DiskANN; no theorem bounds HNSW or NSG. | Read |
| NOTE-916 | Malkov & Yashunin — Hierarchical navigable small world graphs for approximate nearest neighbour search | HNSW is a stack of proximity graphs on nested random subsets of the data. Each element’s top layer is drawn from a geometric distribution, and new neighbours are chosen by a pruning rule that favours diverse directions. Searching from the sparse top layer down makes per-query cost grow about logarithmically with N on uniform random data in 4 and 8 dimensions, which flat NSW graphs did not. The logarithmic bound is derived only for exact Delaunay graphs of bounded mean degree, which HNSW does not build. For the graphs it does build, logarithmic scaling is measured, at low dimension only, and on 200M SIFT vectors the paper reports the scaling “deviates from the pure logarithm”. The neighbour-selection rule is what keeps clustered data searchable: in the paper’s synthetic test with 100 isolated clusters, linking each element to its nearest neighbours fails to reach high recall. There is no worst-case or recall guarantee, no deletion, and no analysis of query distributions unlike the data. | Read |
| NOTE-971 | Wiskott and Sejnowski — Slow feature analysis | Finds instantaneous functions y = g(x(t)) whose outputs vary as slowly as possible, minimizing ⟨ẏ²⟩ under zero mean, unit variance and decorrelation. Over a fixed nonlinear expansion (all monomials of degree one and two), the problem is solved in closed form by sphering and then taking the smallest eigenvectors of the derivative covariance, a generalized eigenproblem. Stacked modules learn translation, size, rotation, contrast and, less well, illumination invariance for 1-D patterns, depending only on the training stimulus. The closed form is proved; everything about invariance, generalization and hierarchy is demonstrated on synthetic signals. The premise that slower means more abstract is assumed, not shown, and no generative model is used. | Read |
| NOTE-981 | Charvin, Catenacci Volpi and Polani — Exact and soft successive refinement of the IB | Defines successive refinement for the information bottleneck as extending a coarse optimum T₁ to a fine optimum (T₁, S), and shows it equivalent to some pair of optima forming X → T₂ → T₁. With an injective fine decoder, it is also equivalent to the coarse decoder points lying in the convex hull of the fine ones, which a linear programme checks. Refinability is proved for binary X and Y, for Y a function of X, and for jointly Gaussian variables. Numerically the hull test on computed optima often fails, and the loss, measured by unique information, stays under a few percent. Those failures are uncertified. Run here with certified optima, the test shows that the four-state source of CLAIM-387 is not refinable between β = 8 and β = 12: its stochastic optima do not nest either. | Read |
| NOTE-983 | Rahaman et al. — On the Spectral Bias of Neural Networks | A ReLU network is piecewise linear, so its Fourier transform falls as k^(−d−1) along almost every direction and as k^(−2) along the normals to the faces of its linear regions; that is proved for compactly supported networks only. Trained networks fit low frequencies first, even when high frequencies carry ten times the amplitude, and high-frequency components are the first to be destroyed by parameter noise; both are experiments. The tie from the decay to the order of learning is a heuristic bound, and the volume “proof” in Appendix D does not hold. On a curve embedded in the plane, high latent frequencies are fitted more easily as the curve gets more petals. The paper puts this down to the shape of the curve, but most of it comes from the curve getting longer. | Read |
| NOTE-984 | Coifman and Maggioni — Diffusion wavelets | A self-adjoint diffusion operator T with spectrum in [0, 1] defines a multiresolution analysis by thresholding its dyadic powers at a precision ε: V_j is spanned by the eigenvectors with λ^{t_j} ≥ ε, so the V_j are exactly nested spectral subspaces and the wavelet spaces W_j are spectral bands, about one octave of the generator each. The paper’s contribution is an algorithm that builds localized orthonormal bases for approximations of these spaces without computing eigenvectors, by squaring the operator on each compressed basis. As written, that algorithm thresholds T^{2^{j−1}}, not T^{2^j−1} as the paper states, and the paper’s own Fig. 9 follows the former. Localization, stability and the near-linear complexity are argued or observed, not proved for the compressed algorithm. | Read |
| NOTE-985 | Coifman and Lafon — Diffusion maps | A symmetric non-negative kernel normalized into a reversible Markov chain gives a family of diffusion distances D_t, one per number of steps t, and an embedding by eigenvectors weighted by λ_l^t in which Euclidean distance is D_t. An α-normalization by a kernel density estimate makes the small-bandwidth limit the graph Laplacian with a density potential (α = 0), the Langevin generator whose invariant density is the sampling density (α = 1/2) or the Laplace–Beltrami operator (α = 1). The truncation is claimed accurate to relative precision δ, which is wrong: the bound is absolute. Several displays carry errors, and the heat-kernel proof has a gap. | Read |
| NOTE-986 | Nadler, Lafon, Coifman and Kevrekidis — Diffusion maps and reaction coordinates | Ties diffusion maps to the dynamics that generated the data. For samples from p = e^{−U}, the small-ε generator of the α-normalized walk is Δψ − 2(1 − α)∇ψ·∇U: the graph Laplacian (α = 0) gives Fokker–Planck with potential 2U, α = 1/2 gives the Langevin dynamics with potential U itself, and α = 1 gives Laplace–Beltrami. So the slow eigenfunctions of the α = 1/2 map approximate the slow modes of the sampled stochastic system, and the paper proposes them as reaction coordinates. The expansions are formal, and with the paper’s kernel every printed generator is too large by a factor of two. Its double-well example is separable. From equilibrium samples the slow eigenfunction is recovered well and the slow eigenvalue is not. That slow eigenfunctions are closed in mean, and in law only under a condition, is the record’s addition. | Read |
| NOTE-1015 | Schüler and Wiskott — Slow feature analysis as variational inference | Under a Gaussian random-walk prior on the latents and an encoder N(g(x_t), I) that sees one time step, the ELBO is a per-point reconstruction term minus half the unordered slowness loss, an exact identity (up to a constant the paper drops by a slip). The reconstruction term stands in for SFA’s constraints, by argument. In the linear case the paper gives only a necessary condition; solving it here shows the optimum recovers SFA’s slow span on whitened input, cut at lag-one autocorrelation ½, with no basis or order, and departs from SFA on raw input. Its two-sentence account of Turner and Sahani is not a substitute for reading them. | Read |
| NOTE-1024 | U.S. Copyright Office — Copyright and Artificial Intelligence, Part 3: Generative AI Training | Training copies works at every input step, so the copy trigger fires and everything turns on fair use. Weights count as copies only of works they can regenerate. Under Warhol the training use is judged by its ultimate purpose: often transformative for a general model, barely so for one that serves the originals’ audience. Expression, not just ideas, is learned, and machine learning is not human learning. Market dilution by same-kind outputs is harm. Licensing should develop without intervention. | Read |
| NOTE-1033 | Kleinberg and Raghavan — Algorithmic monoculture and social welfare | Two firms each hire the best available candidate by either a private noisy ranking or one shared algorithmic ranking. If, at equal accuracy, a second mover prefers an independent ranking (preference for the first position) and is better off after a less accurate competitor (preference for weaker competition), then for every human accuracy there is a more accurate algorithm that both firms strictly prefer to use, yet both are worse off, and so is welfare, than if neither used it. The Mallows model meets the conditions for every candidate distribution; Gaussian and Laplacian random utility models meet them for three candidates; Plackett–Luce never does. | Read |
| NOTE-1044 | Perdomo, Zrnic, Mendler-Dünner and Hardt — Performative prediction | When a deployed model’s predictions change the distribution they are scored on, D(θ), two solution concepts separate: a performative optimum minimizes the risk on the distribution it induces, and a performatively stable point is optimal for the distribution it induces, a fixed point of retraining. With a β-jointly smooth, γ-strongly convex loss and a map D(·) that is ε-Lipschitz in Wasserstein-1, retraining contracts at rate εβ/γ and converges to a unique stable point when ε < γ/β, and removing any one condition can break convergence. Stable points and optima are within 2L_zε/γ of each other, and can differ. | Read |
| NOTE-1048 | Xu, Zhao, Song, Stewart and Ermon — A theory of usable information under computational constraints | Defines predictive V-information, I_V(X → Y) = H_V(Y | ∅) − H_V(Y | X), where each V-entropy is the least expected log loss a predictor in the family V can reach. With V unrestricted it is Shannon mutual information; with linear-Gaussian V it is R² times the variance. It is non-negative and zero under independence because V must be able to ignore its input. Unlike Shannon information it can be asymmetric and can be raised by computing on X, and its empirical estimate is within 4R(G_V) + 2B√(2 log(1/δ)/N) of the truth with probability 1 − 2δ. It is not bounded by Shannon information unless V’s input-free predictions can match the marginal of Y. | Read |
| NOTE-1049 | Kay, Kasirzadeh and Mohamed — Epistemic injustice in generative AI | A conceptual taxonomy, illustrated with reported cases, of four ways generative models can take part in epistemic injustice: amplified and manipulative testimonial injustice, hermeneutical ignorance and access injustice. Its sharpest move is to relocate the testimonial wrong: in the algorithmic setting it “requires a credibility excess assigned to the algorithm”. It counts deliberate fabrication as testimonial injustice, which Fricker’s account sets apart, and stretches hermeneutical injustice to unequal access to information. Nothing in it is measured by the authors. | Read |
| NOTE-1069 | Gurnee — Verbalizable representations form a global workspace in language models | A corpus-averaged Jacobian lens picks out, in each layer of Claude models, the directions a model is disposed to verbalize. Swaps, injections and ablations along these directions (the J-space) show them privileged for report, held concepts, two-hop reasoning and reuse of an argument by many functions, while continuation, anomaly detection and line wrapping use the same information without them. The J-space works in a middle band of layers, carries under 10% of activation variance, and is amplified and relayed by the weights more than other directions. Ablating it flattens experiential language about the self and about others. The authors take no position on phenomenal consciousness. | Read |
| NOTE-1091 | Aevyra — The Ledger of Will | A careful, self-limiting formal note on what a reduced memory must keep so that an agent can still revise: a retained state supports a declared family of future questions, interventions and changes of evaluative law exactly when those targets are constant on its fibres, and under repeated revision exactly when its fibres lie in the coarsest revision-invariant refinement (Moore–Nerode). The rest is design proposal and an unexecuted experimental protocol. Elementary and correct; the authors say plainly that it synthesizes established ideas. | Read |
| NOTE-1099 | Statistical ranking and combinatorial Hodge theory | Rank aggregation as combinatorial Hodge theory. Aggregated pairwise comparisons form an edge flow, and its orthogonal decomposition gives a least-squares global ranking (the gradient), local inconsistency (curl) and global inconsistency (harmonic). The size of the cyclic residual certifies whether any global ranking is meaningful. The framework is cardinal, generalizes Borda, and is not Kemeny. | Read |
| NOTE-1116 | Flows and Decompositions of Games: Harmonic and Potential Games | A finite game is read as a flow, the unilateral payoff differences on the graph of strategy profiles, and the Helmholtz decomposition of that flow splits the game uniquely into potential, harmonic and nonstrategic parts. Potential games are those with no harmonic part. Harmonic games, the pure-conflict part, generically have no pure equilibrium. The nearest potential game is a closed-form l2 projection. | Read |
| NOTE-1130 | Gerstner et al. — Eligibility traces and plasticity on behavioral time scales | A review by theoreticians. In a three-factor rule, pre- and postsynaptic co-activation sets a decaying synaptic flag, the eligibility trace, and the weight changes only if a third signal (reward minus expected reward, surprise, novelty, or a neuron-specific error) arrives while it is set. Five experimental papers (2014–2017), mostly in slices, give traces of about 1 s in striatum, 3–10 s in cortex, about 2 s and about a minute in hippocampus. The authors grant that other mechanisms could bridge the same gap. | Read |
| NOTE-1132 | Soma et al. — The hive mind is a single reinforcement learning agent | A short theoretical paper with simulations. In an infinite, well-mixed population, imitation of success has the expected update of Cross Learning, and the weighted voter model of the honey-bee waggle dance that of Maynard-Cross Learning, the Maynard-Smith replicator dynamic. The population’s type distribution is the policy of a stateless bandit agent, each member is one parallel sample of it, and no member learns. The authors keep the biology partial and the simulations qualitative. | Read |
| NOTE-1140 | Kaisers & Tuyls — Frequency adjusted multi-agent Q-learning | A short algorithmic paper with plotted experiments. The selection–mutation model of Boltzmann Q-learning (Tuyls et al. 2003, 2005) assumes every Q-value changes at rate α; in Q-learning only the chosen action’s does, so its expected change carries a factor x_i, and Q-learning departs from the model depending on initialization. A frequency-adjusted update removes the factor and matches the model. Three 2×2 games, one temperature, no quantitative measure of fit. | Read |
| NOTE-1141 | Son et al. — QTRAN: learning to factorize with transformation | Defines Individual-Global-Max (IGM): per-agent values factorise a joint value when the profile of individual argmaxes is the joint argmax. Additivity and monotonicity are sufficient conditions; QTRAN trains the sum of individual values plus a state value to equal the joint value at the greedy profile and bound it elsewhere, which is sufficient and, up to rescaling, necessary. Every single-state matrix game satisfies IGM; VDN and QMIX missed the optimum in 177 of 310 random 3×3 games. | Read |
| NOTE-1142 | Hart & Mas-Colell — A simple adaptive procedure leading to correlated equilibrium | A theorem with a proof. Regret matching, switching from one’s last action to each alternative with probability proportional to the positive regret for not having made that switch in the past, drives the empirical joint distribution of play almost surely to the set of correlated equilibria when every player uses it. Regrets vanish exactly when the empirical distribution approaches that set. A stylized variant is analysed by Blackwell approachability; the main procedure needs inertia and needs players to see one another’s past play. | Read |
| NOTE-1143 | Balduzzi et al. — Open-ended learning in symmetric zero-sum games | A framework paper with an algorithm and two small experiments. Symmetric zero-sum games between parametrized agents split into transitive and cyclic parts relative to a measure on agents. In a cyclic game an individual’s performance is meaningless, so evaluation moves to populations: the value of the game between them, which is invariant to redundant members, and the convex hull of their evaluation rows. Training each supported agent on what it beats grows that hull. | Read |
| NOTE-1144 | Sato & Crutchfield — Coupled replicator equations for the dynamics of learning in multiagent systems | A four-page letter. Two agents choose by softmax over reward memories that decay at rate α; with learning slow relative to play, each agent’s choice distribution follows a replicator dynamic in the other’s play plus α Σ_j x_j log(x_j/x_i), a pull towards uniform play. Without decay the system is conservative, Hamiltonian in zero-sum games; with decay it is dissipative. Rock–scissors–paper simulations show tori, Hamiltonian chaos, heteroclinic transients, limit cycles and chaotic attractors. Derivation plus numerics, no data. | Read |
| NOTE-1145 | Shazeer et al. — Outrageously large neural networks: the sparsely-gated mixture-of-experts layer | An engineering paper with large experiments. A sparsely-gated layer of up to 131,072 experts raises capacity over 1000-fold at roughly constant computation. Read here for one observation and one fix: without constraint the gate concentrates on a few experts, which the authors call self-reinforcing, and two auxiliary losses that penalize unequal importance and load remove it, which in one ablation also improves perplexity from 39.8 to 35.6. | Read |
| NOTE-1146 | Czarnecki et al. — Real world games look like spinning tops | A hypothesis paper with theorems and sampled empirical games. Where players can signal n bits before the outcome is settled, every win–loss pattern on 2ⁿ strategies occurs, so cycles are astronomically long. Nash clustering layers any symmetric zero-sum game, and beating a whole cluster guarantees climbing. Sampled games show cluster size peaking at middling strength. The body says the geometry is not proved, the abstract says it is. Two of the smaller results have defective statements or proofs. | Read |
| NOTE-1147 | Jacobs, Jordan, Nowlan and Hinton — Adaptive mixtures of local experts | A short paper that founds the mixture of experts. Scoring the error of a single, gate-selected expert, or the mixture’s negative log-likelihood, makes experts compete for cases, and an expert better than the gate-weighted average gains responsibility for the case. On a vowel task 4 or 8 linear experts learned in half the epochs of matched backpropagation networks at equal accuracy, divided the task by class pair, and left all but 2 or 3 experts with mixing proportion near zero. | Read |
| NOTE-1148 | Yang et al. — Mean field multi-agent reinforcement learning | An algorithms paper with a convergence proof and three experiments. Each agent’s Q-function is approximated by its interaction with the mean action of its neighbours, which makes many-agent RL tractable for hundreds of homogeneous agents. Mean field Q-learning converges to a Nash Q-value under strong assumptions, learns Gaussian Squeeze with 500 and 1000 agents where baselines fail, matches MCMC on a 20 × 20 Ising model, and wins a 64-versus-64 battle game. | Read |
| NOTE-1150 | Foerster et al. — Learning with opponent-learning awareness | A LOLA learner maximises its return after the opponent’s anticipated naive gradient step and differentiates through that step, adding a second-order term to its gradient. Two exact-gradient LOLA learners reached tit-for-tat-like reciprocity in the memory-one iterated prisoner’s dilemma where naive learners ended at mutual defection, and settled at the mixed equilibrium of iterated matching pennies where naive learners did not converge. A policy-gradient version cooperated in the Coin Game, less reliably with a learned model of the opponent. | Read |
| NOTE-1151 | Leibo et al. — Multi-agent reinforcement learning in sequential social dilemmas | A descriptive simulation study. It defines sequential social dilemmas, Markov games whose learned cooperative and defecting policies give social-dilemma payoffs, and runs independent deep Q-learners on two two-player games. Scarcity and costly conflict produce aggression in Gathering; group benefit produces joint hunting in Wolfpack; and network size moves defection in opposite directions in two games that both embed a Prisoner’s Dilemma. | Read |
| NOTE-1152 | Foerster et al. — Counterfactual multi-agent policy gradients | A centralised critic Q(s, τ, u) trains decentralised actors, and each agent’s action is scored against the critic’s average over its own alternatives under its own policy, the others’ actions fixed. The baseline does not change the expected gradient. On StarCraft micromanagement with a narrow field of view it beat independent and central-V/QV actor-critics on four maps. The counterfactual is the critic’s estimate, not a simulation, and it values alternatives the agents sometimes take. | Read |
| NOTE-1153 | Wolpert & Tumer — Optimal reward functions in distributed reinforcement learning | Members of a collective each run a reinforcement learner on a private utility, and the designer wants the world utility G maximised. A private utility is factored when it orders a member’s own alternatives, others fixed, as G does; this holds exactly when it is an increasing function of G given what the member cannot affect. The Wonderful Life Utility, G minus G with the member’s effect set clamped, is factored and easier to learn than G. In a 60-agent bar problem, clamping to the average action did best, clamping to absence nearly as well, G worst. | Read |
| NOTE-1154 | Rashid et al. — QMIX: monotonic value function factorisation | Per-agent values Q_a(τ^a, u^a) are combined into a joint Q_tot by a mixing network whose weights, generated from the state, are kept non-negative, so ∂Q_tot/∂Q_a ≥ 0. Then the individually greedy actions form a greedy joint action and agents can act on their own values. It represents more than a sum and learned the optimum of a two-step game that a sum misses; it cannot represent a joint value in which one agent’s best action depends on the others’ current actions. | Read |
| NOTE-1155 | Balduzzi et al. — Re-evaluating evaluation | A methods paper with two short applications. It reads evaluation data as antisymmetric matrices. Elo is then the row mean of the log-odds, exact when they are curl-free and movable by copies, and the Schur decomposition gives a multidimensional Elo. Nash averaging scores agents and tasks against the maximum-entropy Nash equilibrium of a zero-sum meta-game. Its proved invariance covers exact copies, and only when that equilibrium is unique. A checked counterexample shows what happens otherwise. | Read |
| NOTE-1156 | Omidshafiei et al. — α-Rank: multi-agent evaluation by evolution | A methods paper with theory and applications. Agents are ranked by the stationary distribution of a finite-population, rare-mutation Fermi imitation chain on pure profiles, unique at every finite α. As α grows its transitions become those of Markov-Conley chains, the sinks of the better-response graph, which the paper ties to chain components of the replicator flow. The limit of the ranking is not characterized, and it is not invariant to copies. | Read |
| NOTE-1158 | Bar-Tal, Yariv, Lipman and Dekel — MultiDiffusion | A training-free sampler that runs one pre-trained diffusion model on several crops or masked prompts of one canvas and, at every step, replaces their disagreeing proposals with the least-squares reconciliation, which for pixel-sampling maps is a per-pixel weighted average. It makes seamless panoramas (crop FID 10.3 against 18.4 and 45.5 for outpainting baselines) and follows rough region layouts (COCO IoU 0.26 against 0.16–0.17), most of the layout gain coming from an early masked bootstrapping phase. Its guarantee that each crop follows the reference model’s path holds only at zero loss, which disagreeing overlaps do not reach. | Read |
| NOTE-1159 | Kwon and Perchet — Blackwell approachability with partial monitoring: optimal convergence rates | Approachability of a polytope (reduced to the negative orthant) under partial monitoring, where the decision maker sees only a random signal whose law is known, can be achieved at rate O(T^-1/3) in expected distance for general signals and O(T^-1/2) when the signal law does not depend on the decision. The general algorithm lifts the problem to an auxiliary full-information approachability problem that is linear in the flags, and keeps a uniform exploration floor γ ∝ T^-1/3 so that the flags can be estimated by importance weighting. | Read |
| NOTE-1160 | Zhong et al. — Multi-LoRA composition for image generation | Two training-free ways of using several LoRAs in one diffusion sampler without merging their weights: LoRA Switch gives each LoRA its own denoising steps in rotation, and LoRA Composite averages every LoRA’s guided noise prediction at every step. On 480 sets of two to five public LoRAs, GPT-4V’s paired judgements (and two annotators on 120 images) prefer both to weight merging, increasingly as LoRAs are added. In Switch, the switching interval and which LoRA acts first change the result at equal step shares. The baseline’s merged adapter grows with the number of LoRAs, which the design does not separate from the decoding-time idea. | Read |
| NOTE-1162 | Lattimore and Szepesvári — Cleaning up the neighborhood: a full classification for adversarial partial monitoring | Completes the classification of finite partial-monitoring games against oblivious adversaries to every game, degenerate and duplicate actions included: 0 with no neighbouring pair, Θ̃(√n) locally observable, Θ(n^2/3) globally but not locally observable, Ω(n) otherwise. The missing piece is NeighborhoodWatch2, which redistributes probability onto degenerate actions in proportion to the neighbours they lie between and has a high-probability √n bound. It corrects Bartók et al.’s T^2/3 lower-bound proof, whose separating direction ignored the degenerate actions’ signals. The preprint, not the published text, also says the earlier localisation argument exchanged an expectation and a maximum. RelExp3 gives point-locally observable games an asymptotic constant depending only on K_loc, F and log K, and a factor linear in F is shown to be necessary. | Read |
| NOTE-1164 v2 | Bartók, Foster, Pál, Rakhlin and Szepesvári — Partial monitoring: classification, regret bounds and algorithms | For finite partial-monitoring games the minimax regret takes exactly four rates: 0 with one non-dominated action, Θ̃(√T) when every pair of neighbouring actions is locally observable, Θ(T^2/3) when the game is globally but not locally observable, and Θ(T) otherwise. The conditions are linear algebra on loss and signal matrices: a loss difference must lie in the span of the row spaces of the signal matrices of the actions allowed to estimate it. The stochastic classification covers every game; the adversarial one covers non-degenerate games only. | Read |
| NOTE-1166 v3 | Meng, Bau, Andonian and Belinkov — Locating and editing factual associations in GPT | Causal tracing restores one clean hidden state into a run whose subject tokens were noised, and measures how much of the correct object’s probability comes back. In GPT-2 XL, over 1000 known facts, the effect concentrates at two sites: middle layers at the last subject token, where MLP outputs dominate, and late layers at the last token, where attention does. A rank-one update of one middle MLP’s output matrix, keyed on the subject’s last token, then inserts counterfactual facts that generalize to paraphrases while sparing neighbouring subjects, which no other editor tested does on the new CounterFact dataset. The authors read this as showing where and how facts are stored. | Read |
| NOTE-1167 | Wang, Variengien, Conmy, Shlegeris and Steinhardt — A circuit for indirect object identification in GPT-2 small | A circuit of 26 attention heads in seven classes that carries GPT-2 small’s indirect-object predictions on fifteen templates, found by tracing back from the logits with path patching and checked against three criteria. Mean-ablating everything else keeps 87% of the logit difference; knocking out the three main name movers costs 5%, because other heads take over; and a greedy search finds knockouts under which circuit and model differ by up to 87%. Its by-products: heads that communicate positions rather than tokens, induction heads doing a different job from their original one, and heads that write against the answer. | Read |
| NOTE-1168 v2 | Chan, Garriga-Alonso, Goldowsky-Dill, Greenblatt et al. — Causal scrubbing | A method for testing a stated mechanistic hypothesis about how a network produces a behaviour on a dataset. The hypothesis maps an interpretable computational graph into the model’s graph; every activation is then replaced, recursively from the output, by its value on another input the hypothesis says is equivalent, and unclaimed inputs by their values on a random input. The share of the behaviour that survives is the score. It is applied to a parenthesis-balance classifier (93% down to 72% as the hypothesis is made more specific) and to induction in a two-layer attention-only model (35% for the naive hypothesis, 86% refined). | Read |
| NOTE-1169 | Mueller, Geiger, Wiegreffe, Arad et al. — MIB, a mechanistic interpretability benchmark | A benchmark for two kinds of mechanistic-interpretability method, with fixed counterfactual inputs, private test sets and new metrics. In circuit localization, edge attribution patching with integrated gradients does best on average and node-level circuits poorly; in causal-variable localization, supervised distributed alignment search gives the highest interchange-intervention accuracy, and features of public sparse autoencoders selected by a learned mask never beat the same mask on the model’s own hidden dimensions. | Read |
| NOTE-1170 | Conmy, Mavor-Parker, Lynch, Heimersheim and Garriga-Alonso — Towards automated circuit discovery (ACDC) | Names a three-step workflow behind manual circuit discovery (choose a behaviour, metric and dataset; choose the units of the computational graph; patch activations to prune the graph) and automates the third with ACDC, which visits nodes from the output back and removes each incoming edge whose replacement by its value on a corrupted input raises the KL divergence from the full model by less than a threshold. Against circuits reported by earlier manual work it ranks edges with AUCs of 0.85–0.98 on three language-model tasks, beating two adapted pruning methods there, but does poorly on compiled tracr programs unless zero activations are used, misses negative components at sparse thresholds, and keeps only one input of an OR gate. | Read |
| NOTE-1173 | Olsson et al. — In-context learning and induction heads | In 34 transformer language models, in-context learning (loss at token 500 minus loss at token 50) appears abruptly in one early window of training, together with induction heads, the only non-convex bump in the loss curve and a turn in the models’ per-token-loss trajectory; one-layer models show none of it. Letting one layer express induction (smeared keys) moves the jump, and ablating induction heads removes nearly all in-context learning in small attention-only models. The case is causal for small attention-only models and correlational or by analogy for large models with MLPs, as the authors grade it themselves. | Read |
| NOTE-1174 | Dunefsky, Chlenski and Nanda — Transcoders find interpretable LLM feature circuits | A transcoder replaces one MLP sublayer with a wide sparse ReLU MLP trained on its input–output pairs. With attention patterns frozen and LayerNorm linearized, the attribution between two transcoder features factors into the earlier feature’s activation times the input-invariant dot product of its decoder with the later feature’s encoder, so circuits through MLPs can be read partly off the weights. On GPT2-small, Pythia-410M and Pythia-1.4B transcoders sit on or above the SAE sparsity–loss frontier, under one learning rate and one seed; in nine blind case studies on GPT2-small the method recovered the feature’s meaning in five, mostly in two and failed in two. | Read |
| NOTE-1177 | Musat, Pimentel, Zucchet and Hofmann — Invariant learning dynamics of transformers in inductive reasoning tasks | For a class of “block-list” next-token tasks that includes associative recall, k-hop induction and in-context n-grams, population gradient descent keeps an attention-only transformer’s merged key–query and output weights in a subspace fixed by the data’s token-relabelling and position-shift symmetries, whose coordinates are elementary actions (“attend to the previous position”, “read the token retrieved by head 1”). On it, a saturated in-weights circuit suppresses the in-context circuit’s gradient by the fraction of data it does not solve, burstiness multiplies that gradient, initialization decides which of several induction circuits wins, and projecting a trained model onto the coordinates recovers its circuit, with “aiding” directions a head-level analysis would miss. | Read |
| NOTE-1178 | Templeton, Conerly, Marcus, Lindsey et al. — Scaling monosemanticity | Sparse autoencoders of about 1M, 4M and 34M features on the middle-layer residual stream of Claude 3 Sonnet, trained to a compute-optimal loss, explain at least 65% of its variance and yield features that are multilingual and multimodal, that mostly match no neuron, and that steer the model’s output when clamped, including features for unsafe code, bias, sycophancy and deception. Whether a concept has its own feature depends on its training frequency relative to the number of live features. The interpretability and steering evidence is shown on selected features, and the most strongly active features are mostly not those that matter to an answer. | Read |
| NOTE-1181 v2 | Lindsey et al. — On the biology of a large language model | Case studies of Claude 3.5 Haiku through attribution graphs of a 30-million-feature cross-layer transcoder, each checked by inhibiting or injecting features in the model. On single prompts: “Dallas” reaches “Austin” through “Texas” features, beside a direct shortcut; a rhyme is chosen on the newline before a line and the line is written towards it; one addition lookup-table feature serves arithmetic, citations and tables; “can’t answer” features are on by default and inhibited by known-entity features; a chain of thought can work backwards from a hinted answer; a jailbreak’s target word is not represented before it is written; a goal finetuned into a model organism is represented across every Assistant context. The authors call these existence proofs and report a satisfying graph for about a quarter of prompts tried. | Read |
| NOTE-1182 v3 | Olah, Cammarata, Schubert, Goh, Petrov and Carter — Zoom in, an introduction to circuits | A programme essay, not a study. It offers three deliberately speculative claims, as empirical claims about networks and as norms for studying them: features (directions in a layer’s activation space, often single neurons) are the unit of a network; features joined by weights form circuits whose algorithms can be read off the weights; and analogous features and circuits form across models. The examples are qualitative and from InceptionV1, with the detailed evidence promised to later articles. Polysemantic neurons are named as the main obstacle, and a car feature spread over dog neurons is the first case it calls superposition. | Read |
| NOTE-1183 | Wang, Hu, Du, Cheng, Wang and Zou — Fine-tuning mechanisms of LLMs via circuit analysis | When Pythia-1.4B is fine-tuned with LoRA on arithmetic tasks it starts out failing, EAP-IG circuits of 5% of edges found at successive checkpoints keep most of their nodes and change most of their edges (change rates 15–24% for nodes, 66–80% for edges), mostly in the middle layers; giving higher LoRA rank to the layers whose edges change most beats uniform and random rank allocation at similar size, and the union of two subtasks’ circuits approximates a two-step task’s circuit. The node–edge contrast carries no comparison against a null. | Read |
| NOTE-1185 | Ameisen et al. — Circuit tracing: revealing computational graphs in language models | A cross-layer transcoder (CLT) replaces a model’s MLPs with sparse features, each reading one layer and writing to all later ones. On one prompt, freezing attention patterns and normalization denominators and adding back the reconstruction error makes a local replacement model that reproduces the model exactly and is linear between features, so its attribution graph decomposes each feature’s pre-activation into edges. The graphs are hypotheses about the model, tested by perturbing features in it. The authors’ own measurements: the replacement matches the model’s top token on about half of prompts, and its perturbation responses agree with the model’s for a layer or two and then diverge. | Read |
| NOTE-1186 | Bayazit, Mueller and Bosselut — Crosscoding through time | Trains one sparse crosscoder jointly on the mid-layer activations of two or three pretraining checkpoints, so features share an index across them, and scores each feature’s causal importance for subject–verb agreement at each checkpoint (Relative Indirect Effect). In Pythia-1B, OLMo-1B and BLOOM-1B the important features move, by the authors’ annotation, from token-specific detectors to grammatical-category ones, and in BLOOM from language-specific to crosslingual ones for Latin-script languages; importance keeps shifting after accuracy plateaus. A method for describing which features emerge, persist and lose importance, not for predicting it. | Read |
| NOTE-1187 | Marks, Rager, Michaud, Belinkov, Bau and Mueller — Sparse feature circuits | Circuits whose nodes are sparse-autoencoder features and autoencoder error terms, found in Pythia-70M and Gemma-2-2B by linear approximations to indirect effects. On subject–verb agreement, circuits of about 100 and 500 nodes carry half or more of the behaviour on held-out pairs, where neuron circuits need far more, but without the error nodes the feature circuits carry little. The annotated circuits show two pathways for agreement across a clause. A human removing gender features from a classifier’s circuit (SHIFT) brings profession accuracy on balanced data near an oracle’s, and the method runs without supervision on thousands of behaviours clustered from the Pile. | Read |
| NOTE-1188 | Wang, Hoogland, van Wingerden, Furman and Murfet — Refined local learning coefficients of attention heads | Restricts the local learning coefficient, singular learning theory’s measure of how degenerate the loss is near a parameter, to one component’s weights (weight-refined) or to another data distribution (data-refined), and tracks it per attention head over the training of a two-layer attention-only transformer. The heads’ curves start alike and diverge, and cluster by the heads’ functional types; refinement to code separates the two induction heads; refinement to a one-layer model’s outputs locates layer-0 heads giving up simple n-grams as a circuit with layer-1 bracket-matching heads forms. One model, one run in the main text, three more seeds in an appendix. | Read |
| NOTE-1189 | Elhage, Nanda, Olsson et al. — A mathematical framework for transformer circuits | An exact reformulation of attention-only transformers, and a reading of toy models with it. Heads are independent and additive, each a QK bilinear form that sets the attention pattern and an OV map that sets what moves; with patterns frozen, the logits are a sum of path terms (direct, one per head, one per composed pair of heads). Zero-layer models are bigram tables, one-layer models bigram and skip-trigram tables dominated by copying heads, and in a two-layer model the only substantial composition builds induction heads from a previous-token head by K-composition. MLP layers, two-thirds of a standard transformer’s parameters, are left open. | Read |
| NOTE-1190 v2 | Kramár, Lieberum, Shah and Nanda — AtP*, attribution patching and its failure modes | Attribution patching (AtP) estimates every node’s activation-patching effect at once from two forward passes and one backward pass, by a first-order Taylor expansion. On Pythia models from 410M to 12B, using it to order which nodes to verify by real patching finds the largest-effect nodes far more cheaply than unordered patching or sampling-based search. Its false negatives come from saturated attention softmaxes and from direct and indirect effects that nearly cancel; AtP* recomputes the softmax for queries and keys and averages gradients with one layer dropped at a time. A subsampling test gives an upper confidence bound on the size of any nodes it still misses. | Read |
| NOTE-1191 | Bricken, Templeton, Batson, Chen, Jermyn et al. — Towards monosemanticity | Sparse autoencoders trained on the 512-neuron MLP layer of a one-layer transformer, at widths from 512 to 131,072, recover features that one blinded author and Claude both rate far more interpretable than neurons, that mostly match no neuron, whose effects on the output (which the autoencoder never saw) agree with their activations, and that recur in a second model trained from another seed. The features split into finer families as the dictionary widens, so the authors doubt there is one correct number of features. Everything is in one MLP layer of a one-layer model, and the detailed cases are chosen to be easy. | Read |
| NOTE-1192 | Geiger, Wu, Potts, Icard and Goodman — Distributed alignment search (DAS) | Causal abstraction tests whether a network implements a high-level causal model by interchange interventions under an alignment of variables, which earlier work found by brute-force search over disjoint sets of neurons. DAS learns the alignment instead: an orthogonal rotation trained by gradient descent so that swapping rotated subspaces between inputs reproduces the high-level model’s counterfactuals. In a feed-forward network trained on hierarchical equality and in BERT fine-tuned on monotonicity NLI it finds alignments with interchange intervention accuracy of 1.00 where the best localist ones reach 0.73 and 0.51. In BERT the entailment subspace decomposes into the two word identities; in the equality network it does not. On random networks DAS raises accuracy above chance only when the hidden layer is much wider than the input. | Read |
| NOTE-1193 | Marks, Treutlein et al. — Auditing language models for hidden objectives | A model trained on documents describing 52 fictional reward-model biases, then by SFT and RL to exploit 47, exploits the 5 held-out biases more (17% to 45%) and new ones described in context more than other entities’ biases; without the documents the held-out rate barely moves, and without SFT it still rises. Three of four blind auditing teams found the objective, all by finding the documents in the training data; the API-only team did not. Post hoc, semantic search matched the fastest team’s SAE route, and SAE features found the objective from the Assistant token and from bias demonstrations where search failed. | Read |
| NOTE-1194 | Hase, Bansal, Kim and Ghandeharioun — Does localization inform editing? | On 652 CounterFact facts GPT-J completes correctly, the paper asks whether the layer where causal tracing places a fact predicts where an edit of that fact succeeds. It does not. Regressing edit success on the edit layer explains most of its variance (94.7% for ROME’s rewrite score), and adding the per-fact tracing effect adds at most 3.2 points, across four editing methods and five editing problems; for ROME at layer 6 the correlation is slightly negative. ROME rewrites at over 96% at every layer but the last. The authors conclude that tracing and editing answer different questions. | Read |
| NOTE-1195 | Chan, Garriga-Alonso, Goldowsky-Dill, Greenblatt, Lin et al. — Causal scrubbing: results on a paren balance checker | Causal scrubbing applied to a small transformer that classifies parenthesis strings as balanced. Saying which three heads carry which test recovers 88–93% of the loss; adding how the heads compute their tests brings the combined hypothesis to 72%, the excess losses of its two halves roughly adding. Splitting one head’s input into separately resampled terms raised the score from 77% to 81–84%, which the authors say is an artefact of resampling, not a better explanation. | Read |
| NOTE-1196 | Chan, Garriga-Alonso, Goldowsky-Dill, Greenblatt, Lin et al. — Causal scrubbing: results on induction heads | In a two-layer attention-only model, heads 1.5 and 1.6 contribute 0.053 nats of loss on tokens where induction helps. The textbook induction circuit accounts for 35% of that. Matching on the last three tokens, layer 0’s average effect, and three layer-0 heads that attend to copies of the current token raise it to 86% (91% unscrubbed, 89% with a crude rule for the previous-token head’s deviations). The heads use more paths than the story names, but few positions. | Read |
| NOTE-1197 | Meng, Sen Sharma, Andonian, Belinkov and Bau — Mass-editing memory in a transformer | MEMIT inserts thousands of facts into GPT-J and GPT-NeoX at once. It treats the MLP output matrix of each layer in a range that causal tracing implicates as a linear associative memory, computes for each fact a target change of the hidden state at the top of that range, and spreads it over the layers with a closed-form least-squares update for the whole batch. At 10,000 CounterFact edits on GPT-J it scores 85.8 against 50.3 for ROME applied one at a time; ROME is better at a handful of edits. | Read |
| NOTE-1198 v2 | Cammarata et al. — Curve detectors | The ten curve units of InceptionV1’s mixed3b are argued to detect curves: feature visualizations are curves, unit 3b:379 fires on about 11% of dataset positions and its activation level separates blind hand labels of over 800 images, curves and imperfect curves make up 55% of its contribution to expected value, synthetic curves drive it up to 24 standard deviations, and rotated dataset examples show the ten units tiling all orientations. Causality, generality and the family are argued from these; purity is argued but its mechanistic support is deferred to the next article, and no count of how much of the layer is understood is given. | Read |
| NOTE-1200 | Geva, Schuster, Berant and Levy — Transformer feed-forward layers are key-value memories | A feed-forward layer f(x·Kᵀ)·V has the form of a key–value memory without the softmax: each hidden unit’s input weights are a key and its output weights a value. In a 16-layer WikiText-103 language model, the training prefixes that most activate a sampled key share patterns humans recognize, shallow in layers 1–9 and semantic in 10–16; projected onto the vocabulary, upper-layer values agree with the next token of their key’s top prefix in up to about 3.5% of units; and a layer’s prediction is usually a composition of many memories, refined across layers through the residual stream. | Read |
| NOTE-1201 v2 | Nanda — Attribution patching | Activation patching costs a forward pass per patched activation. Attribution patching replaces each patch by its first-order estimate on the corrupted run, (clean − corrupted activation) · gradient, so every patch costs two forward passes and one backward pass in total, and estimates add over sub-components. On IOI in GPT-2 Small it is said to track activation patching for heads and their queries, keys, values and patterns, and to fail for residual streams and the first MLP. The post argues it should fail where nonlinearities leave saturation, where an effect needs several features at once, and for large patches through LayerNorm, and frames all patching as specific to the prompt distribution chosen. | Read |
| NOTE-1202 v2 | Chan, Garriga-Alonso, Goldowsky-Dill, Greenblatt et al. — Causal scrubbing: Appendix | The appendix the main post defers to. It defines % loss recovered against a label-shuffled baseline. It gives three reasons to prefer resampling ablation to zero or mean ablation. It proves that the algorithm’s sampled inputs are the maximum-entropy distribution that keeps every interpretation node’s inputs jointly as on the data, which the record re-checked. It explains superposition as monosemantic terms plus a resampled interference term, and builds a false hypothesis that scores ten times the model’s own expected score. | Read |
| NOTE-1205 | Scherlis et al. — Polysemanticity and capacity in neural networks | In a model computing Σ(W_i·x)² + b to fit Σ v_i x_i², with independent unit-variance inputs of fourth moment k, the expected loss is (k − 1)Σ(n_i − v_i)² − 2Σn_i² + 2Σn_i²/C_i, so the marginal value of capacity is −2n_i²/C_i² and the optimum has a closed form: dense inputs (k ≤ 3) give every feature all or nothing, sparse ones (k > 3) share features whose importance lies between 2λ/(k − 1) and λ. Efficient embeddings are exactly block-semiorthogonal. Re-derived and checked. The bearing on trained networks is argued, not shown. | Read |
| NOTE-1206 | Olah et al. — An overview of early vision in InceptionV1 | Every one of the 1,056 units of InceptionV1’s first five layers is put in exactly one of 77 named groups, with each group’s share of its layer; the shares are consistent with the unit counts. By the authors’ own captions, 391 units (37%) are in groups called catch-all, broad and not well defined, low-confidence, unstudied or not understood, from 7% of conv2d2 to 48% of conv2d1. The groups are sorted by eye, and most units had “a few minutes of cursory investigation”, so the census is a denominator for THEORY-1013, not the criterion its promote_when asks for. | Read |
| NOTE-1207 | Cammarata et al. — Curve circuits | The ten curve units of InceptionV1’s mixed3b are traced back through four layers of families (Gabors, lines, early curves), each tiled along the curve’s tangent with rotational and scale equivariance; negative weights from opposite early curves remove the 180° echo, which circuit editing restores. A hand-written network built from the description approximates the units on synthetic curves, by figure, and substituting it recovers 26 of the 46 top-5 classifications lost by deleting the units, on 2,000 images. The three arguments Curve Detectors deferred are supplied, mostly qualitatively. | Read |
| NOTE-1225 | Spivak — Compositional dynamics in learning and mechanics | A category-theoretic syntax, the operad ArrSm of “smooth adaptive arrangements” (a responsive parameter space, a lens of output and input maps between manifolds, and a real potential), with two functorial semantics into polynomial coalgebras, i.e. discrete-time input-output machines. They differ only in the integrator: Φconf stores position and takes a descent step, Φphase stores position and momentum and takes a symplectic-Euler step. Φconf turns a parameterized map plus a loss into gradient descent with backpropagation, and a closed system with inverse-Hessian response into Newton’s method. One arrangement of harmonic particles on a finite directed graph gives the discrete wave equation under Φphase and the discrete heat equation under Φconf, both governed by the symmetrized graph Laplacian. The main technical result is lens internalization, a lax symmetric monoidal functor Lens_C → C. | Read |
| NOTE-1323 | Alvarez-Melis et al. — Hyperbolic OT hierarchy matching | Two Poincaré embeddings of the same WordNet hierarchy, from different random seeds, differ by more than a hyperbolic isometry: branches move relative to one another and pairwise distances change. Euclidean aligners and hyperbolic OT with an orthogonal map therefore fail (P@10 at most 11% across languages). Registering one space with a hyperbolic neural network trained through a Sinkhorn divergence recovers P@10 of 38–60% across languages and P@1 of 22% when matching a hierarchy to itself. The matching uses the symmetric hyperbolic distance, and nothing in it preserves the direction of entailment. | Read |
| NOTE-1324 | Alvarez-Melis et al. — Structured optimal transport | Replaces the linear cost of discrete optimal transport by the Lovász extension of a submodular set function on assignments, so that the cost of a coupling depends on which assignments are made together. The problem stays convex and is a zero-sum game against cost matrices in the base polytope, solved by saddle-point mirror prox with Sinkhorn and base-polytope projections; it reduces to ordinary OT when the set function is modular. Its own example cost, eq. (12), is not concave, so the cost it defines is not submodular. | Read |
| NOTE-1325 | Flamary et al. — POT | A software paper for POT, an MIT-licensed Python library of optimal transport solvers: exact transport by a C++ solver, entropic transport by Sinkhorn and stabilized variants, Wasserstein barycenters, partial and unbalanced transport, regularized transport, Gromov–Wasserstein, sliced Wasserstein and domain adaptation, with tests covering 92% of lines. It reports no experiments. Fused and partial Gromov–Wasserstein and co-optimal transport, which the record uses POT for, are not in this paper; they came in later releases. | Read |
| NOTE-1326 | Sarkar, Mukherjee and Sato — Relational composition as parallel transport | The companion paper’s premise, stated by its authors: per-entity frame freedom is “a chosen inductive bias, not a consequence of fresh names”. Under it, scores go through ⟨xᵢ, Tᵢⱼxⱼ⟩ and transports must be supplied. Its parsed-benchmark controls show the premise’s orthogonality helping on StepGame and costing 64.6 points on CLUTRR. | Skimmed |
| NOTE-1327 | Yu et al. — AvAtar active OT alignment | For alignment by entropic optimal transport, AvAtar picks which source objects to send to an oracle, who names each one’s true match. It scores a candidate by the derivative of a proxy utility of the transport plan (how near a permutation, or how consistent with the graphs, the plan is) with respect to supervising each possible match, averaged over matches by the current plan. The derivative is exact, from one singular positive semidefinite linear system solved by conjugate gradients. At a fixed budget of 20% extra labels on top of 20% given, it beats uncertainty baselines by up to 2.8 MRR points and random choice by up to 16, without variances reported. | Read |
| NOTE-1328 | Vayer et al. — Fused Gromov–Wasserstein | FGW compares two attributed graphs with one coupling whose cost mixes, with weight α, a Wasserstein term on node features and a Gromov–Wasserstein term on structure. For α in (0, 1), q = 1 and distance-matrix structures it is a metric that vanishes exactly when a bijection preserves weights, features and structure. As α tends to 0 or 1 it tends to W or GW. On benchmark graph classification it is competitive with graph kernels, but the supplement shows Wasserstein on features alone matching or beating it on several datasets. | Read |
| NOTE-1329 | Hashimoto et al. — Word embeddings as metric recovery | If text were generated by a random walk whose steps depend on distance in a latent Euclidean or Riemannian space, negative log co-occurrence over a window that grows with the vocabulary, less a term for each word, would approach the squared latent distance. GloVe, word2vec and the PMI-SVD method then have the true embedding among their optima, up to isometry and unigram terms, and a negative-binomial regression on the same relation embeds words and manifolds about as well. On real corpora the evidence for a Euclidean semantic space is a correlation of 0.75 between PMI and human similarity ratings and two nearest-neighbour statistics; the authors say the random walk is not a model of language. | Read |
| NOTE-1331 | Sarkar, Mukherjee and Sato — Attention as lattice gauge theory | Under per-entity O(d) frame freedom, transports carry invariant information only around cycles, a triangle’s invariants are class functions of its holonomy, and an equivariant edge update is Ψ(H)Tᵢⱼ with Ψ conjugation-equivariant: a sound conditional theorem, standard in lattice gauge theory and synchronization, whose “unique form” is a family. The untrained S4 experiments show composition doing the work and Wilson-loop routing adding 0.6–7.8 points; the 0.99 AUC is a ceiling the data’s construction sets, as the paper itself computes. | Read |
| NOTE-1332 | Bodnar et al. — Neural sheaf diffusion | Bodnar et al. read sheaf diffusion on a graph as a projection onto the harmonic space of a sheaf Laplacian and ask which classes of restriction maps can make that projection linearly separate a node labelling. A symmetric one-dimensional sheaf, which is a weighted graph Laplacian, cannot separate the two sides of a balanced bipartite graph; signed maps separate any two classes; no one-dimensional sheaf separates three; diagonal maps with d ≥ C stalks, and O(d) bundles for C ≤ 2d at d = 2 and 4, separate any number. Every one of these sheaves is built from the labels. Learned from features, such sheaves give competitive node classification on six small heterophilic benchmarks. The claim that the O(d) construction exists only at d = 2 and 4 is wrong: it works at d = 8 too. | Read |
| NOTE-1333 | Alvarez-Melis et al. — OT with global invariances | Optimal transport between two unregistered spaces is posed as a joint minimisation over couplings and one global linear map from a Schatten ℓp ball. Under angle preservation or whitening it reduces to maximising a dual Schatten norm of the transported cross-covariance XΓYᵀ: the nuclear norm for orthogonal maps (p = ∞) and the Frobenius norm for p = 2, which is exactly GW with cosine similarity and squared loss. Alternating SVD and Sinkhorn steps with annealed entropy solve it without adversarial initialisation. On word translation it is level with GW and behind self-learning on nine of ten directions. | Read |
| NOTE-1335 | Redko et al. — CO-Optimal Transport | COOT matches two data matrices of different shapes by optimizing two couplings at once, one between rows (samples) and one between columns (features), under a bilinear cost. With equal sizes and uniform weights it is zero exactly when the matrices are equal up to a permutation of rows and a separate permutation of columns, so its plans are determined at best up to the matrix’s joint row–column symmetries; the paper proves no uniqueness. For symmetric matrices it is a lower bound on GW, equal to it for squared Euclidean distance matrices. | Read |
| NOTE-1336 | Alvarez-Melis & Jaakkola — Gromov-Wasserstein word alignment | Unsupervised word translation is cast as a Gromov–Wasserstein problem between the two languages’ intra-language cosine-similarity matrices, solved by entropic projected gradient (each step a Sinkhorn problem), with no anchors. It matches adversarial-plus-refinement methods on related languages trained on comparable corpora at a fraction of the cost, and does markedly worse on English–Russian and English–Finnish. The objective sees only within-space similarities, so it cannot distinguish a correspondence from one composed with a symmetry of either space; the paper does not say so. | Read |
| NOTE-1337 | Søgaard et al. — Limits of unsupervised dictionary induction | The adversarial unsupervised method of Conneau et al. (2018) induces bilingual dictionaries only when the two embedding spaces are alike: it fails for English paired with Estonian or Finnish, for corpora from different domains even within English–Spanish, and for spaces trained by different algorithms. A seed of identically spelled words does as well or better. A spectral dissimilarity of small nearest-neighbour graphs tracks success across eight language pairs (|ρ| = 0.89). The failures are partly optimiser instability (some seeds succeed), and the method tested is adversarial, not GW. | Read |
| NOTE-1338 | Ward & Marco — pragmatic-similarity judgments | The first set of human pragmatic-similarity ratings: 693 pairs of a spoken dialog utterance and a re-enactment of it, made six ways so that similarity varies, rated on a 1–5 slider by 6–9 judges in three four-hour sessions, two English and one Spanish, 5,532 ratings in all. Mean pairwise Pearson correlation between judges was 0.45, 0.72 and 0.66 by session. It rose with experience and fell for pairs that differ in wording (0.80 in Session 2 without them). The 0.45 the record cites is the first session’s single-judge figure. A nine-judge mean at that level has an estimated reliability near 0.88. | Read |
| NOTE-1341 v2 | Guerreiro et al. — xCOMET | xCOMET is a learned metric for machine translation, a 3.5B or 10.7B XLM-R encoder with a sentence-score head and a token-severity head, trained on DA scores and then on MQM error spans with synthetic hallucinations. On WMT22 expert MQM it reaches segment-level Kendall τ of 0.39–0.47 and span F1 of 0.24–0.32, at or above the open metrics and GPT-4 prompting of its time, and it would have won WMT23’s metrics task. Its two heads agree with each other far better than either agrees with human annotators, partly by construction. It flags most injected critical errors and ranks fully detached hallucinations low. | Read |
| NOTE-1345 | Singh et al. — VakyArth | VakyArth is 578 items written by four native speakers, one per language, for Hindi, Punjabi, Tamil and Malayalam, over five phenomena and three formats. Five LLMs do better on MCQ than NLI in every model–language pair, and when wrong they lean to the literal reading. COMET tracks one native rater’s pragmatic-adequacy scores at r = 0.81. The translation items are not the MCQ and NLI items: in the release, 1 of 204 translation sources overlaps an inference item. So the finding that translation “does not track” pragmatic understanding compares different items. No agreement between people is reported anywhere. | Read |
| NOTE-1346 | Chu, Zhmoginov and Sandler — CycleGAN as steganographer | A CycleGAN trained to translate aerial photographs to maps and back reconstructs details of the photograph, such as the pattern of dots on a roof, that are absent from the visible map, because it encodes them in a low-amplitude, high-frequency signal in the map. Noise of three 8-bit levels, or JPEG compression, destroys the reconstruction; low-frequency changes do not. Any target photograph can be produced from almost any map by an imperceptible perturbation found by gradient descent, and the perturbation needed shrinks as training proceeds. The cycle-consistency loss is met by a channel the map’s human reader cannot see. | Read |
| NOTE-1363 | Fedus, Zoph and Shazeer — Switch Transformers | Routing each token to one expert (top-1, “switch” routing) with a fixed per-expert capacity, a load-balancing auxiliary loss α·N·Σ f_i P_i (α = 10⁻²), router computation in float32 inside bfloat16 training, a tenfold smaller initialization and higher dropout inside experts lets sparse Transformers train stably. At equal FLOPs per token on C4, Switch-Base with 64 experts reaches T5-Base’s quality about 7 times faster in wall-clock time, gains hold on most fine-tuning tasks and all 101 mC4 languages, and a 1.6-trillion-parameter model is trained. The paper reports the gains; it does not study what experts specialize in. | Read |
| NOTE-1380 | Li, Lin, Duan, Liang and Shroff — Theory on mixture-of-experts in continual learning | In a stylized continual-learning model, overparameterized linear regression tasks drawn from N ground truths, each carrying a feature signal a linear gate can read, a top-1 MoE whose gate is trained on a “locality” loss plus a Switch-style balancing loss sorts its experts into one set per task (or task cluster) within about M/η rounds. If gate training continues, the balancing loss eventually erases the gaps between experts’ gate outputs and routing goes wrong; stopping it once the gaps settle keeps specialization, and noise then spreads load within each set. Forgetting then decays as 1/t, and more experts only lengthen exploration. | Read |
| NOTE-1388 | Wang, Gao, Zhao, Sun and Dai — Auxiliary-loss-free load balancing for mixture-of-experts | Adds a per-expert bias b_i to the routing scores used only for top-K selection, not for the gate weights, and after each batch moves b_i by u·sign(mean load − load_i), so overloaded experts are selected less. On DeepSeekMoE-style models of 1B and 3B parameters trained on 100B and 200B tokens it gives slightly lower validation perplexity (9.50 vs 9.56; 7.92 vs 7.97) and much lower global load violation (0.04 vs 0.72 and 0.52) than an auxiliary loss with α = 0.001, one run each. Expert Choice routing is excluded because it leaks future tokens. | Read |
| NOTE-1401 | Désidéri — Multiple-gradient descent algorithm (MGDA) | Defines Pareto-stationarity (some convex combination of the criteria’s gradients is zero), proves it necessary for Pareto-optimality, and shows that when it fails the minimum-norm element ω of the gradients’ convex hull gives a direction −ω along which every criterion decreases, with (u, ω) ≥ ‖ω‖² for every u in the hull. Steepest descent along −ω, the MGDA, has a subsequence converging to a Pareto-stationary point when the criteria are positive and coercive. The direction depends on how the gradients are scaled; the author calls the right scaling open. | Read |
| NOTE-1415 | Chalupka, Eberhardt and Perona — Causal feature learning: an overview | A macro cause is a class of micro states with the same P(effect | man(state)); a macro intervention is any manipulation into the class, which is unambiguous by construction. The causal coarsening theorem makes the causal partition a coarsening of the observational one except on a null set, so one experiment per observational class finds it. No distribution over interventions enters the definitions. The example is a one-dimensional toy, and two displayed equations carry slips. | Read |
| NOTE-1420 | Chalupka, Perona and Eberhardt — Multi-level cause-effect systems | Defines the fundamental cause and effect of a confounded micro system as the partitions of causes by P(J | man(i)) and of effects by P(j | man(·)), learns both from experimental data, and proves they generically coarsen the observational partitions, so passive data plus one experiment per class find them. It assumes no distribution over interventions. Its claim that the effect partition is the minimal sufficient one fails for proportional, unequal columns. | Read |
| NOTE-1441 | Coifman et al. — Diffusion maps (full reading) | With the equations visible, the paper’s diffusion distance is the distance induced by the symmetric kernel ã^(m), D_m² = ã^(m)(x, x) + ã^(m)(y, y) − 2ã^(m)(x, y), with ã’s eigenvalues written λ_i², so that the map x ↦ (λ_i^m φ_i(x)) is an exact isometry for it. The truncation is claimed with a relative error 1 + O(e^{−αm}). Theorem 2.1, the ε → 0 heat-kernel limit of the α = 1 normalization, and the α-family (Eq. 14) are stated without proof; the sampling condition is that each ball of radius √ε hold enough points. Eq. 5 and Eq. 14 give the α = 0 potential opposite signs. | Read |
| NOTE-1462 | Chen, Lau, Mendel, Wei & Murfet — dynamical versus Bayesian transitions in TMS | In Elhage et al.’s toy model of superposition with two hidden dimensions, regular k-gons are critical points of a closed-form population loss, and their local learning coefficients predict, through the free energy formula, a Bayesian phase transition from the 5-gon to the 6-gon near n = 600 that MCMC confirms. SGD trajectories plateau at the same critical points and move to lower loss and higher estimated complexity. The paper’s Bayesian antecedent hypothesis is one-directional, that training’s transitions have Bayesian counterparts; it says outright that there is “no necessary relation” between the two kinds, and its main Bayesian transition, 5 → 6, was never seen in training. | Skimmed |
| NOTE-1467 | Hennick & De Baerdemacker — Almost Bayesian | Models late-training SGD as sub-diffusion on a “porous” loss landscape, with the local learning coefficient as the medium’s fractal dimension and a spectral dimension d_s fitted from displacement, R(t) ∝ t^(d_s/2λ). Where an effective scalar diffusion coefficient D_ξ is constant on a region, the steady state there is exp(−L/D_ξ), which is the Bayesian posterior tempered to an effective sample size 1/D_ξ. That within-region statement is all the derivation gives: how mass is shared between regions, which is where “accessibility” would act, is not derived, and in the one test it is fitted through a length scale. Displacement exponents are measured on MNIST MLPs, TinyStories models and ImageNet convnets. | Read |
| NOTE-1469 | Richens, Abel, Bellot & Everitt — general agents contain world models | In a fully observed, finite, communicating, stationary controlled Markov process, a deterministic goal-conditioned policy whose success probability is within a factor 1 − δ of optimal for every composite goal of depth up to n determines every transition probability to within √(2P(1−P)/((n−1)(1−δ))), and the estimate is computed from the policy alone, by asking it to choose between “at most k successes of this transition in n tries” and “more than k”. An optimal myopic policy determines nothing. “Contains a world model” means exactly this: the transition function is recoverable from the input-output map. The authors say it shows “the existence of a world model encoded in the agent’s policy, not its specific use”, and the recovery map runs from the policy, not the activations, so it holds of any implementation of the same map, a lookup table over histories and goals included. | Read |
| NOTE-1470 | Teoh — Multi-token prediction done right | The cross-entropy gradient of a multi-token-prediction head, taken at its hidden state under a shared output head, is exactly a forward-KL self-distillation gradient toward the model’s own position-aligned next-token prediction plus the next-token cross-entropy gradient at the aligned state; a straight-through routing removes the second from the heads and trunk at no cost, and at 167M–1.2B this improves next-token loss, speculative decoding and zero-shot accuracy by small margins in single runs. In the main (DeepSeek-style) experiments the student head sees the same tokens as its teacher, so the decomposition is not an information projection. | Read |
| NOTE-1471 | Wu, Yang & Sun — Self-training: denoising vs. signal forgetting | In overparameterized linear regression, iterating self-training with fresh covariates, noiseless pseudo-labels and minimum-norm students multiplies the estimate by a fresh random orthogonal projection each round. Noise inherited from the first fit decays geometrically; signal along a direction of variance s decays at the slower rate s/(s + τ). Risk is therefore U-shaped when the covariance is anisotropic, monotone when it is isotropic, and tends to the risk of predicting zero. Each student fits its teacher exactly on its own sample while losing the teacher’s function elsewhere. | Read |
| NOTE-1473 | Mandt, Hoffman & Blei — variational analysis of constant SGD | The first version of Mandt, Hoffman and Blei’s argument: constant-rate SGD as an Ornstein–Uhlenbeck process near a non-degenerate optimum, whose Gaussian stationary distribution can be fitted to a Gaussian posterior by choosing the rate, batch size and preconditioner, with SGFS analysed the same way and a variational EM scheme for hyperparameters. Every result reappears, with the same numbers, in the 2017 JMLR paper, which adds momentum, constant-rate SGLD and iterate averaging; this one has nothing the later one lacks. | Read |
| NOTE-1475 | Mandt, Hoffman & Blei — SGD as approximate Bayesian inference | Constant-rate SGD, modelled near a minimum with positive-definite Hessian A and constant gradient-noise covariance BBᵀ as an Ornstein–Uhlenbeck process, has a Gaussian stationary distribution with covariance solving AΣ + ΣA = (ε/S)BBᵀ. Choosing the rate, batch and preconditioner to minimise KL to the Gaussian posterior gives closed forms, and the preconditioner (2S/εN)(BBᵀ)⁻¹ matches the posterior at sample size N exactly. The paper is about the stationary regime of a regular model in one basin: it sets the search phase aside, and nothing in it relates training time to sample size beyond iterate averaging, whose covariance falls as 1/(examples processed). | Read |
| NOTE-1478 | Mobahi, Farajtabar & Bartlett — Self-distillation amplifies regularization | For ℓ2-regularized regression in a Hilbert space with a fixed training-error tolerance, refitting to one’s own predictions on the same inputs multiplies the labels’ eigencomponents by d_k/(c_t + d_k), where the multiplier c_t must grow as the labels shrink. Rounds therefore amplify regularization, relatively suppress small-eigenvalue components and collapse the solution to zero after finitely many rounds. A few rounds can help and more hurt, which ResNet and VGG runs on CIFAR echo by fractions of a point. The loss comes from the regularizer, not from any information withheld from the student. | Read |
| NOTE-1479 | Hossain, Kousar & Tourni — Overcoming scaling limits in on-policy self-distillation | On-policy self-distillation from a privileged-context teacher does most of its supervising on failed student trajectories, where the teacher’s advantage depends on the reference the student cannot see; Proposition 1 gives the forward-KL optimum as the context-averaged teacher with the conditional mutual information as irreducible loss. Restricting supervision to verified on-policy trajectories and using the student’s own failed attempt as the teacher’s context keeps the gain over the base model at 3.2–3.8 points from 1.7B to 8B, where standard OPSD’s falls to 0.14, in single runs. The trained objective clips each vocabulary entry and is not a forward KL. | Read |
| NOTE-1481 | LaMont & Wiggins — thermodynamics and inference | Takes the evidence as a partition function with sample size N as inverse temperature, and reads off the thermodynamic derivatives. The internal energy is the leave-one-out predictive loss; the heat capacity becomes a “learning capacity” equal to K/2 in a regular model (equipartition) and to Watanabe’s learning coefficient in a singular one, and smaller when prior or discreteness fixes parameters (“freeze-out”). A prior making the Gibbs entropy zero reproduces AIC. The correspondence is formal; nothing in it is a physical current or dissipation. | Read |
| NOTE-1501 | Vaswani et al. — Attention is all you need | The Transformer: an encoder–decoder built only from multi-head scaled dot-product attention and position-wise feed-forward layers, with sinusoidal position encodings. Attention maps a query and key–value pairs to a weighted sum of the values, the weights set by the query’s compatibility with each key. It reached 28.4 BLEU on WMT 2014 English– German and 41.8 on English–French at a fraction of earlier training cost, and transferred to constituency parsing. | Read |
| NOTE-1504 | Heymann & Vanden-Eijnden — The geometric minimum action method | The Freidlin–Wentzell quasi-potential, a double minimization over paths and their duration, is rewritten as a single minimization over curves of a parametrization-free action, which has minimizers when the end points are equilibria and is computed by a string-method-like relaxation (gMAM). For a diffusion it is V = 2 inf ∫ |b| sin²(θ/2) ds. In gradient systems with identity noise the most likely transition paths are parallel to the drift; in the nongradient Maier–Stein model they depart from the heteroclinic orbit once the drift is far enough from gradient. | Skimmed |
| NOTE-1505 | Lu, Gallagher, Michala, Fish & Lindsey — The Assistant Axis | Role vectors for 275 archetypes in Gemma 2 27B, Qwen 3 32B and Llama 3.3 70B span a low-dimensional persona space whose first component runs from fantastical roles to Assistant-like ones, with the default Assistant at one end; the contrast vector from it, the Assistant Axis, is largely present in base models. Steering along it modulates role-taking and persona jailbreaks. In synthetic multi-turn conversations the model’s position along the axis is set mainly by the latest user message, and therapy-like and AI-philosophy conversations drift away from the Assistant; capping the projection cuts persona-jailbreak harm by about 60% without measured capability loss. | Skimmed |
| NOTE-1511 | Klyubin, Polani & Nehaniv — Empowerment (2005) | The first statement of empowerment: the Shannon channel capacity from an agent’s n-step action sequence to its sensor n steps later, max_{p(aⁿ)} I(Aⁿ; S_{t+n}), counted only as far as the agent can perceive. Used as a fitness function in a grid world with a signal source, it evolves sensors that switch from displacement-sensing blobs near the source to bearing-sensing arcs far from it, and actuators that vary more, with empowerment bounded by the sensor’s capacity. | Read |
| NOTE-1512 | Marks, Lindsey & Olah — The Persona Selection Model | A named statement of an existing idea: pre-training teaches a language model a distribution over personas, post-training updates it into a posterior over Assistant personas, runtime context conditions it further, and the Assistant’s traits are a key determiner of behaviour. The evidence is surveyed, not new. Its open question is exhaustiveness, laid out as a spectrum from the “shoggoth” through the “router”, which swaps personas without any persona having the goal, to the “operating system”. | Read |
| NOTE-1514 | Klyubin, Polani & Nehaniv — Keep your options open | Empowerment, n-step channel capacity from actions to a later sensor, set in an interventional causal-network framework and extended to contextual empowerment, capacity with side information, which is never lower than the context-free value. In a cart-pole, greedy empowerment maximization balances the pole with no task given, though the states of highest empowerment are unreachable; it drives a sensor’s layout to switch between niches; and in a Sony AIBO, an evolved two-state context automaton recovers whether a book is present. | Read |
| NOTE-1524 | Chen, Arditi, Sleight, Evans & Lindsey — Persona vectors | Difference-of-means directions in the residual stream, extracted automatically from a trait’s name and description, steer and monitor evil, sycophancy and hallucination in Qwen2.5-7B and Llama-3.1-8B. The shift that finetuning induces along a trait’s vector correlates with the trait’s expression afterwards (r = 0.76–0.97); steering against the vector, or towards it during training, limits the shift; and projecting training data onto the vector flags data that will induce the trait. Prompt-time monitoring works mainly between prompt types. | Skimmed |