causality.
69 of 1538 NOTE documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| NOTE-010 v5 | Greater than the parts | A review, with no new theorem, of the ΦID theory of causal emergence. A supervenient feature V_t is k-th-order causally emergent iff Un⁽ᵏ⁾(V_t; X_t′ | X_t) > 0. A system hosts such features iff Syn⁽ᵏ⁾(X_t; X_t′) > 0 (cited as Rosas et al. 2020, Theorem 1), and Syn⁽ᵏ⁾ = D⁽ᵏ⁾ + G⁽ᵏ⁾ (downward causation plus causal decoupling). The review’s own additions are interpretive: the formalism is neutral between Granger and Pearl, coordinate dependence is defended as a feature, and it rebuts the charge that the theory makes no macro-versus-micro comparison. | Read |
| NOTE-027 | Adding causality to individuality | Of Krakauer et al.’s (2020) three mutual-information “individuality” terms (A* = I(S_{t+1}; S_t), A = I(S_{t+1}; S_t | E_t), nC = I(S_{t+1}; E_t | S_t)), Bourrat argues that nC and A* are not individuality measures. He then proposes the interventional  = I(S_{t+1}; Ŝ_t) (Eq. 6) and two ETI criteria: C1, a normalised Â_{S2}/H(Ŝ2) ≥ Â_{S1}/H(Ŝ1)|θ₀, and C2, low Var(Â_{S2} | Ê2 = ê2k). None of these is computed on any system. The paper’s claim that  ≡ I(S_{t+1}; S_t | E_t) = A in Krakauer’s model is false in general. | Read |
| NOTE-050 v2 | The relativity of causal knowledge | The paper defines SCMs as functors I → Prob and interventions as natural transformations (which requires invertible mixing functions). It encodes each non-intervened SCM as the convex space Δ(V, Ω) of all probability measures on its endogenous space, and puts these spaces on a graph as stalks of a network sheaf and cosheaf. The restriction maps are interventionally consistent α-abstractions and the extension maps are unspecified affine maps. “Relative causal knowledge” is the image of alternating restriction and extension along a path (Def. 11). The two substantive-looking theorems are either unsound or vacuous. Theorem 1 (every convex combination of a model’s interventional measures is a soft-interventional measure of it) rests on a proof step that is false. Theorems 2 and 3 hold for any pushforward map, so interventional consistency does no work in them. There are no results about the sheaf. | Read |
| NOTE-053 v4 | Transfer entropy as a log-likelihood ratio | For a parametric Markov model f(x_t | x^k_t, y^k_t; θ) of X, the likelihood-ratio estimator T̂ = −(1/(n−k)) log Λ for the null “no dependence on Y’s past” converges almost surely to the transfer entropy T_{Y→X} (Theorem 1a). Under the null, 2(n−k)T̂ is asymptotically χ²(d), with d the difference in parameter counts (Theorem 1b). For finite-state chains the ordinary plug-in estimator is exactly this estimator, with d = (a−1)a^k(b^k−1). For a linear Gaussian VAR, T̂ is half the Granger-causality statistic. | Read |
| NOTE-083 v2 | Garg & Fetzer, causal claims in economics | GPT-4o-mini is run in nine passes per paper over the first 30 pages of 44,852 NBER/CEPR working papers. Edges kept in at least 4 of 9 passes, JEL-mapped, give “claim graphs” in which the share of edges backed by DiD/IV/RCT/RDD/synthetic control rises from 7.7% (1990) to 31.7% (2020). In year-fixed-effect-only regressions, doubling causal edges is associated with +1.36 pp top-five probability and +11.2% citations, while non-causal counterparts are flat or negative. The validation checks paper-level method labels, and only weakly: RCT and field labels have precision ≤ 0.10. It never checks the edges themselves against human labels. | Read |
| NOTE-091 v2 | Pernu — Mental causation via neuroprosthetics? | The interventionist/difference-making case for nonreductive mental causation from BCIs (List & Menzies 2009; Menzies 2015; Woodward 2008a,b, 2017) misplaces multiple realisation. What the ensemble recordings show is that one cortical-level variable N (the ensemble’s average firing) is realised by many single-neuron patterns N1, N2. Each intention M corresponds to one value of N. So the counterfactual contrast “∼M □→ ∼B true, ∼N □→ ∼B false” fails, and neuroprosthetics supports mental causation only by identifying M with N, not nonreductive physicalism. | Read |
| NOTE-092 v2 | Quillien — Luck and intentional action | Across three Prolific studies (N = 198, 673, 142; Likert 1–7), judgments that “X happened because the agent wanted X” track intentionality judgments under manipulations of luck. Causation is the strongest predictor of intentionality: standardized β = .46, .56, .53, against know-how β = .33, .20, .27 and probability-raising β = .14, .16, .10. Across the 12 condition means, r = .98. Causation only partly mediates the effect of luck (64%, 45% and 49%; exploratory analyses). As the causal account predicted, Study 3 found that the effect of Process (normal vs mix-up) on intentionality is larger when the agent knows how the buttons work (interaction F(1, 423) = 32). In the morally neutral Kraemer vignette, intentionality and causation came apart. | Read |
| NOTE-111 v2 | Baedke et al. — Unknotting reciprocal causation | The paper gives an epistemic, not a metaphysical, tool for organism–environment reciprocal causation. The loop is “unrolled” into two ordered series of states, O_n and E_n, joined by interventionist (Woodward) causal arrows, and an experienced-environment node Ex_n is added as the interface. Its authors argue that this one diagram keeps an epistemically set organism–environment boundary and maps constitutive, mediational, relational, relocational and physical niche construction onto shared causal patterns. They support this with worked cases, including two multi-species ones (bumblebee–eggplant; rumen microbes–ruminant host). | Read |
| NOTE-112 v3 | Poltorak — Wave-function realism and causal closure | A closure principle has two tacit parameters, its domain and its completeness relation (causal, dynamical or explanatory). Wave-function realism falsifies spatiotemporal-domain closure (STC) but is compatible with fundamental-physical closure (FPC), so there is no “trans-domain causal breach”. The physicalist is left with a dilemma: STC is too narrow, and FPC needs an independent criterion of physicality plus a dynamical-to-causal bridge. The paper tests this across primitive-ontology, spacetime-state, multi-field, Everettian, nomological and dispositional readings. | Read |
| NOTE-137 v2 | Hazelwood — Emerging dilemma for reciprocal causation | Hazelwood defines RC as “the process of natural selection is both a cause and effect of processes such as niche construction” and EC as “natural selection is an emergent causal process”. He argues they are incompatible. Emergence here is Wilson’s synchronic dependence plus autonomy, cashed out as Pence’s composition. If EC holds, niche construction partly composes selection, so the two fail Woodward’s independent-fixability condition and cannot stand in a causal relation. If EC fails, selection is either non-causal (statisticalism) or individual-level, which is incoherent because relative fitness needs a population (Millstein, Sober). EES proponents should drop RC but can keep niche construction as a genuine cause of the evolutionary product. | Read |
| NOTE-585 | Causation and Manipulability | A survey of manipulability theories that states Woodward’s own. X causes Y iff some intervention on X with respect to Y would change Y, where an intervention is a surgical, exogenous cause of X meeting conditions M1–M4. The account is non-reductive but not viciously circular, needs no human agent, and defines total, direct and contributing causes by combinations of interventions. Invariance and higher-level causation are not discussed. | Read |
| NOTE-612 | The Topology of Causality | Builds the sheaf-theoretic framework for causality over spaces of input histories: causal functions are free assignments of outputs to tip events, causality is continuity in the lowerset topology, and the presheaf of causal functions is a sheaf exactly when the space has no “solipsistic contextuality” (always when tight). Empirical models are compatible families on any open cover; standard empirical models on causal switch spaces, including total orders, are always local. | Read |
| NOTE-624 v2 | The Geometry of Causality | Empirical models on any cover of a space of input histories are exactly the points of a causaltope, a product of simplices sliced by linear causality equations, which makes supported and causally separable fractions linear programs. Defines causal separability relative to an ambient space and shows, with entangled and contextually controlled quantum switches, that the causally separable fraction is bounded below by the separable local fraction and can equal it. | Skimmed |
| NOTE-629 | Sheaf-theoretic structure of definite causality | Extends the Abramsky–Brandenburger sheaf framework from non-locality (discrete order) to any finite causal order of events. Sections are causal functions over families of inputs on lower sets. Empirical models are causal conditional distributions forming a polytope. Locality is a global section, equivalently a mixture of deterministic causal functions. The authors withdrew the paper in 2024 because its “locale of inputs” is not fit for purpose. This reading finds that Proposition 5’s meet can leave the poset and that distributivity fails. | Read |
| NOTE-635 | The Combinatorics of Causality | Replaces causal orders by spaces of input histories (join-prime sets of partial input assignments), which can express causal constraints that depend on inputs. Defines free choice, tip events, causal completeness and tightness, three kinds of composition, and a lattice of spaces; proves that the maximal causally complete spaces are exactly the causal switch spaces; and finds 2644 complete spaces on 3 binary-input events, against 19 from definite orders. | Read |
| NOTE-646 | Combining contextuality and causality | Extends sheaf-theoretic contextuality to causal scenarios by an enabling relation on measurements and a game reading: Nature’s deterministic strategies, which may depend on causal history, replace the event sheaf, and the Abramsky–Brandenburger definitions then go through. Recovers flat contextuality and causal Bell scenarios, and expresses adaptive MBQC by Experimenter strategies. The strategy presheaf is not a sheaf: compatible deterministic local strategies can have no gluing or several. | Read |
| NOTE-696 | Causality: Models, Reasoning, and Inference | From four excerpts the author posts: an intervention do(X = x) is a surgery that deletes X’s structural equation, sets X to x and solves the rest, while observing X = x violates no equation and is ordinary conditioning, “given that we see”. Structural equations are therefore not algebraic equalities: they stand for every submodel an intervention can produce, and a structural coefficient is ∂E[Y | do(x)]/∂x, not a regression slope. The Epilogue contrasts two components of science, observation and intervention, and states the three rules of the do-calculus without their derivation. The excerpts have no three-level hierarchy of association, intervention and counterfactual. | Skimmed |
| NOTE-707 | Kenton et al. — Discovering Agents | An interventional, causal definition of agents. Mechanised causal models add a mechanism variable for each object-level variable. Decision nodes are found as variables whose mechanisms respond to downstream mechanisms (terminal edges), and utility nodes as the targets of those responses. Recovery of the true game graph is proved under five assumptions, including interventional data on mechanisms. The verdicts are sensible. The labels are frame-dependent and granularity-sensitive, which the paper states but does not treat as a threat. | Read |
| NOTE-726 | Hoel, Albantakis & Tononi — Quantifying causal emergence shows that macro can beat micro | Effective information (EI), the mutual information between uniformly imposed interventions on a system’s states and their effects, can be higher for a coarse-grained macro description than for the micro description it supervenes on. This happens when coarse-graining makes the mechanisms more deterministic or less degenerate by more than the smaller state space costs. Four toy systems of binary logic gates give gains of 0.40 to 2.92 bits. Causal emergence is defined as that gain. The examples are computed exactly; the reading of the EI peak as the level where causation happens rests on the choice of a uniform intervention distribution at each level, which the paper adopts and does not test. | Read |
| NOTE-737 v2 | Schmid, Selby & Spekkens — Unscrambling the omelette of causation and inference | In a process-theoretic framework that keeps causal and inferential structure apart, generalized noncontextuality becomes the demand that a realist representation preserve inferential equivalence. A Leibnizian classical realist representation implies a generalized-noncontextual one, so every noncontextuality no-go theorem is a no-go for Leibnizian representations; the converse is a conjecture. Bell-like theorems deny any classical realist representation, a weaker assumption, while noncontextuality theorems apply even to a prepare-measure scenario with no Bell-like constraint. | Skimmed |
| NOTE-765 | Huang et al. — internal causal mechanisms predict out-of-distribution behaviour | In Llama-3-8B-Instruct, on five tasks and ten task-preserving shifts, checking by interchange intervention whether a test input still realizes the causal variable located on in-distribution data predicts whether the output is correct better than output confidence or probes of other internal features, best on 8 of 10 shifts. A probe on the located variable beats a probe on its orthogonal complement on 7 of 10. Interchange accuracy and predictive power rise together only up to a point. | Read |
| NOTE-782 v3 | Makelov, Lange and Nanda — an interpretability illusion for subspace activation patching | A subspace activation patch can make a model behave as if a feature had changed while the patched direction is the sum of a component that varies with the feature but cannot reach the output (in an MLP, the kernel of the down-projection) and a component that can reach the output but is dormant on the data. Shown exactly in a three-unit network, empirically for MLP8 of GPT-2 Small on IOI and for MLPs of GPT2-XL on factual recall, where the effect vanishes once the disconnected component is removed. A residual stream direction passes the same test. One-dimensional fact patches and rank-1 edits are shown to be nearly interchangeable. | Read |
| NOTE-788 v2 | Das, Manikandan and Banerjee — Information flow with hydrodynamic interactions | Two colloids are symmetrically coupled by the fluid, and the trap of one is moved by recorded Ornstein–Uhlenbeck noise. Mutual information, its time-delayed form and transfer entropy recover the order drive → driven particle → other particle. Under moderate drive they also show some backflow, which disappears under strong drive (numerics only). Stronger coupling lowers the information the driven particle shares with the drive and raises what the particles share. Without the drive, the coupled particles are statistically independent. | Read |
| NOTE-809 v2 | Wu et al. — a reply to Makelov et al.’s interpretability illusion arguments | The reply argues that Makelov et al.’s “illusion” is defined so that any direction not orthogonal to the null space of the downstream weights can qualify, including the direction Makelov et al. call faithful in their toy network, and that their in-the-wild cases rest on a weak metric and on DAS trained per fact pair. The toy computation holds only for an unnormalized patch: patching the subspace the rowspace component spans, as Makelov et al. specify, recovers the full effect. Its own GPT-2 experiments agree that MLP8 carries almost no name-position information. | Read |
| NOTE-934 | Barnett, Barrett and Seth — Granger causality and transfer entropy are equivalent for Gaussian variables | For jointly Gaussian stationary processes X, Y and Z, Geweke’s Granger causality F_{Y→X|Z} = ln(|Σ(X|X⁻⊕Z⁻)|/|Σ(X|X⁻⊕Y⁻⊕Z⁻)|) equals twice the transfer entropy T_{Y→X|Z}, for multivariate predictor, predictee and conditioning variables (Eq. 13). The proof: OLS residual covariance is the partial covariance Σ(X|Y), and a Gaussian’s conditional entropy is ½ln|Σ(X|Y)| + ½n ln(2πe). The authors keep G-causality apart from physically instantiated causation. | Read |
| NOTE-935 | Seth — Measuring autonomy and emergence via Granger causality | G-autonomy ga_{X|E} is the log ratio of a variable’s prediction-error variance without and with its own past, given external variables E. G-emergence ge_{M|m} = ga_{M|m}·(1/N)Σ gc_{mᵢ→M} makes a macro variable emergent to the extent that it is both autonomous from and Granger-caused by its micro variables. Validated in a predator–prey model and a boids model, where flocking shows high G-emergence and Granger causation from the flock’s centre of mass to each boid. Seth says the measures are relative to the predictor and the variables included. | Read |
| NOTE-939 | Massey — Causality, feedback and directed information | The textbook DMC, P(y^N|x^N) = Π P(y_n|x_n), is the DMC used without feedback; the right definition is P(y_n|x^n y^{n−1}) = P(y_n|x_n). Directed information I(X^N→Y^N) = Σ I(X^n; Y_n | Y^{n−1}) is at most I(X^N;Y^N), equal without feedback (Theorem 1), and in a causal system bounds what the output says about the source (Theorem 3), so feedback cannot raise DMC capacity. Statistical dependence has no direction; directed information does. | Read |
| NOTE-940 | Schreiber — Measuring information transfer | Defines transfer entropy T_{J→I} as the Kullback entropy between I’s transition probabilities with and without J’s past, at history lengths k for I and l for J (Eq. 4), which is the conditional mutual information I(i_{n+1}; j_n^{(l)} | i_n^{(k)}). It is directional and ignores static correlation. A kernel estimator is proposed. Tent-map and Ulam-map lattices show it detecting the coupling direction where time-delayed mutual information fails; a heart–breath pair shows an asymmetry that the author says a common trigger could also explain. | Read |
| NOTE-941 | Eichler and Didelez — Causal reasoning in graphical time series models | Defines the causal effect of intervening on one component of a stationary multivariate time series on another at a later time, and relates it to strong Granger noncausality through mixed graphs. Under Assumption 2.3 (an intervention independent of the system’s past and of the other contemporaneous variables, and acting on the future only through the present), a missing Granger edge a → b means no effect on b at the next step, a ∉ an(b) means no effect at any horizon, and time-series back-door and front-door criteria identify effects from observational data. A present edge is only “potentially causal”. | Read |
| NOTE-945 | James, Barnett and Crutchfield — Information flows? A critique of transfer entropies | Information flow from X to Y is information in Y attributable solely to X’s past. Transfer entropy is not it. For Y_t = X_{t−1} ⊕ Y_{t−1} it is 1 bit, but the bit is synergistic, carried by both pasts jointly (co-information −1). For Z_t = X_{t−1} ⊕ Y_{t−1} both transfer entropies are 0, and the causation entropies (conditional transfer entropies) are 1 bit each, again localizing a joint bit. By PID, transfer entropy is unique plus synergistic information; conditioning adds synergy, it does not only remove redundancy. | Read |
| NOTE-946 | Ay and Polani — Information flows in causal networks | In a finite causal Bayesian network, X_B is causally independent of X_A imposing X_S if setting x_A after setting x_S leaves X_B’s post-interventional distribution unchanged. For every model on a DAG this holds iff every directed path from A to B meets S (ud-separation, Theorem 2). Information flow I(X_A → X_B | X̂_S) is conditional mutual information with interventional probabilities in place of observational ones, zero under causal independence. Examples show it can differ from conditional mutual information and from transfer entropy in either direction. Example 6’s one bit holds at k = 1, by the reading’s check. | Read |
| NOTE-987 | Rosas, Mediano et al. — Reconciling emergences | The primary paper behind LIT-025. A supervenient feature V is causally emergent of order k iff Un⁽ᵏ⁾(V_t; X_t′ | X_t) > 0; a system has one iff Syn⁽ᵏ⁾(X_t; X_t′) > 0, and the witness the proof gives is the microstate itself. Syn⁽ᵏ⁾ = D⁽ᵏ⁾ + G⁽ᵏ⁾ by definition of the ΦID atoms. The practical criteria Ψ, Δ, Γ are redundancy-free and sufficient only, and as proved they certify the system’s capacity, not the candidate feature: under the minimum-MI decomposition the parity of two independent self-copying bits has Ψ = 1 bit and no emergence. Ψ is the whole-minus-sum information about V’s next value minus V’s one-step dynamical dependence, which ties it to Barnett–Seth dynamical independence (THEORY-764). | Read |
| NOTE-1030 | Rosas et al. — Software in the natural world | Three exact closures of a coarse-graining Z of a stationary discrete process X, built from computational mechanics. Informational closure (the micro past adds nothing to predicting Z, which is Barnett–Seth dynamical independence) is proved equivalent to causal closure (Z’s ε-machine equals its υ-machine). For spatial maps it implies computational closure (Z’s ε-machine is a coarse-graining of X’s), and the converse fails. For Markov chains closure is tied to strong lumpability. The proofs of the main equivalences hold, re-derived in THEORY-820 and THEORY-822. The lattice claims are asserted and in part false. Several examples contain slips. The modular random walk closes by symmetry, not modularity. Nothing is said about approximate closure. | Read |
| NOTE-1032 | Beckers and Halpern — Abstracting causal models | Shows that Rubenstein et al.’s exact transformation is too permissive, because a distribution or a sparse intervention set can hide essential differences, and strengthens it in four steps: uniform transformation (exact for every distribution, equivalent to a map on contexts that commutes with every allowed intervention), τ-abstraction (τ surjective, the map on interventions fixed by τ), strong abstraction (every intervention τ can translate is allowed) and constructive abstraction (τ acts variable-cluster by variable-cluster), which it says all three of Rubenstein et al.’s examples are. | Read |
| NOTE-1037 | Rubenstein et al. — Causal consistency of structural equation models | Defines an exact transformation between structural equation models: a map τ on states and a surjective, order-preserving map ω on a declared set of perfect interventions such that pushing each low-level interventional distribution forward by τ gives the high-level model’s distribution under the image intervention. It proves that marginalising childless or never-intervened variables, averaging linear micro-variables when every column of the coupling matrix has the same sum, and taking the equilibrium of a contracting linear process with constant noise are each exact, and stresses that each is exact only for the intervention set declared. | Read |
| NOTE-1045 | Beckers, Eberhardt and Halpern — Approximate causal abstraction | Turns τ-abstraction into a graded relation. Given a metric on high-level states, the distance of a high-level model from a low-level one is the worst case, over allowed interventions and contexts, of how far the coarse-grained low-level outcome is from the high-level outcome, for the best context map; zero is exact abstraction. Probabilistic versions average over contexts and, with an intervention distribution, over the low-level ways of implementing each high-level intervention. An abstraction followed by an approximation is an approximate abstraction, the reverse order costs a distortion factor, and an approximate abstraction need not decompose into the two. | Read |
| NOTE-1089 | Acharya, Blackwell & Sen — The political legacy of American slavery | Among Southern whites today, those in counties where a larger share of the 1860 population was enslaved are more Republican, more opposed to affirmative action, more racially resentful and colder toward blacks; the association survives 1860 controls and a cotton-suitability instrument, and is not accounted for by today’s black share, by 1935–40 migration or by income inequality. The divergence in voting appears after Reconstruction, not before the war, and is smaller where agriculture mechanized early, which the authors read as a post-emancipation amplification of attitudes then passed down locally. | Read |
| NOTE-1166 v3 | Meng, Bau, Andonian and Belinkov — Locating and editing factual associations in GPT | Causal tracing restores one clean hidden state into a run whose subject tokens were noised, and measures how much of the correct object’s probability comes back. In GPT-2 XL, over 1000 known facts, the effect concentrates at two sites: middle layers at the last subject token, where MLP outputs dominate, and late layers at the last token, where attention does. A rank-one update of one middle MLP’s output matrix, keyed on the subject’s last token, then inserts counterfactual facts that generalize to paraphrases while sparing neighbouring subjects, which no other editor tested does on the new CounterFact dataset. The authors read this as showing where and how facts are stored. | Read |
| NOTE-1167 | Wang, Variengien, Conmy, Shlegeris and Steinhardt — A circuit for indirect object identification in GPT-2 small | A circuit of 26 attention heads in seven classes that carries GPT-2 small’s indirect-object predictions on fifteen templates, found by tracing back from the logits with path patching and checked against three criteria. Mean-ablating everything else keeps 87% of the logit difference; knocking out the three main name movers costs 5%, because other heads take over; and a greedy search finds knockouts under which circuit and model differ by up to 87%. Its by-products: heads that communicate positions rather than tokens, induction heads doing a different job from their original one, and heads that write against the answer. | Read |
| NOTE-1168 v2 | Chan, Garriga-Alonso, Goldowsky-Dill, Greenblatt et al. — Causal scrubbing | A method for testing a stated mechanistic hypothesis about how a network produces a behaviour on a dataset. The hypothesis maps an interpretable computational graph into the model’s graph; every activation is then replaced, recursively from the output, by its value on another input the hypothesis says is equivalent, and unclaimed inputs by their values on a random input. The share of the behaviour that survives is the score. It is applied to a parenthesis-balance classifier (93% down to 72% as the hypothesis is made more specific) and to induction in a two-layer attention-only model (35% for the naive hypothesis, 86% refined). | Read |
| NOTE-1169 | Mueller, Geiger, Wiegreffe, Arad et al. — MIB, a mechanistic interpretability benchmark | A benchmark for two kinds of mechanistic-interpretability method, with fixed counterfactual inputs, private test sets and new metrics. In circuit localization, edge attribution patching with integrated gradients does best on average and node-level circuits poorly; in causal-variable localization, supervised distributed alignment search gives the highest interchange-intervention accuracy, and features of public sparse autoencoders selected by a learned mask never beat the same mask on the model’s own hidden dimensions. | Read |
| NOTE-1170 | Conmy, Mavor-Parker, Lynch, Heimersheim and Garriga-Alonso — Towards automated circuit discovery (ACDC) | Names a three-step workflow behind manual circuit discovery (choose a behaviour, metric and dataset; choose the units of the computational graph; patch activations to prune the graph) and automates the third with ACDC, which visits nodes from the output back and removes each incoming edge whose replacement by its value on a corrupted input raises the KL divergence from the full model by less than a threshold. Against circuits reported by earlier manual work it ranks edges with AUCs of 0.85–0.98 on three language-model tasks, beating two adapted pruning methods there, but does poorly on compiled tracr programs unless zero activations are used, misses negative components at sparse thresholds, and keeps only one input of an OR gate. | Read |
| NOTE-1174 | Dunefsky, Chlenski and Nanda — Transcoders find interpretable LLM feature circuits | A transcoder replaces one MLP sublayer with a wide sparse ReLU MLP trained on its input–output pairs. With attention patterns frozen and LayerNorm linearized, the attribution between two transcoder features factors into the earlier feature’s activation times the input-invariant dot product of its decoder with the later feature’s encoder, so circuits through MLPs can be read partly off the weights. On GPT2-small, Pythia-410M and Pythia-1.4B transcoders sit on or above the SAE sparsity–loss frontier, under one learning rate and one seed; in nine blind case studies on GPT2-small the method recovered the feature’s meaning in five, mostly in two and failed in two. | Read |
| NOTE-1181 v2 | Lindsey et al. — On the biology of a large language model | Case studies of Claude 3.5 Haiku through attribution graphs of a 30-million-feature cross-layer transcoder, each checked by inhibiting or injecting features in the model. On single prompts: “Dallas” reaches “Austin” through “Texas” features, beside a direct shortcut; a rhyme is chosen on the newline before a line and the line is written towards it; one addition lookup-table feature serves arithmetic, citations and tables; “can’t answer” features are on by default and inhibited by known-entity features; a chain of thought can work backwards from a hinted answer; a jailbreak’s target word is not represented before it is written; a goal finetuned into a model organism is represented across every Assistant context. The authors call these existence proofs and report a satisfying graph for about a quarter of prompts tried. | Read |
| NOTE-1185 | Ameisen et al. — Circuit tracing: revealing computational graphs in language models | A cross-layer transcoder (CLT) replaces a model’s MLPs with sparse features, each reading one layer and writing to all later ones. On one prompt, freezing attention patterns and normalization denominators and adding back the reconstruction error makes a local replacement model that reproduces the model exactly and is linear between features, so its attribution graph decomposes each feature’s pre-activation into edges. The graphs are hypotheses about the model, tested by perturbing features in it. The authors’ own measurements: the replacement matches the model’s top token on about half of prompts, and its perturbation responses agree with the model’s for a layer or two and then diverge. | Read |
| NOTE-1187 | Marks, Rager, Michaud, Belinkov, Bau and Mueller — Sparse feature circuits | Circuits whose nodes are sparse-autoencoder features and autoencoder error terms, found in Pythia-70M and Gemma-2-2B by linear approximations to indirect effects. On subject–verb agreement, circuits of about 100 and 500 nodes carry half or more of the behaviour on held-out pairs, where neuron circuits need far more, but without the error nodes the feature circuits carry little. The annotated circuits show two pathways for agreement across a clause. A human removing gender features from a classifier’s circuit (SHIFT) brings profession accuracy on balanced data near an oracle’s, and the method runs without supervision on thousands of behaviours clustered from the Pile. | Read |
| NOTE-1190 v2 | Kramár, Lieberum, Shah and Nanda — AtP*, attribution patching and its failure modes | Attribution patching (AtP) estimates every node’s activation-patching effect at once from two forward passes and one backward pass, by a first-order Taylor expansion. On Pythia models from 410M to 12B, using it to order which nodes to verify by real patching finds the largest-effect nodes far more cheaply than unordered patching or sampling-based search. Its false negatives come from saturated attention softmaxes and from direct and indirect effects that nearly cancel; AtP* recomputes the softmax for queries and keys and averages gradients with one layer dropped at a time. A subsampling test gives an upper confidence bound on the size of any nodes it still misses. | Read |
| NOTE-1192 | Geiger, Wu, Potts, Icard and Goodman — Distributed alignment search (DAS) | Causal abstraction tests whether a network implements a high-level causal model by interchange interventions under an alignment of variables, which earlier work found by brute-force search over disjoint sets of neurons. DAS learns the alignment instead: an orthogonal rotation trained by gradient descent so that swapping rotated subspaces between inputs reproduces the high-level model’s counterfactuals. In a feed-forward network trained on hierarchical equality and in BERT fine-tuned on monotonicity NLI it finds alignments with interchange intervention accuracy of 1.00 where the best localist ones reach 0.73 and 0.51. In BERT the entailment subspace decomposes into the two word identities; in the equality network it does not. On random networks DAS raises accuracy above chance only when the hidden layer is much wider than the input. | Read |
| NOTE-1194 | Hase, Bansal, Kim and Ghandeharioun — Does localization inform editing? | On 652 CounterFact facts GPT-J completes correctly, the paper asks whether the layer where causal tracing places a fact predicts where an edit of that fact succeeds. It does not. Regressing edit success on the edit layer explains most of its variance (94.7% for ROME’s rewrite score), and adding the per-fact tracing effect adds at most 3.2 points, across four editing methods and five editing problems; for ROME at layer 6 the correlation is slightly negative. ROME rewrites at over 96% at every layer but the last. The authors conclude that tracing and editing answer different questions. | Read |
| NOTE-1195 | Chan, Garriga-Alonso, Goldowsky-Dill, Greenblatt, Lin et al. — Causal scrubbing: results on a paren balance checker | Causal scrubbing applied to a small transformer that classifies parenthesis strings as balanced. Saying which three heads carry which test recovers 88–93% of the loss; adding how the heads compute their tests brings the combined hypothesis to 72%, the excess losses of its two halves roughly adding. Splitting one head’s input into separately resampled terms raised the score from 77% to 81–84%, which the authors say is an artefact of resampling, not a better explanation. | Read |
| NOTE-1196 | Chan, Garriga-Alonso, Goldowsky-Dill, Greenblatt, Lin et al. — Causal scrubbing: results on induction heads | In a two-layer attention-only model, heads 1.5 and 1.6 contribute 0.053 nats of loss on tokens where induction helps. The textbook induction circuit accounts for 35% of that. Matching on the last three tokens, layer 0’s average effect, and three layer-0 heads that attend to copies of the current token raise it to 86% (91% unscrubbed, 89% with a crude rule for the previous-token head’s deviations). The heads use more paths than the story names, but few positions. | Read |
| NOTE-1197 | Meng, Sen Sharma, Andonian, Belinkov and Bau — Mass-editing memory in a transformer | MEMIT inserts thousands of facts into GPT-J and GPT-NeoX at once. It treats the MLP output matrix of each layer in a range that causal tracing implicates as a linear associative memory, computes for each fact a target change of the hidden state at the top of that range, and spreads it over the layers with a closed-form least-squares update for the whole batch. At 10,000 CounterFact edits on GPT-J it scores 85.8 against 50.3 for ROME applied one at a time; ROME is better at a handful of edits. | Read |
| NOTE-1201 v2 | Nanda — Attribution patching | Activation patching costs a forward pass per patched activation. Attribution patching replaces each patch by its first-order estimate on the corrupted run, (clean − corrupted activation) · gradient, so every patch costs two forward passes and one backward pass in total, and estimates add over sub-components. On IOI in GPT-2 Small it is said to track activation patching for heads and their queries, keys, values and patterns, and to fail for residual streams and the first MLP. The post argues it should fail where nonlinearities leave saturation, where an effect needs several features at once, and for large patches through LayerNorm, and frames all patching as specific to the prompt distribution chosen. | Read |
| NOTE-1202 v2 | Chan, Garriga-Alonso, Goldowsky-Dill, Greenblatt et al. — Causal scrubbing: Appendix | The appendix the main post defers to. It defines % loss recovered against a label-shuffled baseline. It gives three reasons to prefer resampling ablation to zero or mean ablation. It proves that the algorithm’s sampled inputs are the maximum-entropy distribution that keeps every interpretation node’s inputs jointly as on the data, which the record re-checked. It explains superposition as monosemantic terms plus a resampled interference term, and builds a false hypothesis that scores ten times the model’s own expected score. | Read |
| NOTE-1216 | Flack — Coarse-graining as a downward causation mechanism | A conceptual paper, not a report of new data. Downward causation is made operational as components tuning their behaviour to their own estimates of coarse-grained variables that they collectively compute. In its weak, “apparent” form only one component need tune, to a variable slower than the behaviour producing it, and estimates need be neither correct nor shared. The strong, “effective” form needs the variable to be predictive (slow), robust, used nearly universally, agreed on, and gaining mutual information with the microscopic behaviour; only then does Flack speak of a new organizational level. Regeneration of the variable by its use is not among the criteria; what reinforces itself is convergence of the estimates. The macaque power distribution is the worked example, reviewed from earlier papers. | Read |
| NOTE-1342 | Felekis et al. — Causal optimal transport of abstractions | COTA learns a stochastic map from a low-level to a high-level causal model’s state space from samples of paired interventional distributions, given both DAGs and the map between interventions, by solving one entropic optimal-transport problem per intervention with a cost that rewards compatibility with the paired interventions and a term meant to impose do-calculus relations between plans. As written, that term depends only on the plans’ marginals, which the transport constraints fix, so it is constant on the feasible set and its weight acts only by rescaling the other terms. The reported gains come from the compatibility cost when interventions are many; with few, a Hamming cost does better. | Read |
| NOTE-1347 | Simoes, Dastani and van Ommen — Causal information bottleneck | The causal information bottleneck trades I(X; T), the information a variable abstraction T keeps about X, against Ic(Y | do(T)), the reduction in Y’s entropy when T is set by intervention under a chosen protocol over X. An intervention on T is defined as a mixture of interventions on X, weighted by Bayes inversion of the encoder under a uniform prior, and is identifiable by a backdoor formula. At full weight on control, the optimum is the partition of X by p(y | do(x)), the causal macrovariables of Chalupka et al., which the ordinary bottleneck misses under confounding. Three toy SCMs confirm it; nothing larger is tried. | Read |
| NOTE-1359 | Jansma and Hoel — Engineering emergence | Extends causal emergence 2.0 from one micro→macro path to the whole lattice of partitions of a Markov chain’s states: a partition’s ∆CP is its CP minus the largest CP of any finer partition, and the partitions with ∆CP > 0 form the “emergent hierarchy”. Shown on 5- and 8-state chains by enumeration and on 40-state preferential-attachment chains by a greedy sampler, where the spread of ∆CP within and across levels peaks near linear attachment; cycles let a single emergent scale be placed at any level. No theorem; the dynamical-consistency filter of CE 2.0 is dropped. | Read |
| NOTE-1387 v2 | Hoel — Causal emergence 2.0 | Replaces CE 1.0’s single EI-maximizing macroscale with a path from the microscale up through coarse-grainings of a Markov chain, along which each step’s gain in CP (determinism minus degeneracy, both normalized, uniform interventions per scale) is that scale’s causal contribution; causal emergence is the total gain and “emergent complexity” the entropy of the gains along the path. Only dynamically consistent coarse-grains are admitted. Shown on three hand-built 8-state chains; the multi-path version is left open, and the paper’s claim to capture all macroscale causation is asserted, not proved. | Read |
| NOTE-1399 v2 | Zhang et al. — Dynamical reversibility and causal emergence based on SVD | Measures how close a Markov transition matrix is to a permutation by Γ_α = Σσᵢ^α of its singular values, shows log Γ_α − log N ≤ EI ≤ (2/α) log Γ_α, and defines causal emergence without a coarse-graining as the rise in the average singular value when those at or below a threshold ε are dropped. With ε = 0 it is a rank test; with ε > 0 it depends on ε, and any matrix with unequal singular values shows it for some ε. Its SVD-plus-k-means coarse-graining matches EI maximization on Hoel’s Boolean networks, but its claim that the lumping commutes with the dynamics for any grouping (Proposition 5) is false. | Read |
| NOTE-1413 v2 | Eberhardt and Lee — Causal emergence: when distortions in a map obscure the territory | Under Hoel’s effective information, with uniform interventions at each level and macro interventions as uniform averages over micro states, the EI-optimal coarse-graining can merge micro states whose effects differ widely, so the macro variable has ambiguous manipulations; and the EI-optimal abstraction of a two-step matrix differs from the square of the one-step abstraction, so abstraction and marginalization do not commute. Both are traced to the maximum-entropy distribution, which the authors call extraneous to the system. The numbers check. | Read |
| NOTE-1415 | Chalupka, Eberhardt and Perona — Causal feature learning: an overview | A macro cause is a class of micro states with the same P(effect | man(state)); a macro intervention is any manipulation into the class, which is unambiguous by construction. The causal coarsening theorem makes the causal partition a coarsening of the observational one except on a null set, so one experiment per observational class finds it. No distribution over interventions enters the definitions. The example is a one-dimensional toy, and two displayed equations carry slips. | Read |
| NOTE-1416 | Comolatti and Hoel — Causal emergence is widespread across measures of causation | Writes a dozen measures of causal strength as functions of sufficiency and necessity, or of determinism and degeneracy, and finds every one showing causal emergence on a bipartite Markov chain whose macro is a deterministic NOT gate, under three intervention distributions; the system-level EI under the stationary distribution shows none, as data processing requires. The model is hand-built for a deterministic macro, so it is the kind of result THEORY-787’s promote_when says cannot settle it. | Read |
| NOTE-1420 | Chalupka, Perona and Eberhardt — Multi-level cause-effect systems | Defines the fundamental cause and effect of a confounded micro system as the partitions of causes by P(J | man(i)) and of effects by P(j | man(·)), learns both from experimental data, and proves they generically coarsen the observational partitions, so passive data plus one experiment per class find them. It assumes no distribution over interventions. Its claim that the effect partition is the minimal sufficient one fails for proportional, unequal columns. | Read |
| NOTE-1421 | Lynch et al. — RefraMED, the NIHR EME monograph | The full report of RefraMED. Beyond the BJP paper it gives the trial’s mechanism design and its failure: RO DBT participants were also randomised to a therapy room, written feedback and mode of reimbursement, as instruments for sessions attended, alliance and DBT skills, and the instruments were too weak to use. Without them, the decomposition of the 7-month effect is implausible, and the planned analyses of temporal order are not reported. Its adverse-event test, p < 0.01 for eight reactions in the RO DBT arm against none, does not reproduce (about 0.029), and differs from the BJP paper’s 0.004. | Read |
| NOTE-1490 | Baumgartner & Casini — abductive theory of constitution | Mutual manipulability fails in every case, not some: the surgical interventions it needs cannot exist, and relaxed ones are all fat-handed, so every constitutive model has a pure common-cause model with the same correlations under all manipulations. Unlike causal underdetermination, this survives ideal experimental conditions, because the manipulation itself confounds; there is no crucial experiment for constitution. Their alternative defines constitution by the impossibility of decoupling: a redundancy-free set of parts such that every cause of the phenomenon is a common cause of it and some member, and no expansion of the variable set breaks this. The inference is abductive and only inductively corroborated. | Read |
| NOTE-1491 | Craver & Bechtel — top-down causation without top-down causes | Within levels of mechanisms, talk of top-down and bottom-up causation can be translated into mechanistically mediated effects: intralevel causal chains plus the constitutive relation between a whole and its components. On this account the relation between levels is constitution, not causation. The paper also concedes that the experiments used to find components are symmetrical, share a form with Mill’s methods, and show only prima facie that something is a component rather than a background condition. So they do not by themselves say where a mechanism’s boundary lies. | Read |
| NOTE-1493 | Baumgartner & Gebharter — fat-handedness | If a mechanism’s macro behaviour supervenes on its constituents’ behaviours and constitution is non-causal, every cause of the macro behaviour is a common cause of it and of some constituent’s. Under Woodward 2003 no top-down intervention can then change a constituent, so mutual manipulability entails that nothing constitutes anything; under Woodward’s relaxed interventions it makes whole and part causally related; with a temporal clause it is consistent but its mutual manipulations are all fat-handed and fit a model with no constitution equally well. Constitution is supported only by the second-order fact that no surgical cause of the whole turns up as the variable set grows. | Read |
| NOTE-1494 | Baumgartner, Casini & Krickel — horizontal surgicality | Upper and lower levels of a mechanism can only be manipulated via common causes, so an interventionist analysis of constitution needs a restricted kind of fat-handed intervention. Counting every common cause whose effects stand in supervenience begs the question; counting every common cause cannot tell constituents from sterile effects or detached parts. An intervention that directly changes at most one behaviour on each lower level, and changes its target and the phenomenon simultaneously, is horizontally surgical. Given that every change in a phenomenon is simultaneously realized on every lower level, that simultaneous changes are not causally related, and that co-occurrence is transitive, one such intervention suffices for constitution. The proof presupposes settled parthood and levels. | Read |