nucleation

LIT-1583: Causal scrubbing: Appendix

1 min read

Properties6
StatusActive — read 2026-10-27 (NOTE-1202) in full, from the Alignment Forum's own API; worth reading for the details the main post defers: the definition of the metric, the full case against zero and mean ablation, how unimportant inputs are grouped, a proof that the scrubbed inputs are the maximum-entropy distribution consistent with the hypothesis (THEORY-1025), how superposition is handled, and a second constructed false hypothesis that scores above the model. Read it knowing that its Lemma 1 says "product distribution" where its proof shows the joint, and that it says both results posts use a randomized-loss baseline when the induction post does not.
Version2
Tagscausality, representation-learning, anthology-candidate
ExtendsLIT-1549 — Causal Scrubbing: a method for rigorously testing interpretability hypotheses
Cited as “paper” byNOTE-1202 — Chan, Garriga-Alonso, Goldowsky-Dill, Greenblatt et al. — Causal scrubbing: Appendix
Cited as “source” byTHEORY-1025 — The inputs causal scrubbing samples for a treeified hypothesis are the unique maximum-entropy distribution, on finite inputs, among those that give the inputs of every interpretation node the joint distribution they have on the data, so whatever the hypothesis does not claim is resampled as independently as the claims allow, THEORY-1029 — On a three-layer parenthesis-balance classifier, causal-scrubbing hypotheses that resample the terms of one head's input separately scored 81% and 84% of the loss recovered against 77% for the less specific hypothesis they refine, and the authors attribute the rise to resampling washing out a signal, not to a better explanation

Graph View

Chan, L., Garriga-Alonso, A., Goldowsky-Dill, N., Greenblatt, R., Nitishinskaya, J., Radhakrishnan, A., Shlegeris, B. & Thomas, N. (2022). Causal scrubbing: Appendix. AI Alignment Forum, 3 December 2022.

Read in full on 2026-10-27 (NOTE-1202). One account was drawn from it: THEORY-1025.

Why this is here

The owner asked on 2026-10-27 to “get started on the seven works and markov IBP” that the circuits readings of 2026-10-26 had named without registering. The causal-scrubbing reading (NOTE-1168) took its case for resampling ablation, the field’s standing convention (LIT-1547, LIT-1531), from the main post’s summary, “details deferred to the appendix post”. This is that appendix. Its maximum-entropy result says precisely what a scrubbed run assumes about everything a hypothesis leaves unsaid, which bears on THEORY-978 and on what a passing hypothesis certifies (CLAIM-454).

Backlinks