THEORY-451: A singular causal claim, that C will contribute or did contribute to E in a particular setting, is warranted only by joint evidence on each step of the causal process there, its causal principle, its support factors and the absence of its derailers, which no single method supplies; a well-run RCT secures at most the study population's average effect, and often not even that rigorously, and its result is not by its method evidence about any other population

Source

Cartwright (2021), LIT-213 (Synthese 199:13095–13119), read in full in NOTE-161. The full key to Fig. 1 is in Cartwright et al. (2020), not read.

What was actually shown

Rigour defined (§3, p. 13105). A method is rigorous when there is a valid argument that its results are evidence for the conclusion, and its premises are well established or “secured by the design”. The author reconstructs this from the credibility-revolution literature, having found no explicit definition.

RCTs against that standard (§3). Under (i) a potential-outcomes equation y(u) = β(u)·x(u) + w(u) and (ii) orthogonality of β and w to x, the difference in arm means is an unbiased estimate of the study-population average effect, “unbiased” meaning over endless repetitions on the same population. Randomisation secures orthogonality only at assignment; blinding removes bias only “in the sense of prejudice”. Post-assignment differences between arms need further, non-design evidence, and premise (i) carries metaphysics (Davidson’s principles, Mackie’s INUS conditions) that no rigorous method makes credible. “External validity” is a misnomer: nothing about the method makes the result evidence about another population Q, unless P was randomly sampled from Q (fn. 13).

The alternative (§4). A causal-process-tracing theory of change has five elements: the significant steps, the causal principle by which each leads to the next, support factors, derailers, and safeguards. Three evidence demands follow: that each principle operates and is triggered, that support factors take the right values, that derailers are absent or guarded. Evidence is gathered “by whatever method” suits each item, and “to the extent that any have no evidence in their favor, to that extent our case … is weakened” (p. 13112). The World Vision Indonesia mHealth programme illustrates it: untrained volunteer health workers make an ability-to-use support factor salient.

What it can do that an RCT cannot (p. 13109). Give evidence about individual units; predict for an unstudied setting without transport assumptions; supply the mechanistic evidence the Russo–Williamson thesis asks for.

What could have come out otherwise. Little in the paper is a test. The What Works crime toolkit is a case where the review is rigorous about whether alley-gating has reduced crime and says the contextual factors “could not be tested”, which are what a prediction for one place needs (p. 13107).

What this does not say

• That RCTs are worthless. Rigour should be used where available; it should not “dictate what methods can be used” (§5).
• How often RCTs fail to secure the average effect. “Often” is not quantified (C3, moderate).
• That theory-of-change predictions are reliable. “Can yield reliable predictions” (p. 13110) is asserted; no record of such predictions is reported (C6, weak).
• How to weigh the diverse evidence. There is no aggregation method (caution c), pointing to van Evera’s tests and Bayesian process tracing.
• How to transport a theory of change. “I don’t have much to offer” (caution e).
• That one traced pathway settles the outcome. Masking, dual capacities and other causes in w can still defeat E, and showing that C caused a bad E does not show that removing C prevents it (caution b).
• That advocates who mean something weaker by rigour are answered. A reading of rigour as “best available protection against selection bias” is not addressed.
• That every step of every process falls under a principle. That is the method’s most controversial assumption, and where it fails the template loses its guide.

Connections

• Worrall (THEORY-491) argues that randomisation’s only sound rationale is protection against selection bias. Cartwright grants randomisation its internal role and argues that its reach stops at the study population. The two deflate the evidence hierarchy from different sides.
• THEORY-138 is Woodward’s interventionist account of causation. Cartwright needs causal principles at each step, which that account does not supply.
• Garg et al., LIT-079, document the rise of RCT-, DiD- and IV-backed causal claims in economics working papers, the narrowing Cartwright describes in §2. Neither cites the other.
• Reiss, LIT-118, and Leonelli (THEORY-637) are the record’s other readings against a single privileged method.