Source
Worrall (2007), LIT-175 (Philosophy Compass 2(6), 981–1022), read in full in NOTE-126. Page references are the journal’s.
What was actually shown
The target (§4 frame). Evidence hierarchies let high-quality RCTs “trump any number of observational studies” (Barton 2000), however well matched and however large. The question is whether general principles of evidence underwrite that. The shared core Worrall assumes is that a theory gets little support from data its plausible rivals also entail (fn. 6).
Five arguments, four rejected.
- (a) Fisher’s significance-test logic. In the tea-tasting example a perfect score has p = 1/70 under randomised order, but after randomisation there remain indefinitely many ways the null can be false (a confederate, visual cues, a belief that cups alternate, which gives p = 1/2). It fails “even in its own terms”.
- (b) Unknown confounders. “Probable balance” is defined only over indefinite repetitions of the division, “But medical researchers … do it once!” (p. 1007). Over an open-ended list of unknown factors it is less defined still. The Leibovici retroactive-prayer trial (n = 3393; length of stay p = 0.01, fever duration p = 0.04) was impeccably randomised and double-blind. Randomisation is neither necessary (appendectomy and other untrialled standard treatments) nor sufficient for real evidence.
- (d) Observational studies “exaggerate”. Inferring exaggeration presupposes that RCTs give the true effect. Newer comparisons (Benson & Hartz; Concato et al.; Kunz & Oxman; LeLorier et al.) do not show systematic overestimation.
- (e) Probabilistic causality (Cartwright, Papineau, Pearl): the common-cause argument is (b) in another guise. This is asserted here and argued elsewhere.
One accepted. (c) Selection bias: if clinicians choose arms, their prognostic views can confound the trial. This is “a cast-iron argument for randomization”, endorsed by Bayesians too (p. 1009). But it makes randomisation “very much a means to an end”, and haphazard allocation (alternation, birthday order) would serve (p. 1010).
Conclusion. What matters is balance on plausible confounders, a placebo or active control, and blinding. Whether balance comes from randomising and checking or from matching “seems epistemically unimportant”. Carefully matched historically controlled trials can give strong evidence of large effects; RCTs are “useful” for small expected effects (p. 1018).
What could have come out otherwise. The meta-research on observational estimates could have shown consistent overestimation by matched designs. It is mixed. The rest is argument, and the reading grades it moderate (C3, C4, C9) except the selection-bias concession (strong).
What this does not say
- That randomisation is worthless. It is sound protection against selection bias, and the paper keeps RCTs for small effects.
- That randomised designs are no more reliable in practice. The argument against (b) shows that frequentist balance is no single-case guarantee. It does not compare track records, and the cited comparisons disagree with each other.
- That background knowledge suffices to name the plausible confounders. That is what defenders of randomisation doubt for small effects, and the reply, that such effects may not be worth having, is one undeveloped sentence (C11).
- That haphazard allocation is safe. Alternation and birth order can be predictable or correlate with prognosis. The paper does not discuss it.
- Anything about allocation concealment as a separate virtue. It is folded into selection bias.
- That repeated randomisation is idle. The single-case argument does not touch designs that do repeat the division, where the long-run rationale applies (NOTE-126).
Connections
- Cartwright (THEORY-451) argues, from a different definition of rigour, that even a well-run RCT secures at most a study-population average and is not by its method evidence about another population. The two accounts deflate the hierarchy from different sides and do not conflict. Worrall lists Cartwright’s 1989 book among randomisation’s defenders.
- Reiss, LIT-118, states the eliminative core Worrall calls “scientific commonsense”: warrant comes from eliminating relevant alternatives.
- Leonelli (THEORY-637) criticises the RCT as a universal exemplar of reproducible research, from a practice-oriented direction.
- THEORY-138 is the interventionist account of causation. The probabilistic causality defences in (e) are close relatives, and Worrall answers them only in summary.