nucleation

CLAIM-520: The value of keeping a corrective capacity is conditional on objectives, uncertainty and cost: a free, optional audit cannot lower a single time-consistent decision-maker's expected performance, but an audit that costs C to maintain and bears on k decisions with error rate ε is worth k(0.25 − 0.5ε) − C in the exchange's model, negative when C is large or k small, and where no hypothesis fits the world, correction gains no reward and improves only calibration

2 min read

Properties8
StatusActive
Rolethesis
Warrantdemonstrated
Tagsself-governance, information-theory
Linedistributed-agency
UndercutsARG-042 — The record's constructions of maintained correction show that self-maintaining organizations ought to preserve their corrigibility
ComplementsCLAIM-462 — An arrangement of institutions can keep, or increase, its capacity for effective action while its capacity to be effectively corrected by others degrades, because the two are maintained by different relations, so the first is no evidence of the second
Rendered intodocs/arguments-by-conclusion.md

Graph View

The claim

Free information. For a single decision-maker with a fixed objective and time-consistent preferences, an optional observation that costs nothing cannot lower optimal expected performance. The agent can always ignore it. This is the ordinary non-negative value of information.

Costly correction. The exchange’s model is the regime-switching world of CASE-186. With the audit, the agent acts on the right regime with probability 1 − ε and earns 0.75 − 0.5ε per decision. Without it the agent earns 0.5, so the gain is 0.25 − 0.5ε per decision. Over k decisions, against a maintenance cost C, the net value is k(0.25 − 0.5ε) − C. It is negative when C is large, k small or ε near ½.

Correction with nothing to gain. In the regime no hypothesis fits, where Y is ½ under either action, correcting the model gains no reward. It improves calibration only, by KL(Bern(½) ‖ Bern(¾)) = ½ ln(4/3) ≈ 0.144 nats per observation. Whether that is worth paying for is a question about objectives.

What it does not say

  • That costless information never hurts. The first result assumes one time-consistent decision-maker. A divided agent can do better with fewer options (CASE-128), and A29 endorsed that in the same synthesis without noting the tension.
  • Anything about accountability. Epistemic correction answers to evidence and organizational correction to viability. Correction by others answers to their standing, and self-correction is not accountability (CLAIM-462).
  • That a corrigible process is good. “A process that obeys every attempted correction is merely vulnerable to redirection. A process that resists every correction is insulated from accountability” (A24).

Backlinks