nucleation

CLAIM-511: A self-maintaining organization ought to preserve its corrigibility

2 min read

Properties10
StatusProposed
Version2
Roledisclaimed
Warrantinterpretive
Tagsself-governance, ethics
Linedistributed-agency
Bears onQUESTION-128 — Does the record's organizational account give a path to normativity from first principles: can norms of function and failure be defined from an organization's own dynamics, and what more do practical and moral authority need?, QUESTION-129 — Can relationally indexed normative judgments be objective about the standards and perspectives themselves, so that some conflicts between standpoints have objectively better resolutions, without presupposing the normativity they were meant to explain?, QUESTION-087 — Can an authority structure that reproduces itself be pathological by a standard grounded in the organization's own organization, and not in goals assigned to it from outside?
ComplementsCLAIM-462 — An arrangement of institutions can keep, or increase, its capacity for effective action while its capacity to be effectively corrected by others degrades, because the two are maintained by different relations, so the first is no evidence of the second
Cited as “concludes” byARG-042 — The record's constructions of maintained correction show that self-maintaining organizations ought to preserve their corrigibility
Rendered intodocs/arguments-by-conclusion.md

Graph View

Not claimed

The record does not claim this. Its constructions do not establish it (ARG-042), and the value of corrigibility is conditional (CLAIM-520). The text of version 1 follows.

What is not claimed, and why (version 1)

The constructions of the exchange (CASE-190, CASE-186) and the record’s own (CASE-161, CASE-162) show what corrigibility is made of and what it costs to keep: evidence that does not pass through the interpretation it tests (CLAIM-342), a reserved budget for it, and machinery that survives its own corrections. They show it is a capacity. They do not show an organization ought to keep it.

What was shown is conditional. In A24’s model, a free and optional audit cannot lower expected performance, the ordinary result that costless information has non-negative value. With k decisions affected, an error rate ε and a maintenance cost C, the audit’s net value is k(0.25 − 0.5ε) − C, which is negative when C is large or k small. In the regime no hypothesis fits, correction gains no reward and improves only calibration, by ½ ln(4/3) ≈ 0.144 nats. Whether that is worth paying for is a question about objectives.

Three further limits:

  • Costless information is not always harmless to an agent that is not one time-consistent decision-maker. A divided agent can do better with fewer options (CASE-128), which A29 endorsed in the same synthesis without noting the tension.
  • Corrigibility to whom. Epistemic correction answers to evidence, organizational correction to viability, and accountability to others’ standing. Self-correction is not accountability (CLAIM-462).
  • A process that obeys every correction is not thereby good. “A process that obeys every attempted correction is merely vulnerable to redirection. A process that resists every correction is insulated from accountability” (A24).

What would change this

A normative argument, from a declared standard, that an organization of a stated kind ought to keep a stated corrective capacity. That is QUESTION-128’s and QUESTION-129’s territory, and viability alone cannot supply it (CLAIM-075).

Backlinks