NOTE-1247: The epistemic challenge to longtermism

Contribution

It turns a common worry, that we cannot predict the far future well enough for it to guide us, into a model with numbers. Tarsney grants the longtermist every normative premise (precise probabilities, total welfarist consequentialism, expected value maximization, which he calls “expectational utilitarianism”), errs against longtermism on every empirical input, and asks whether a longtermist intervention still beats a good neartermist one. His answer is a qualified yes: by the rules of expected value, but on some views only through minuscule probabilities.

Key insight

The chance that an intervention’s effect is undone or made irrelevant acts like a discount rate. If such “nullifying events” arrive at a constant rate, value that grows only polynomially (cubically, as settlement spreads) is eventually overwhelmed by the exponential discount. So the size of the future does not automatically win. What decides the comparison is the long-run rate of those events, and EV(L) is “nearly (inverse) quartic in some ranges of r, so that an order-of-magnitude decrease in r corresponds to a four-order-of-magnitude increase in EV(L)” (p. 25).

Assumptions

• Expectational utilitarianism (p. 4): precise probabilism, total welfarist consequentialism, expected value maximization. The point is to test the epistemic challenge where the normative frame favours longtermism.
• The ought is evidence-relative (p. 5): what one should do given the probabilities one’s evidence and reasoning capacities support.
• Persistent-difference strategies only (pp. 6–7): put the world in a better state S and hope it stays there. Speed-up strategies are set aside. Their value turns on four factors: importance, tractability, persistence and complement persistence. The paper models the last two.
• Exogenous nullifying events (ENEs) (pp. 13–14). Negative ENEs put the world in not-S (an existential catastrophe); positive ENEs put it in S (another civilization arises). After the first ENE, the earlier choice no longer matters. Their rate r is constant, which Tarsney defends as conservative against the “time of perils” view (p. 18).
• Value tracks resources. In the cubic growth model, value per year grows with the number of stars settled, at constant speed in all directions. In the steady state model, humanity stays in the Solar System at constant value.
• The far future begins 1000 years from now (p. 13), the year 3022.

Key results

• The working example (pp. 11–12). A philanthropist chooses how to spend $1 million.

  • N, the neartermist benchmark: vitamin A supplementation at about $3000 per life saved, 333⅓ lives at 30 QALYs each, so 10,000 QALYs. This defines the unit: one “valon” is 10,000 QALYs, and EV(N) = 1 V.
  • L, the longtermist option: research on pandemic risk from novel pathogens, to raise the probability that “the accessible universe contains an intelligent civilization” in 3022.

• Parameter values (pp. 18–23).

  • p = 2 × 10^-14 (p. 19, fn. 23): a “very safe” 1% change in survival to 3022 if all of humanity worked on it for 1000 years, times the fraction of humanity’s labour $1 million buys (2 × 10^-12). Other sources would give at least 6.4 × 10^-14 and up to 8 × 10^-10 (Millett and Snyder-Beattie) or about 10^-7 (Todd) (fn. 24).
  • v_s, value per settled star per year. “Space Opera”, 300 million people per star at 1 QALY a year: 3 × 10^4 V. “Dyson Spheres”, Bostrom’s figure of 10^25 human-equivalent lives per star: 10^20 V. The model uses the first.
  • v_e, value on Earth: today’s 6 billion QALYs a year, so 6 × 10^5 V a year.
  • Star densities 2.2 × 10^-5 per cubic light year in the Milky Way and 2.9 × 10^-9 in the Virgo Supercluster; settlement speed s = 0.1c; settlement starts at t_l = 0; the bound t_f = 10^14 years.
  • r is left free. Tarsney thinks r below 10^-2 per year is safe (the most pessimistic estimate of near-term risk in the academic literature, Rees’s 0.5 for this century, implies about 6.9 × 10^-3 a year) and sees no clear reason to rule out 10^-20 (p. 23).

• Cubic growth model, Table 1 (pp. 23–25).

  • EV(L) > EV(N) if and only if r < about 0.000135 per year.
  • EV(L) in valons: r = 10^-3, 3.44 × 10^-4; r = 10^-4, 3.32; r = 10^-5, 3.32 × 10^4; r = 10^-6, 4.74 × 10^7; r = 10^-8, 4.39 × 10^12; r = 10^-10, 4.37 × 10^20. With r = 0, about 10
  • At r = 10^-5, 99% of EV(L) is realized by about 1 million years.
  • Starting settlement 500 years later moves the threshold only from 0.000135 to 0.000133 (fn. 28).

• Steady state model, Table 2 (pp. 26–27). With t_f = 5 × 10^8 years and no settlement, L wins only if r < about 0.000000012 (1.2 × 10^-8) per year. Even r = 0 gives EV(L) = 6 V. Raising p to 10^-10 and Earth’s value a hundredfold lifts the threshold to r < about 0.006.

• Adding uncertainty, Table 3 (pp. 27–30). Lower bounds on credence: 10^-3 in the cubic growth model; 10^-3 in r ≤ 10^-6 (given it); 10^-2 in s ≥ 0.8c; 10^-6 in Dyson-sphere values of v_s. The rest goes to the steady state model with r = 10^-3. Minimum EV(L):

  • cubic growth alone, about 1.23 × 10^-5 V; with r, 47.4 V; with s, 1.38 × 10^-5 V; with v_s, 1.11 × 10^3 V;
  • with r and s, 48.2 V; r and v_s, 1.58 × 10^11 V; s and v_s, 6.76 × 10^3 V; all three, 1.62 × 10^11 V.

  The cubic model plus any two of these bounds puts L above N by at least an order of magnitude. Uncertainty “will tend to strengthen rather than weaken the case for longtermism” (p. 30), because a little credence in enormous upside goes a long way.

• Fanaticism (pp. 31–32). Tarsney measures how Pascalian a choice is by how much of each prospect’s tails must be cut away before the best option changes. On some views, the component of EV(L) that beats N “depends on a conjunction of improbable assumptions with joint probability on the order of (say) 10^-18 or less” (p. 32). On more optimistic views it might be 10^-8 or more.

• Conclusions (pp. 33–35).

  • The case for longtermism survives this challenge under expectational utilitarianism.
  • On some plausible worldviews, it does so only through Pascalian probabilities.
  • It leans on interstellar settlement.
  • Persistence favours existential-risk work over reforming institutions or values, whose effects are easier to undo or would come about later anyway.
  • “We should be ‘longtermists’ on the scale of thousands or millions or years, rather than billions or trillions of years” (p. 34).
  • Overall the challenge is “serious” and probably has “significant practical implications”, “but is not fatal” (p. 34).

Claims

id	claim	strength	support
C1	A constant risk that an intervention’s effect is nullified acts as an exponential discount that eventually beats polynomial growth in value, so the size of the future does not by itself make longtermism true	strong: follows from the model’s form	§4.1
C2	At conservative point estimates, existential-risk reduction beats a top global-health charity only if far-future nullifying events are rarer than about 1.35 per 10,000 years (interstellar) or 1.2 per 100 million years (Earth-bound)	moderate: computed, but every input is a guess chosen to be pessimistic	Tables 1–2
C3	Small credences in optimistic scenarios make the longtermist option’s expected value exceed the benchmark’s by orders of magnitude	moderate: depends on the stated credence floors, which are subjective	Table 3
C4	On some plausible worldviews the longtermist advantage rests on probabilities of order 10^-18 or less, so the case is open to the charge of fanaticism	weak to moderate: Tarsney calls these numbers “much less robust” than the expected values	§6.2
C5	If longtermism is false for us, the reason is epistemic, not a lack of power to affect the future	weak: an argument from what a chance-omniscient agent could do	§3
C6	The value of a successful longtermist intervention may be concentrated within thousands or millions of years	moderate within the model; sensitive to r	Table 1, §7

Concepts

• longtermism (p. 5): in most of the most important choices, what one ought to do is mainly determined by possible effects on the far future. Close to Greaves and MacAskill’s deontic strong longtermism.
• persistent-difference strategy: bringing about a better state in the hope that it lasts where it otherwise would not.
• control challenge versus epistemic challenge (pp. 7–8): we cannot improve the far future even with perfect knowledge of chances, versus we cannot tell which actions would.
• exogenous nullifying event (ENE): a far-future event after which the world’s state no longer depends on the intervention.
• valon: 10,000 QALYs, the expected value of the benchmark.
• expectational utilitarianism: precise probabilism plus total welfarism plus expected value maximization.
• Pascalian-ness: 1 − 2μ*, where μ* is the smallest share of each tail that must be ignored to change the best option.

Connections

• Greaves and MacAskill (LIT-1684). Tarsney’s target is their thesis: his statement of longtermism “closely resembles” their deontic strong longtermism (fn. 8), cited from the 2021 working paper. He grants them the normative frame and contests only the empirical inputs.
• Bostrom (LIT-1739, LIT-1773). Tarsney names Bostrom’s “naive back-of-the-envelope estimates” (p. 8) as what persistence worries cut down. Bostrom 2003 also supplies the Dyson-sphere value of 10^25 lives per star. One of the four credence floors that rescue longtermism is credence in exactly that scenario.
• Thorstad (LIT-1688). Tarsney assumes a constant rate of nullifying events and cites Thorstad’s 2022 working paper as “a case against” the time of perils (fn. 22). Thorstad’s 2023 paper works in per-century risk from the present, where Tarsney starts the clock 1000 years out. Inference: Tarsney’s p, the change in the chance of surviving to 3022, is where Thorstad’s problem would land. If risk stays high for the next ten centuries, a cut in this century’s risk changes survival to 3022 very little.
• Ord (LIT-1678) is cited on existential risk and the time of perils, and for a model in the same spirit (Appendix E).
• Discounting. Uncertainty about r makes the effective discount hyperbolic in the long run (fn. 37). That is a discount for risk, not for time as such, and the record’s QUESTION-299 cites Ramsey’s refusal to discount for time.

Bearing on the record

• QUESTION-299. The paper bears on its first ground, the impersonal good, and on the “can longtermism guide action” line the question already names. It does not refute longtermism; Tarsney concludes the challenge is “not fatal”. What it shows is what longtermism’s action-guidance rests on: an unknown long-run hazard rate, interstellar settlement, and, on some views, credences of order 10
• The owner’s thesis (CLAIM-491). The paper says nothing about motivation. Its bearing is indirect, and these are inferences.
  • The question calls concern for far-future strangers the thesis’s hardest case. That case is pressing only if such concern is owed and acted on. If the impersonal case for it rests on Pascalian probabilities, the normative pressure behind the hardest case is weaker, and more of what the living actually owe falls in the nearer future. Concern for that future (one’s descendants, one’s community, humanity going on) is the kind a self extended in time or across people can explain.
  • But Tarsney’s “nearer” is still thousands to millions of years. Even where his model favours a short horizon, it lies far past any self the thesis could reasonably name. So the paper narrows the hardest case; it does not remove it.
  • It offers a rival explanation of why concern fades with time. Rational concern declines with the hazard that an effect is undone, with no reference to the self. A test of whether concern for the future tracks felt continuity with it (Hershfield, LIT-1702; the test QUESTION-299 and QUESTION-301 propose) must separate that from expected-hazard discounting.
• The triviality objection (CLAIM-492). Not addressed directly. The confound above is a point about how to run its test across generations.
• No dated note on CLAIM-491. The paper does not touch motivation. Its bearing goes into QUESTION-299 (section 3 below).
• No THEORY filed. The model is a contribution to a normative debate the record holds as a question, not an account of what is true.
• No ML instruction.

Limitations

• One intervention, one kind of strategy. Existential-risk reduction is chosen because it is easy to quantify. Speed-ups and other non-persistence strategies are not modelled.
• The inputs are guesses, chosen to be conservative, and Tarsney admits s = 0.1c is not (fn. 27). The credence floors in Table 3 are “unavoidably subjective” (p. 29).
• Only persistence is tested. Tractability (p) is a lower bound with “only a minimal argument”, and importance is set aside. Tarsney thinks the stronger worry may be that a civilization’s goodness is not persistent even if its existence is (p. 35).
• Only precise probabilism. Imprecise probabilities, unawareness and bounded rationality are left open, and Tarsney suspects longtermism fares worse in them (pp. 35–36).
• Not compared with the Synthese version.
• Typos and slips.
  • “suitbable” (p. 9); “integrand of of EV(L)” (p. 25); “these bounds quite conservative” (p. 29); “we will chose” (fn. 35, p. 29); “peristent” and “millions or years” (p. 34); “rather than those objectives” for “rather that” (p. 35).
  • Table 2 gives the 99th percentile at r = 10^-5 as “460,517 million years”; the series (4605, 46,052, then 4.6 million years) shows it should be about 460,517 years (p. 27).
  • “p = 2 × 10^14” for 2 × 10^-14 (p. 35).
  • Footnote 41 calls the sure payoff “3000 QALYS (1 valon)” and converts with it, though a valon was defined as 10,000 QALYs (pp. 12, 33).

Open questions

• How does the case change if r falls over time, as the time of perils hypothesis says? Tarsney’s appendix lists time-dependence as the second most important simplification, and expects it to favour longtermism. Thorstad’s paper (LIT-1688) asks how fast and how far it would have to fall.
• Does lay concern for the far future fall off with time the way an expected-hazard discount would, or the way felt self-continuity does? The two predict different curves, and no work the record holds compares them.