Hide table of contents

A Response to Owen Cotton-Barratt and Rose Hadshar's "What Would a Longtermist Society Look Like?" (Chapter 19 of Essays on Longtermism)

I. Introduction

In their contribution to Oxford University Press's Essays on Longtermism, Owen Cotton-Barratt and Rose Hadshar ask what society would look like if longtermist perspectives shaped whole institutions rather than just individual choices at the margin.[^1] This is an important question that has received surprisingly little systematic attention.

Their central reassurance: even the most extreme longtermist society would invest substantial resources in current people's welfare because instrumental reasons mandate it. People whose basic needs are unmet cannot do complex work. People who are unhappy are less productive. The authors offer this analogy: just as profit-maximizing companies provide "high-quality office space, food, and entertainment for their staff" to boost shareholder returns, longtermist states would provide for current citizens to maximize future benefit.[^2]

Here's the miss: they treat existence like a budget across time—present people existing now, future people existing later, with the question being how to allocate resources between these temporal slices. But existence is not quantity distributed across time—it is a continuous adaptive process operating in networks. There is no "short-term" existence separate from "long-term" existence. There is only a loop—sense, learn, decide, act, repair—or there isn't.

The question isn't how to sacrifice present for future, but whether the structures that enable existence to persist—the adaptive loop itself—are being maintained or destroyed. The implementation Cotton-Barratt and Hadshar describe mistakes optimization toward imagined future targets for maintenance of the adaptive process. In doing so, it destroys the very structures that enable anything to persist.

To understand why, we must examine both the incentive gradient in systems with unverifiable goals, and the nature of existence as continuous network adaptation.

II. The Incentive Gradient

Cotton-Barratt and Hadshar distinguish partial from strict longtermism, acknowledge "imperfections," and note legitimacy constraints. But nuance doesn't stop a structural drift: systems with certain properties tend toward their worst configurations, regardless of initial intentions.

The gradient emerges when a system has:

  • Goals that cannot be verified for decades or centuries
  • Officials who must use measurable proxies for success
  • Power holders claiming to represent voiceless beneficiaries
  • No feedback loop from actual beneficiaries

Six mutually-reinforcing dynamics create the drift:

  • Measurability bias: Optimize what's measurable (productivity, compliance) over what matters (adaptive capacity, error correction)
  • Goodhart drift: Once proxies become targets, they diverge from goals—optimizing the metric destroys what it measured
  • Selection effects: The system promotes true believers in current orthodoxy while sidelining skeptics
  • Legitimacy ratchet: Early restrictions on dissent normalize later, stronger restrictions
  • Exit barriers: Each brain drain incident justifies more controls
  • Institutional capture: Power concentrates with those who define "success" in unverifiable domains

This isn't deterministic—call it a gravity well. You can resist, but the default trajectory is drift toward configurations where these dynamics reinforce each other. We've seen this drift: post-crisis surveillance that outlives the crisis; centrally set quotas that beat the numbers but miss reality.

When long-horizon efforts do succeed—the Montreal Protocol, for instance—it's because they generate near-term verifiable feedback, reversible steps, and co-benefits. These are precisely the features missing in strict, state-imposed longtermism.

A strict longtermist state faces this gradient by design: goals unverifiable for generations, proxies required, beneficiaries voiceless, power centralized claiming authority to represent trillions who cannot check those claims.

But the gradient is not the deepest problem. The deepest problem is the conceptual error underlying the entire framework: mistaking existence for a quantity distributed across time, when existence is actually the adaptive process itself.

III. Existence as Loop Maintenance

We don't keep the future by thinking about it harder. We keep it the way we keep a city running through the night.

Think of what has to hold, quietly, for an ordinary Tuesday to work: the lights stay on; clean water comes out of a tap; your phone finds a signal; the clinic has antibiotics; the train arrives roughly when it should. None of these is produced by a single heroic actor. They're held up by relationships—power stations balancing loads, engineers reading dials, rules that keep us from cutting corners, people repairing what breaks before anyone notices.

That is the shape of our survival. It isn't a jackpot we win later; it's a rhythm we keep now—sense, learn, decide, act, repair—over and over, across many hands. When that rhythm falters, there is no "long term" waiting patiently for us to catch up.

Only then is the right word clear: this is a network. Not a social-media buzzword, but the plain fact that existence is a shared maintenance loop among many different parts coordinating under rules, information flows, and repair—households, labs, farms, courts, grids, schools—each doing its own work, each coupled to the rest. You could call this autonomous interdependence: we self-regulate, and we stay reliably connected by reciprocal constraints—transparency, coordination, repair.

The loop has five components:

  • Learn: Gather evidence, update models, explore scenarios
  • Decide: Choose based on evidence, commit resources, explain choices
  • Act: Deliver on commitments
  • Coordinate: Share information, align diverse units, enforce mutual constraints
  • Repair: Fix breakage—technical and relational

The loop requires network structures:

  • Diversity: Multiple approaches maintained simultaneously (enables exploration)
  • Autonomy: Units capable of self-regulation (enables genuine diversity, not cosmetic variation)
  • Interdependence: Reliable coupling through coordination (enables scale without fragmentation)
  • Error correction: Capacity to detect failures and shift toward alternatives (enables adaptation)

Note: Interdependence has capacity (bandwidth to coordinate) and quality (contestability, transparency, and repair). Coercion can raise capacity while destroying quality.

Remove any element and the loop degrades: no diversity means single failure mode; no autonomy means apparent diversity collapses to centralized control; no interdependence means fragmentation; no error correction means lock-in to wrong models.

The profound insight: There is no "longtermism" separate from this loop. You don't "sacrifice present for future"—you either maintain the loop that enables continued existence, or you destroy it. The capacity to continue adapting IS what persists. The search for how to keep the Tuesday-morning services running IS the keeping of them.

Cotton-Barratt and Hadshar's framework makes a category error: treating "serving the far future" as optimization toward distant targets (reduce AI risk, prevent pandemics, produce longtermist goods) rather than maintenance of the adaptive loop. This error becomes self-defeating when combined with the incentive gradient.

IV. How the Gradient Destroys the Loop

Cotton-Barratt and Hadshar are explicit: current welfare matters "because unhappy people are less productive than happy people."[^3] When current needs don't enhance productivity toward longtermist goals, the state might choose not to fund them—they note it's "conceivable that a longtermist state might choose not to fund palliative care, or choose not to support lengthy retirement..[^4]

In optimizing productivity toward unverifiable far-future targets, the gradient systematically destroys the adaptive loop. Consider what happened in Soviet central planning—a system that began with genuine concern for future generations but faced the same structural dynamics.

Soviet planners couldn't verify their goals for decades (industrialization targets, collectivization outcomes). They used measurable proxies: tons of steel, hectares planted, quotas met. The gradient operated:

Measurability bias kicked in: Factories optimized for tonnage metrics. Heavy, useless products counted the same as valuable ones. Farms reported planted hectares regardless of soil quality.

Goodhart drift followed: Production metrics diverged completely from genuine productivity. The economy appeared to grow while actual welfare stagnated.

Selection effects compounded: Officials who questioned the metrics didn't advance. Those who met quotas through creative accounting did.

The legitimacy ratchet tightened: Early questioning of targets was dismissed as counter-revolutionary. Dissent became risky, then dangerous, then impossible.

Exit barriers rose: Emigration became treason. Internal movement was restricted. Alternative economic organization was criminalized.

Institutional capture completed: Those who controlled the definition of "progress toward communist future" had no accountability to actual beneficiaries.

The loop degraded: Engineers stopped reporting true problems (learning failed). Farms stopped adapting to local conditions (adaptation failed). Alternative models were suppressed (diversity failed). Coordination became coercion (interdependence quality degraded). Error correction became impossible (dissent was suppressed).

Proxies kept rising—right up to the shock that showed learning, dissent, and reversibility were unfunded. By the time feedback arrived—empty store shelves, technological stagnation—the capacity to sense problems, learn from evidence, adapt approaches, and coordinate solutions was gone.

This is precisely what the longtermist state faces. It optimizes for productivity toward targets verifiable only centuries later (AI risk reduction, longtermist goods production). The gradient operates through the same mechanisms. Learning becomes confirmation of current priorities. Adaptation becomes execution. Simulation collapses to projection of current assumptions. Evidence-based decisions become authority-based (what the state defines as "longtermist goods"). Coordination transforms from chosen interdependence to coercion. Error correction fails as alternatives are suppressed.

The authors acknowledge the risk: "A state that dogmatically adopted a longtermist perspective might indoctrinate its citizenry, in a way that prevented them from adopting better perspectives."[^5] They frame this as an "imperfection." But the gradient reveals it as structural necessity. Officials need legitimacy through citizen buy-in around current priorities. The system selects for convergence. When unforeseen challenges arrive, there's no capacity to adapt—the loop has been optimized away.

The conceptual error: You cannot serve "the far future" by destroying present adaptive capacity, because the far future is not separate people later; it is the next iteration of the same adaptive loop. Destroy the Tuesday-morning maintenance network now—the sensing, learning, adapting, coordinating, repairing—and there is no network later to maintain anything. The gradient optimizes for distant targets while destroying loop maintenance. Without the loop, nothing exists.

V. What Loop Maintenance Requires

If genuine concern for persistence means maintaining the adaptive loop rather than optimizing toward distant targets, what does that require?

Loop-maintaining projects share patterns:

  • Generate feedback within learning cycles (climate action prevents disasters now, provides immediate evidence)
  • Explore multiple approaches (medical research tries many paths, maintains genuine alternatives)
  • Build autonomous capacity (education enables independent judgment, not just compliance)
  • Strengthen coordination quality (improve cooperation through reciprocity, not coercion)

Loop-destroying projects share patterns:

  • No present feedback (optimize for targets verifiable only centuries later)
  • Suppress alternatives (channel all resources to current priorities)
  • Require compliance (instrumentalize autonomy for productivity)
  • Degrade coordination quality (exit barriers rise, surveillance replaces trust)

The test isn't present benefit versus future benefit—it's whether the project maintains or degrades the adaptive loop. When you repair infrastructure, you generate feedback (did it work?), maintain expertise (learning continues), explore different approaches (diversity preserved), and keep the Tuesday-morning loop running. When you commit irreversibly to projects optimized for one far-future scenario, you sacrifice feedback, suppress alternatives, and bet everything on current models being correct.

Loop-maintenance principles (illustrative):

Institutions that resist the gradient's erosion of the loop might include approaches such as:

Feedback within years: Projects generate evidence within timeframes that enable learning—years to decades, not centuries. Verification happens through current results, not distant promises.

Maintained alternatives: Resources flow to multiple approaches, including those that challenge current orthodoxy. Genuine diversity requires autonomous units capable of self-regulation according to different assessments.

Reversibility by default: No irreversible commitments that prevent future iterations from choosing differently when evidence accumulates. Preserve adaptive capacity over optimization for current best guesses.

Resourced dissent (structurally independent): Critics receive funding from the budgets they criticize. Error correction requires institutional support, not just tolerance.

Exit viability (low switching costs): Switching costs remain low. Interdependence must be chosen coordination among autonomous units, not forced dependency.

Periodic re-evaluation: Automatic review at set intervals. No indefinite lock-in to approaches that may have become obsolete.

Polycentric approval: Major commitments require sign-off from multiple bodies with conflicting incentives, preventing institutional capture.

If a proposal raises output on a chosen target while failing two or more of these, it's not stewardship; it's loop erosion.

None of this indicts partial longtermism embedded within loop-maintaining institutions; it warns against strict, state-imposed longtermism that optimizes proxies while eroding the loop.

These aren't optimal policies but recognition that loop maintenance requires active resistance to the gradient. The default trajectory is drift toward target optimization that destroys adaptive capacity. Preserving the loop—the Tuesday-morning maintenance across power grids, water systems, knowledge networks, coordination mechanisms—requires structures that actively prevent this drift, continuously, as part of the iteration itself.

VI. Conclusion

Cotton-Barratt and Hadshar ask what longtermist societies would look like. Their answer reveals a fundamental misunderstanding of what enables existence to persist.

They treat existence as quantity distributed across time—present people now, future people later—and frame the question as resource allocation between temporal slices. But existence is process: the continuous adaptive loop operating in networks. The Tuesday-morning rhythm of sensing, learning, adapting, coordinating, repairing. This loop either maintains itself or collapses. There is no "present existence" separate from "future existence"—only ongoing maintenance of adaptive capacity.

Their framework optimizes for productivity toward far-future targets. But this mistakes target optimization for loop maintenance. The incentive gradient then destroys each component: learning degrades to confirmation, adaptation becomes execution, simulation collapses to projection, evidence-based decisions become authority-based, coordination transforms to coercion, error correction fails, iteration stops. We saw this pattern in Soviet planning. The gradient operates through the same mechanisms in any system with unverifiable goals, necessary proxies, voiceless beneficiaries, and power asymmetries.

Proxies keep rising—high productivity metrics, measurable progress toward longtermist goals—right up until unforeseen challenges expose that the adaptive loop is gone. The capacity to sense honestly, learn from evidence, adapt approaches, and coordinate solutions has been systematically removed. Not as an "imperfection" but as the necessary consequence of optimizing for unverifiable distant targets while the gradient operates.

The authors' careful distinctions between partial and strict longtermism prove unstable. Without active institutional resistance, the default is drift toward configurations where loop components degrade while productivity toward current targets increases. The nuance fails because it doesn't address the conceptual error: treating existence as quantity-in-time rather than recognizing it as the adaptive loop itself—the ongoing maintenance that keeps the lights on, the water flowing, the coordination functioning.

But this clarifies what persistence requires. There is no "longtermism" separate from maintaining the structures that enable the adaptive loop to continue. The future we claim to serve is not separate people later—it is the next iteration of the same maintenance network. You cannot serve it by destroying present adaptive capacity. You serve it by maintaining the loop: preserving diversity, autonomy, interdependence quality, and error correction as functional requirements for the process that constitutes existence.

The test is simple: Does an approach maintain or destroy the adaptive loop? Cotton-Barratt and Hadshar's longtermist societies destroy the loop while claiming to serve the far future. They optimize for productivity toward unverifiable targets while the gradient eliminates the capacity to sense, learn, adapt, correct errors, coordinate through reciprocity, and iterate. They sacrifice the Tuesday-morning maintenance network—the quiet, continuous work that keeps existence running—for optimization toward distant destinations. They build brittleness while pursuing security.

Those who care about enabling existence to continue should recognize this as a category error with catastrophic consequences. Existence is not a destination to optimize toward but a process to maintain. The far future is not separate from the present loop—it is the continued operation of the same adaptive maintenance. Anything that destroys present adaptive capacity destroys the capacity for anything to exist later.

True concern for persistence means maintaining the prerequisites for the adaptive loop: autonomous interdependence, diversity, error correction, continuous iteration.

The search is the finding.

The loop is the persistence.

The maintenance is the goal.

 

[^1]: Cotton-Barratt, O., & Hadshar, R. (2025). "What Would a Longtermist Society Look Like?" In Greaves, H., Barrett, J., & Thorstad, D. (Eds.), Essays on Longtermism. Oxford University Press, p. 334.

[^2]: Cotton-Barratt & Hadshar (2025), p. 339.

[^3]: Cotton-Barratt & Hadshar (2025), p. 339.

[^4]: Cotton-Barratt & Hadshar (2025), p. 341.

[^5]: Cotton-Barratt & Hadshar (2025), p. 341.

1

0
0

Reactions

0
0

More posts like this

Comments5
Sorted by Click to highlight new comments since:

Afterword: A Note of Appreciation and Reflection

I want to begin by thanking Owen Cotton-Barratt and Rose Hadshar for their thoughtful and important chapter. Their willingness to examine what longtermist societies might actually look like—moving beyond marginal analysis to whole-system thinking—opens necessary terrain. This essay is offered in that same spirit of serious engagement, not as refutation but as extension.

My response is constructive, though it may read as fundamental disagreement. I believe we share a deep concern: how do we enable human flourishing to persist? Where we differ, I think, is in our conceptual starting point, and this difference ramifies through everything that follows.

Two frameworks for thinking about persistence:

Cotton-Barratt and Hadshar work from what might be called a projection framework: existence is distributed across time, and the question is how to allocate resources between temporal slices—present people now, future people later. Within this framework, their insight is important: even extreme longtermism requires substantial investment in present welfare for instrumental reasons. People whose basic needs aren't met cannot do complex work.

My essay works from what might be called a process framework: existence is not quantity distributed across time but a continuous adaptive process. There is no "present existence" separate from "future existence"—only ongoing maintenance of adaptive capacity. The question becomes not how to optimize for distant projected outcomes, but whether we're maintaining the structures that enable any outcomes at all.

Why this difference matters:

These aren't just semantic alternatives. They lead to different institutional designs, different understandings of risk, and different responses to uncertainty.

Cotton-Barratt and Hadshar recognize that instrumental reasons require present welfare. I'm suggesting something stronger: that the process of maintaining present adaptive capacity—the sense-learn-adapt-coordinate-repair loop—isn't instrumental to distant goals but constitutive of what persistence means. The Tuesday-morning maintenance network isn't preparation for a future we're aiming toward; it is the future, continuously instantiated.

This leads to seeing different risks. The incentive gradient I describe—the structural drift toward configurations that optimize measurable proxies while degrading adaptive capacity—isn't visible from a projection framework because it looks like progress on longtermist goals right up until the system can no longer adapt to surprises.

What I hope this contributes:

Cotton-Barratt and Hadshar's analysis helps us think carefully about constraints and resource allocation. Their distinction between partial and strict longtermism, their attention to legitimacy concerns, their recognition of instrumental value—all of this is valuable.

My hope is that the process framework adds something complementary: a way to think about systemic resilience, about what makes persistence possible in the face of deep uncertainty, about why maintaining diversity, autonomy, error correction, and genuine interdependence might not be constraints on longtermism but prerequisites for anything to persist at all.

An invitation:

I'd be genuinely curious to hear how Cotton-Barratt and Hadshar see this difference. Is it a meaningful distinction? Are these frameworks reconcilable at different scales of analysis? When would we know which better serves long-term flourishing?

Perhaps the most important test is this: when unforeseen challenges arrive—as they inevitably will—which approach has preserved the adaptive capacity to sense them early, learn from evidence, coordinate responses, and iterate toward solutions?

I suspect we all want the same thing: a future where human flourishing continues. The question is how we think about—and design for—that persistence. I offer this essay as one contribution to that ongoing conversation.

 

A note on method: For transparency, I used Claude Sonnet 4.5 and ChatGPT-5 as thinking partners and writing tools for this essay—for structure, clarity, and articulation. The core framework, however, emerges from my hands-on work with dynamic network modeling of infectious diseases, and my training across biology, economics, and philosophy. The loop-maintenance perspective reflects years of thinking and exploration and was sparked by conceptual reflection on oak trees. The ideas are mine; the AI helped me say them clearly.

Thanks AJ!

My impression is that although your essay frames this as a deep disagreement, in fact you're reacting to something that we're not saying. I basically agree with the heart of the content here -- that there are serious failure modes to be scared of if attempting to orient to the long term, and that something like loop-preservation is (along with the various more prosaic welfare goods we discussed) essential for the health of even a strict longtermist society.

However, I think that what we wrote may have been compatible with the view that you have such a negative reaction to, and at minimum I wish that we'd spent some more words exploring this kind of dynamic. So I appreciate your response.

Thanks for the generous response. You write that we "may have been compatible" and I'm "reacting to something you're not saying."

Here's my concern: I've come to recognize that reality operates as a dynamic network—nodes (people, institutions) whose capacity is constituted by the relationships among them. This isn't just a modeling choice; it's how cities function, how pandemics spread, how states maintain capacity. You don't work from this explicit recognition.

This creates an asymmetry. Once you see reality as a network, your Section 5 framework becomes incompatible with mine—not just incomplete, but incoherent. You explicitly frame the state as separate from people, optimizing for longtermist goals while managing preferences as constraints. But from the network perspective, this separation doesn't exist—the state's capacity just IS those relationships. You can't optimize one while managing the other.

Let me try to say this more directly: I've come to understand my own intelligence as existing not AT my neurons, but BETWEEN them—as a pattern of activation across connections. I am the edge, not the node. And I see society the same way: capacity isn't located IN institutions, it emerges FROM relationships. From this perspective, your Section 5 (state separate from people) isn't a simplification—it's treating edges as if they were nodes, which fundamentally misunderstands what state capacity is.

That's the asymmetry: your explicit framing (state separate from people) is incompatible with how I now understand reality. But if you haven't recognized the network structure, you'd just see my essay as "adding important considerations" rather than revealing a foundational incompatibility.

Does this help clarify where I'm coming from?

Thanks for your post AJ, and esp this comment which I found clarifying.

I'd be genuinely curious to hear how Cotton-Barratt and Hadshar see this difference. Is it a meaningful distinction? Are these frameworks reconcilable at different scales of analysis? When would we know which better serves long-term flourishing?

I've only skimmed your post, and haven't read what me and Owen wrote in several years, but my quick take is:

  • We're saying 'within a particular longtermist frame, it's notable that it's still rational to allocate resources to neartermist ends, for instrumental reasons'
    • I think you agree with this
    • Since writing that essay, I'm now more worried about AI making humans instrumentally obsolete, in a way that would weaken this dynamic a lot (I'm thinking of stuff like the intelligence curse). So I don't actually feel confident this is true any more.
  • I think you are saying 'but that is not a good frame, and in fact normatively we should care about some of those things intrinsically'
    • I agree, at least partially. I don't think we intended to endorse that particular longtermist frame - just wanted to make the argument that even if you have it, you should still care about neartermist stuff. (And actually, caring instrinsically about neartermist stuff is part of what motivated making the argument, iirc.)
    • I vibed with some of your writing on this, e.g. "The Tuesday-morning maintenance network isn't preparation for a future we're aiming toward; it is the future, continuously instantiated."
    • I'm not a straight out yes - I think Wednesday in a million years might matter much more than this Tuesday morning, and am pretty convinced of some aspects of longtermism. But I agree with you in putting intrinsic value on the present moment and people's experiences in it

So my guess is, you have a fundamental disagreement with some version of longtermism, but less disagreement with me than you thought.

Thank you for engaging, and especially for the intelligence curse point—that's exactly the structural issue I'm trying to get at.

You suggest I'm arguing "we should care about some of those things intrinsically." Let me use AGI as an example to show why I don't think this is about intrinsic value at all:

What would an AGI need to persist for a million years?

Not "what targets should it optimize for" but "what maintains the AGI itself across that timespan?"

I think the answer is: diversity (multiple approaches for unforeseen challenges), error correction (detecting when models fail), adaptive capacity (sensing and learning, not just executing), and substrate maintenance (keeping the infrastructure running).

An AGI optimizing toward distant targets while destroying these properties would be destroying its own substrate for persistence. The daily maintenance—power, sensors, error detection—isn't preparation for the target. It IS what persistence consists of.

I think the same logic applies to longtermist societies. The question would shift from "how to allocate resources between present and future" to "are we maintaining or destroying the adaptive loop properties that enable any future to exist?" That changes what institutions would need to do—the essay explores some specific examples of what this might look like.

Does the AGI example help clarify the reframe I'm proposing?

Curated and popular this week
Relevant opportunities