Thanks for commenting!

I've tried to spell out my position more clearly, so we can see if/where we disagree. I think:

• Most discussion of longtermism, on the level of generality/abstraction of "is longtermism true?", "does X moral viewpoint support longtermism?", "should longtermists care about cause area X?" is not particularly useful, and is currently oversupplied.
• Similarly, discussions on the level of abstraction of "acausal trade is a thing longtermists should think about" are rarely useful.
• I agree that concrete discussions aimed at "should we take action on X" are fairly useful. I'm a bit worried that anchoring too hard on longtermism lends itself to discussing philosophy, and especially discussing philosophy on the level of "what axiological claims are true", which I think is an unproductive frame. (And even if you're very interested in the philosophical "meat" of longtermism, I claim all the action is in "ok but how much should this affect our actions, and which actions?", which is mostly a question about the world and our epistemics, not about ethics.)
• "though I'd be like 50x more excited about Forethought + Redwood running a similar competition on things they think are important that are still very philosophy-ish/high level." —this is helpful to know! I would not be excited about this, so we disagree at least here :)
• "The track record of talking about longtermism seems very strong" —yeah, agree longtermism has had motivational force for many people, and also does strengthen the case for lots of e.g. AI safety work. I don't know how much weight to put on this; it seems kinda plausible to me that talking about longtermism might've alienated a bunch of less philosophy-inclined but still hardcore, kickass people who would've done useful altruistic work on AIS, etc. (Tbc, that's not my mainline guess; I just think it's more like 10-40% likely than e.g. 1-4%.)
• "I feel like this post is more about "is convincing people to be longermists important" or should we just care about x-risk/AI/bio/etc." This is fair! I think it's ~both, and also, I wrote it poorly. (Writing from being grumpy about the essay contest was probably a poor frame.) I am also trying to make a (hotter?) claim about how useful thinking in these abstract frames is, as well as a point on (for want of a better word) PR/reputation/messaging. (And I'm more interested in the first point.)

Thanks for commenting!

> So actually maybe I agree that for now lots of longtermists should focus on x-risks while there are still lots of relatively cheap wins, but I expect this to be a pretty short-lived thing (maybe a few decades?) and that after that longtermism will have a more distinct set of recommendations.

Yeah, this seems reasonable to me. Max Nadeau also pointed out something similar to me (longtermism is clearly not a crux for supporting GCR work, but also clearly important for how e.g. OP relatively prioritises x risk reduction work vs mere GCR reduction work). I should have been clearer that I agree "not necessary for xrisk" doesn't mean "not relevant", and I'm more intending to answer "no" to your (2) than "no" to your (1). 

(We might still relatively disagree over your (1) and what your (2) should entail —for example, I'd guess I'm a bit more worried about predicting the effects of our actions than you, and more pessimistic about "general abstract thinking from a longtermist POV" than you are.)

Whether longtermism is a crux will depend on what we mean by 'long'

Yep, I was being imprecise. I think the most plausible (and actually believed-in) alternative to longtermism isn't "no care at all for future people", but "some >0 discount rate", and I think xrisk reduction will tend to look good under small >0 discount rates.

I do also agree that there are some combinations of social discount rate and cost-effectiveness of longtermism, such that xrisk reduction isn't competitive with other ways of saving lives. I don't yet think this is clearly the case, even given the numbers in your paper — afaik the amount of existential risk reduction you predicted was pretty vibes-based, so I don't really take the cost-effectiveness calculation it produces seriously. (And I  haven't done the math myself on discount rates and cost-effectiveness.) 

Even if xrisk reduction doesn't look competitive with e.g. donating to AMF, I think it would be pretty reasonable for some people to spend more time thinking about it to figure out if they could identify more cost-effective interventions. (And especially if they seemed like poor fits for E2G or direct work.)

Glad you shared this!

Expanding a bit on a comment I left on the google doc version of this: I broadly agree with your conclusion (longtermist ideas are harder to find now than in ~2017), but I don't think this essay collection was a significant update towards that conclusion. As you mention as a hypothesis, my guess is that these essay collections mostly exist to legitimise discussing longtermism as part of serious academic research, rather than to disseminate important, plausible, and novel arguments. Coming up with an important, plausible, and novel argument which also meets the standards of academic publishing seems much harder than just making some publishable argument, so I didn't really change my views on whether longtermist ideas are getting harder to find because of this collection's relative lack of them. (With all the caveats you mentioned above, plus: I enjoyed many of the reprints, and think lots of incrementalist research can be very valuable —it's just not the topic you're discussing.)  

I'm not sure how much we disagree, but I wanted to comment anyway, in case other people disagree with me and change my mind!

Relatedly, I think what I'll call the "fundamental ideas" — of longtermism, AI existential risk, etc — are mildly overrated relative to further arguments about the state of the world right now, which make these action-guiding. For example, I think longtermism is a useful label to attach to a moral view, but you need further claims about reasons not to worry about cluelessness in at least some cases, and also potentially some claims about hinginess, for it to be very action-relevant. A second example: the "second species" worry about AIXR is very obvious, and only relevant given that we're in a world where we're plausibly close to developing TAI soon and, imo, because current AI development is weird and poorly understood; evidence from the real world is a potential defeater for this analogy.

I think you're using "longtermist ideas" to also point at this category of work (fleshing out/adding the additional necessary arguments to big abstract ideas), but I do think there's a common interpretation where "we need more longtermist ideas" translates to "we need more philosophy types to sit around and think at very high levels of abstraction". Relative to this, I'm more into work that gets into the weeds a bit more.

(Meta: I really liked this more personal, idiosyncratic kind of "apply here" post!)

Could you say a bit more about the "set of worldviews & perspectives" represented on Forethought, and in which ways you'd like it to be broader?

Cool, thanks for sharing!
 

I currently use Timing.app, and have been recommending it to people. Is donethat different in any ways? (TBC, "it has all the same features but also supports an E2G effort" would be sufficient reason for me to consider switching).

Mostly the latter two, yeah

Yes, at least initially. (Though fwiw my takeaway from that was more like, "it's interesting that these people wanted to direct their energy towards AI safety community building and not EA CB; also, yay for EA for spreading lots of good ideas and promoting useful ways of looking at problems". This was in 2022, where I think almost everyone who thought about AI safety heard about it via EA/rationalism.)

Interesting post, thanks for sharing. Some rambly thoughts:^[1]

• I'm sympathetic to the claim that work on digital minds, AI character, macrostrategy, etc is of similar importance to AI safety/AI governance work. However, I think they seem much harder to work on — the fields are so nascent and the feedback loops and mentorship even scarcer than in AIG/AIS, that it seems much easier to have zero or negative impact by shaping their early direction poorly.
• I wouldn't want marginal talent working on these areas for this reason. It's plausible that people who are unusually suited to this kind of abstract low-feedback high-confusion work, and generally sharp and wise, should consider it. But those people are also well-suited to high leverage AIG/AIS work, and I'm uncertain whether I'd trade a wise, thoughtful person working on AIS/AIG for one thinking about e.g. AI character.
• (We might have a similar bottom line: I think the approach of "bear this in mind as an area you could pivot to in ~3-4y, if better opportunities emerge" seems reasonable.)
• Relatedly, I think EAs tend to overrate interesting speculative philosophy-flavoured thinking, because it's very fun to the kind of person who tends to get into EA. (I'm this kind of person too :) ). When I try to consciously correct for this, I'm less sure that the neglected cause areas you mention seem as important.
• I'm worried about motivated reasoning when EAs think about the role of EA going forwards. (And I don't think we should care about EA qua EA, just EA insofar as it's one of the best ways to make good happen.) So reason #2 you mention, which felt more like going "hmm EA is in trouble, what can we do?" rather than reasoning from "how do we make good happen?" wasn't super compelling to me.
• That being said, If it's cheap to do so, more EA-flavoured writing on the Forum seems great! The EAF has been pretty stale. I was brainstorming about this earlier —initially I was worried about the chilling effect of writing so much in public (commenting on the EAF is way higher effort for me than on google docs, for example), but I think some cool new ideas can and probably should be shared more. I like Redwood's blog a lot partly for this reason.
• In my experience at university, in my final 2 years the AI safety group was just way more exciting and serious and intellectually alive than the EA group — this is caricatured, but one way of describing it would be that (at extremes) the AI safety group selected for actually taking ideas seriously and wanting to do things, and the EA group correspondingly selected for wanting to pontificate about ideas and not get your hands dirty. I think EA groups engaging with more AGI preparedness-type topics could help make them exciting and alive again, but it would be important imo to avoid reinforcing the idea that EA groups are for sitting round and talking about ideas, not for taking them seriously. (I'm finding this hard to verbalise precisely —I think the rough gloss is "I'm worried about these topics having more of a vibe of 'interesting intellectual pastime', and if EA groups tend towards that vibe anyway, making discussing them feel ambitious and engaging and 'doing stuff about ideas'-y sounds hard". 

1. ^{^}

   I would have liked to make this more coherent and focused, but that was enough time/effort that realistically I just wouldn't have done it, and I figured a rambly comment was better than no comment.

See also https://sb53.info/ for a FAQ and annotated copy of the bill