The defense in depth thesis is that you are best off investing some resources from your limited military budget in many different defenses (e.g. nuclear deterrence; intelligence gathering and early warning systems; an air force, navy and army; command and communication bunkers; diplomacy and allies) rather than specialising heavily in just one.

I'm not familiar with how this concept is used in the military, but in safety engineering I've never heard of it as a tradeoff between 'many layers, many holes' vs 'one layer, few holes'. The swiss cheese model is often meant to illustrate the fact that your barriers are often not 100% effective, so even if you think you have a great barrier, you should have more than one of it. From this perspective, the concept of having multiple barriers is straightforwardly good and doesn't imply justifying the use of weaker barriers. 

Agreed - I should've made it clearer in the title that I was referring specifically to the AI safety people in EA, i.e. this excludes other EAs not in AI safety, and also excludes other non-EAs in AI safety.

I would be interested to hear the counterpoints from those who have Disagree-voted on this post. 

Likewise!

Do you think the "very particular worldview" you describe is found equally among those working on technical AI safety and AI governance/policy? My impression is that policy inherently requires thinking through concrete pathways of how AGI would lead to actual harm as well as greater engagement with people outside of AI safety. 

I think they're quite prevalent regardless. While some people's roles indeed require them to analyze concrete pathways more than others, the foundation of their analysis is often implicitly built upon this worldview in the first place. The result is that their concrete pathways tend to be centred around some kind of misaligned AGI, just in much more detail. Conversely, someone with a very different worldview who does such an analysis might end up with concrete pathways centred around severe discrimination of marginalized groups. 

I have also noticed a split between the "superintelligence will kill us all" worldview (which you seem to be describing) and "regardless of whether superintelligence kills us all, AGI/TAI will be very disruptive and we need to manage those risks" (which seemed to be more along the lines of the Will MacAskill post you linked to - especially as he talks about directing people to causes other than technical safety or safety governance).

There are indeed many different "sub-worldviews", and I was kind of lumping them all under one big umbrella. To me, the most defining characteristic of this worldview is AI-centrism, and treating the impending AGI as an extremely big deal — not just like any other big deals we have seen before, but this will be unprecedented. Those within this overarching worldview would differ in terms of the details, e.g. will it kill everyone? or will it just lead to gradual disempowerment? are LLMs getting us to AGI? or is it some yet-to-be-discovered architecture? should we focus on getting to AGI safely? or start thinking more about the post-AGI world? I think many people move between these "sub-worldviews" as they see evidences that update their priors, but way fewer people move out of this overarching worldview entirely. 

(semi-commitment for accountability)
I'm considering writing more about how a big part of AI safety seem to be implicitly built upon an underlying worldview and we have rarely challenged that worldview. 

I'm using Chrome on Windows desktop. 

Strangely enough, the links here works:
> Many others have written about their rejection experience, here, here, and here.

I think there might be some missing links:
> In the current stage of LLMs, one may reasonably have short timelines for AGI coming in the next 3-5 years, as given here. Here, here, here, and here.

I think GWWC uses the term "active pledger" to refer to the pledgers that are still pledgers. I am #10294, which means I was the 10294th person to sign the pledge, but out of those 10294 people, 294 of them either cancelled their pledges or there were double-counts etc. So at the time when I pledged, there were 9999 existing others who were "active" i.e. had not cancelled their pledge. That doesn't mean the on-paper "active" pledgers actually adhere to their pledges, and I don't know if GWWC has a different term for that. 

<brag>
You don't know me, but I wasn't just one of the first 10,000. I was the 10,000th.
</brag>

Thank you so much! More importantly, congratulations to you and the GWWC team for this milestone!

Thank you very much! I should've known that the pledge # would be different from the active pledgers, but at I'm glad to know it now. I just made the pledge and I hope to be the 10,000th active pledger, even if momentarily :D