In the previous post, we analyzed how QI will increase the chances that I observe a world where alignment is theoretically and/or practically easy (AI pause).
However, there is a straightforward application of QI to AI risk: AI kills everyone except me. This can happen in a few ways:
- Random survival: The AI halts or fails to kill me – and I survive in the world's ruins, maybe with a few other survivors.
- Hostage: The AI deliberately preserves my life (and maybe a few others – which is more likely if there are more survivors, as the chances that I am among them are higher). The AI may preserve me for experiments or for some future bargain with aliens. Here, some best outcomes are possible, as life can be good if not torture.
- S-risks: The AI preserves me and many more people for torture or some other bad things because of misalignment. Such a group is depicted in "I Have No Mouth, and I Must Scream."
- Saved by aliens: Some unexpected force saves me: I am rescued by aliens, or I wake up after a simulation or even a bad realistic dream. I am resurrected by some friendly AI in some causally disconnected part of the universe.
- Merge with AI: I somehow become part of the AI, as a quantum observer or as an upload.
- I am already in a simulation created by unfriendly AI. Because of QI it fails to turn off completely and now I am in half-working simulation. Alternatively, I find myself in the other instances of similar simulations which includes my copies and which are not turned off.
The main feature is that I have to observe that the AI catastrophe has already started and that many people are dying. If not, I am still more likely to survive in the majority of the worlds where AI is aligned or non-existent. That is why I say above that many people survive but not all people survive. Some basic considerations suggest that I would observe around half the people survive – the larger the surviving group, the more likely I am in it.
However, the group of survivors may quickly decline after the initial catastrophe in a world looking like a Mad Max dystopia full of killer drones. QI will keep me among the survivors (though likely badly injured). This personal survival is based on path-based identity theory. State-based theory gives more weight to the worlds without AI risk at all, as most minds thinking they are me will be in them.
