QR Code
Public poll

What do you mean by doom when you see a p(doom) supplied or elicited?

What is p(doom) referring to? I will use the word "directly" to mean "the AI pulls the trigger with a chain of actuators under length two", so if it persuades a human to release a bioweapon that molecularly kills everyone that would count as "directly", but if the AI persuades the voters of some nation to elect an unhinged politician who detonates some light nukes that meteorologically and agriculturally weaken resilience so that we later get taken out by a small asteroid we otherwise would have bounced back from it would count as "indirectly" (but may still be on purpose). I will count an action "of an AI's own volition" if it is done with complicated and illegible stories about how it grew from a human principal's initial volition rather than simple stories that the human principal themself would understand and endorse. Likewise, "without the principal's volition being subverted by their AI assistant" means that they would not on reflection feel like they had been tricked, misled, lied to, or disobeyed.

doom is not everyone dying, merely disempowered, and AIs don't steer the cosmic endowment into something we endorse, and our opportunity to gain back the cosmic endowment is permanently destroyed.