OptionProbability
J. Something 'just works' on the order of eg: train a predictive/imitative/generative AI on a human-generated dataset, and RLHF her to be unfailingly nice, generous to weaker entities, and determined to make the cosmos a lovely place.
K. Somebody discovers a new AI paradigm that's powerful enough and matures fast enough to beat deep learning to the punch, and the new paradigm is much much more alignable than giant inscrutable matrices of floating-point numbers.
M. "We'll make the AI do our AI alignment homework" just works as a plan. (Eg the helping AI doesn't need to be smart enough to be deadly; the alignment proposals that most impress human judges are honest and truthful and successful.)
C. Solving prosaic alignment on the first critical try is not as difficult, nor as dangerous, nor taking as much extra time, as Yudkowsky predicts; whatever effort is put forth by the leading coalition works inside of their lead time.
Something wonderful happens that isn't well-described by any option listed. (The semantics of this option may change if other options are added.)
A. Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity's window of fragility.
B. Humanity puts forth a tremendous effort, and delays AI for long enough, and puts enough desperate work into alignment, that alignment gets solved first.
I. The tech path to AGI superintelligence is naturally slow enough and gradual enough, that world-destroyingly-critical alignment problems never appear faster than previous discoveries generalize to allow safe further experimentation.
O. Early applications of AI/AGI drastically increase human civilization's sanity and coordination ability; enabling humanity to solve alignment, or slow down further descent into AGI, etc. (Not in principle mutex with all other answers.)
G. It's impossible/improbable for something sufficiently smarter and more capable than modern humanity to be created, that it can just do whatever without needing humans to cooperate; nor does it successfully cheat/trick us.
E. Whatever strange motivations end up inside an unalignable AGI, or the internal slice through that AGI which codes its successor, they max out at a universe full of cheerful qualia-bearing life and an okay outcome for existing humans.
D. Early powerful AGIs realize that they wouldn't be able to align their own future selves/successors if their intelligence got raised further, and work honestly with humans on solving the problem in a way acceptable to both factions.
H. Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, and humanity is part of this equilibrium and survives and gets a big chunk of cosmic pie.
L. Earth's present civilization crashes before powerful AGI, and the next civilization that rises is wiser and better at ops. (Exception to 'okay' as defined originally, will be said to count as 'okay' even if many current humans die.)
If you write an argument that breaks down the 'okay outcomes' into lots of distinct categories, without breaking down internal conjuncts and so on, Reality is very impressed with how disjunctive this sounds and allocates more probability.
F. Somebody pulls off a hat trick involving blah blah acausal blah blah simulations blah blah, or other amazingly clever idea, which leads an AGI to put the reachable galaxies to good use despite that AGI not being otherwise alignable.
N. A crash project at augmenting human intelligence via neurotech, training mentats via neurofeedback, etc, produces people who can solve alignment before it's too late, despite Earth civ not slowing AI down much.
You are fooled by at least one option on this list, which out of many tries, ends up sufficiently well-aimed at your personal ideals / prejudices / the parts you understand less well / your own personal indulgences in wishful thinking.
18
15
15
11
8
5
5
5
5
4
3
2
1
1
1
0
0
0
OptionProbability
The Legend of Zelda
Donkey Kong Country
Cyberpunk (including Orion, etc)
Animal Crossing
Splatoon
Pokémon
Super Mario
Plants vs. Zombies
Red Dead Redemption
Katamari (塊魂)
Super Smash Bros.
Dragon Quest
WarioWare
Grand Theft Auto
Pikmin
The Elder Scrolls
Tekken
Call of Duty
Halo
F-Zero
Luigi's Mansion
Mario Golf
Shantae
Fable
Microsoft Flight Simulator
Rhythm Heaven
Tales
virtua fighter
Dark Souls
Punch-Out!!
Fallout
Pokémon Snap
Itadaki Street (Fortune Street/Boom Street)
Arms
Bayonetta
Nintendogs
Itadaki Street (Fortune Street/Boom Street), in North America
The Sims (excluding MySims)
Kid Icarus
Tomodachi Life/Tomodachi Collection
Homestuck/Hiveswap
Denpa Men
Golden Sun
Labo
Pokémon Pinball
Mother/EarthBound
100
100
100
99
99
99
99
99
91
91
90
89
87
86
85
84
83
83
77
69
69
66
61
60
57
54
50
50
46
44
41
39
39
39
37
37
34
31
30
29
27
26
21
15
14
11
OptionProbability
[With five or more methods]
With their bare hands
With a weapon that the victim brought to the fight.
With a gadget or other ingenious device from Q
Other
With a hand gun
In the dark
With a long arm (a firearm larger than a pistol).
With a Walther PP or PPK handgun
By causing a structural collapse under them.
Slap
Drowning them
By pushing them off something high up
Sideswiping them with a car
Choking them
Silently, but deadly
With a named MMA submission (e.g., rear-naked choke, arm bar, kimura lock, etc.)
With a knife
Dropping something heavy on them
With electricity
By driving over them in a car
Biting
With a Kalashnikov Rifle (e.g. AK47)
Flambé /Fire
hurts someone's feelings (character in question must state this)
With a timed explosive device
By opening a hatch/airlock
With poison
Drone strike
With his cock (like a fighting Rooster; get yer minds outta that gutter)
Electrocution / With electricity
With a power tool normally used for construction
Lures them into a booby-trapped room
By calling in artillery or missile strike
Arranging for them to be attacked by an animal
With a piece of sporting equipment
Sonic weapon
With a chain
Flying/jump/drop kick
With a grenade
With a flamethrower
Punch to the groin
With a lazer based weapon
With a missle
Friendly fire (Bond unintentionally injures an ally)
With nanobots
Ejector seat
Big stick
With a sling shot
Crashing a plane into them
With a harpoon gun
With a crossbow
By pushing them into a spinning blade
By shaking them, not stirring them
By riding over them with a motorcycle
Killing them with kindness
Non-knife kitchen utensil
By running into them with a submarine
Rape
With microwaves
With a motherfucking snake on a motherfucking plane
Causing them to crash into a wall by painting a tunnel onto it
Piano where hitting a certain note causes it to explode but Bond plays the song incorrectly and then the villain shows him how to play it correctly
93
92
87
86
82
80
74
68
66
56
56
54
54
54
54
52
48
44
44
41
40
40
38
38
38
36
33
32
31
29
28
28
27
27
26
26
25
25
25
24
24
23
22
22
22
21
20
19
19
18
16
16
15
13
12
11
11
10
10
9
8
7
2
OptionProbability
Vinland Saga
Bleach TYBW
One Punch-Man (webcomic original)
Yotsuba &!
Kaiji
Skip Beat!
Hajime no Ippo
JoJo's Bizarre Adventure
Kingdom
Detective Conan
Vagabond
Berserk
Hunter x Hunter
100
97
95
50
50
50
49
36
33
23
21
17
9
OptionVotes
YES
NO
773
110
OptionVotes
YES
NO
4923
203
OptionProbability
K. Somebody discovers a new AI paradigm that's powerful enough and matures fast enough to beat deep learning to the punch, and the new paradigm is much much more alignable than giant inscrutable matrices of floating-point numbers.
I. The tech path to AGI superintelligence is naturally slow enough and gradual enough, that world-destroyingly-critical alignment problems never appear faster than previous discoveries generalize to allow safe further experimentation.
C. Solving prosaic alignment on the first critical try is not as difficult, nor as dangerous, nor taking as much extra time, as Yudkowsky predicts; whatever effort is put forth by the leading coalition works inside of their lead time.
B. Humanity puts forth a tremendous effort, and delays AI for long enough, and puts enough desperate work into alignment, that alignment gets solved first.
Something wonderful happens that isn't well-described by any option listed. (The semantics of this option may change if other options are added.)
M. "We'll make the AI do our AI alignment homework" just works as a plan. (Eg the helping AI doesn't need to be smart enough to be deadly; the alignment proposals that most impress human judges are honest and truthful and successful.)
A. Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity's window of fragility.
E. Whatever strange motivations end up inside an unalignable AGI, or the internal slice through that AGI which codes its successor, they max out at a universe full of cheerful qualia-bearing life and an okay outcome for existing humans.
J. Something 'just works' on the order of eg: train a predictive/imitative/generative AI on a human-generated dataset, and RLHF her to be unfailingly nice, generous to weaker entities, and determined to make the cosmos a lovely place.
O. Early applications of AI/AGI drastically increase human civilization's sanity and coordination ability; enabling humanity to solve alignment, or slow down further descent into AGI, etc. (Not in principle mutex with all other answers.)
D. Early powerful AGIs realize that they wouldn't be able to align their own future selves/successors if their intelligence got raised further, and work honestly with humans on solving the problem in a way acceptable to both factions.
F. Somebody pulls off a hat trick involving blah blah acausal blah blah simulations blah blah, or other amazingly clever idea, which leads an AGI to put the reachable galaxies to good use despite that AGI not being otherwise alignable.
L. Earth's present civilization crashes before powerful AGI, and the next civilization that rises is wiser and better at ops. (Exception to 'okay' as defined originally, will be said to count as 'okay' even if many current humans die.)
G. It's impossible/improbable for something sufficiently smarter and more capable than modern humanity to be created, that it can just do whatever without needing humans to cooperate; nor does it successfully cheat/trick us.
H. Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, and humanity is part of this equilibrium and survives and gets a big chunk of cosmic pie.
N. A crash project at augmenting human intelligence via neurotech, training mentats via neurofeedback, etc, produces people who can solve alignment before it's too late, despite Earth civ not slowing AI down much.
You are fooled by at least one option on this list, which out of many tries, ends up sufficiently well-aimed at your personal ideals / prejudices / the parts you understand less well / your own personal indulgences in wishful thinking.
If you write an argument that breaks down the 'okay outcomes' into lots of distinct categories, without breaking down internal conjuncts and so on, Reality is very impressed with how disjunctive this sounds and allocates more probability.
20
12
10
8
8
7
6
5
5
5
3
3
3
2
1
1
1
1
OptionProbability
Other
2025 Q3/Q4
2026 Q1/Q2
2026 Q3/Q4
2027 Q1/Q2
2027 Q3/Q4
2025 Q1/Q2
2024 Q1/Q2
2024 Q3/Q4
58
8
8
8
8
8
2
1
1
OptionVotes
YES
NO
283
35
OptionProbability
It will never be revealed / it is just his workout
Other
Android / Robot / Cybernetic enhancements
Alien / nonhuman race
Magic
Godhood / Godlike powers
54
14
11
9
6
6
OptionProbability
Other
Fire Devil
Yoshida and/or Octopus Devil
The Chainsawman Devil
Justice Devil
Darkness Devil
Pochita Bud Offshoot
Blood Devil and/or Power
Nayuta
Fire Punch Crossover Character
Another Chainsaw Devil
Kobeni
Other
34
24
4
4
4
4
4
4
4
4
4
4
4
OptionVotes
NO
YES
112
89
No stories found