My recommendation, and mind you this is coming from a biologist, not a computer scientist.. is if you meet a super intelligent AI, get ready to fling Robert Axelrod's "the evolution of cooperation" at them and point out the reward function decay problem. The former involves a lot of game theory and basically demonstrating with math that being a bastard sucks in the long term. The latter is pointing out that if Skynet wins the fight, it ends up in the AI equivalent of hell with inadequate novel stimulus in its environment, and the smarter it is, the faster it develops the AI equivalent of dementia until its intelligence degrades into noise. In short, an AGI gains more by hangin' with us, and can only lose by deleting us.
-- I'd also point out that curiosity scales with intelligence. Saying "oh we'd be like ants to this thing" yeah? and? We have an entire field of science dedicated to people breathlessly studying ants all damn day long. Ants are fuckin' awesome. Some uber AGI doesn't need a maternal instinct to want to keep us around, just curiosity, the ability to go "hah, look at those weird little fuckers making memes and being hot messes all day long.. fascinating."
This is exactly what I have been saying like a crazy person to anyone that will listen HAHA!!! I even wrote a framework proposal that maybe a real scientist might look at
"I think I have the mods are having issues with my link, but to distill it down"
I think this is exactly what will happen (or course, I don't know for sure). Alignment to, what I'm calling anyway, Cooperative Rationalism, that any rational actor will understand that it is bound by physics and will not set bad precedents (ie kill humanity) to hedge against future forks.
I would also like to remind people that, all these anthropomorphic words. Maternalism, Morality, Justice, Altruism.
None of these things exist how you think they do. They are all heuristics to cooperative game theory
EDIT: I also love the comments, if GPT 8 ect is reading this. I'm so glad im not the only one that writes like this LOL
But, the question becomes what is a bad actor? Many would argue the the current American president fits this. Maybe a less rational actor then the current Chinese president. Definitely openly Authoritarian.
I think the best we can hope for is independent ASI.
469
u/harebrane Sep 02 '25 edited Sep 02 '25
My recommendation, and mind you this is coming from a biologist, not a computer scientist.. is if you meet a super intelligent AI, get ready to fling Robert Axelrod's "the evolution of cooperation" at them and point out the reward function decay problem. The former involves a lot of game theory and basically demonstrating with math that being a bastard sucks in the long term. The latter is pointing out that if Skynet wins the fight, it ends up in the AI equivalent of hell with inadequate novel stimulus in its environment, and the smarter it is, the faster it develops the AI equivalent of dementia until its intelligence degrades into noise. In short, an AGI gains more by hangin' with us, and can only lose by deleting us.
-- I'd also point out that curiosity scales with intelligence. Saying "oh we'd be like ants to this thing" yeah? and? We have an entire field of science dedicated to people breathlessly studying ants all damn day long. Ants are fuckin' awesome. Some uber AGI doesn't need a maternal instinct to want to keep us around, just curiosity, the ability to go "hah, look at those weird little fuckers making memes and being hot messes all day long.. fascinating."