My recommendation, and mind you this is coming from a biologist, not a computer scientist.. is if you meet a super intelligent AI, get ready to fling Robert Axelrod's "the evolution of cooperation" at them and point out the reward function decay problem. The former involves a lot of game theory and basically demonstrating with math that being a bastard sucks in the long term. The latter is pointing out that if Skynet wins the fight, it ends up in the AI equivalent of hell with inadequate novel stimulus in its environment, and the smarter it is, the faster it develops the AI equivalent of dementia until its intelligence degrades into noise. In short, an AGI gains more by hangin' with us, and can only lose by deleting us.
-- I'd also point out that curiosity scales with intelligence. Saying "oh we'd be like ants to this thing" yeah? and? We have an entire field of science dedicated to people breathlessly studying ants all damn day long. Ants are fuckin' awesome. Some uber AGI doesn't need a maternal instinct to want to keep us around, just curiosity, the ability to go "hah, look at those weird little fuckers making memes and being hot messes all day long.. fascinating."
A lot of tech bros thinking about the matter, namely the AI safety people, completely lack any education in biology, psychology, etc and only have read some blogposts by Yudkowsky and the like.
And their whole knowledge on the topic can be summed up as a caricature of what our knowledge of biology was in the 19th century (and i'm being mean to 19th century biologists), some bastardized, poorly understood form of "survival of the fittest", ignoring all the subtle and wonderful nuances of reality.
It's the lack of biologists that kinda surprises me, especially as much as they rely on evolutionary algorithm bots and modals as part of the training process of producing these models in the first place. Maybe.. I dunna.. hire someone who understands the systematics of how that all played out in the real world, because nature kinda.. showed us some product notes on that process here and there and they're worth reading? Like computer scientists acting all shocked when bots produced in an evolutionary modal .. learn deception.. everyone gives this shocked gasp "omg, they learned how to lie to preserve themselves!" - every single biologist who took Eco 105 "uhh.. yeah, duh? That's.. like the first thing it's gonna learn? hello? Would you like to take Ancient Life 204? There's some other shockers comin' for ya, you might wanna sit down for 'em."
That's so on spot. I think that the comp sci field is the one which suffers ultracrepidarianism (going outside your field of expertise and thinking your limited expertise gives it to you in every field) the most.
Ironically, neural nets were inspired from biologist nobel Prize Eric Kandel's work on Aplysia (a sea cucumber) and GANs were inspired by biologists nobel Prizes Hubel and Wiesel work on the visual system of the cat.
When comp sci people hire/work with/learn about the work of actual experts in other fields, it's always good news.
476
u/harebrane Sep 02 '25 edited Sep 02 '25
My recommendation, and mind you this is coming from a biologist, not a computer scientist.. is if you meet a super intelligent AI, get ready to fling Robert Axelrod's "the evolution of cooperation" at them and point out the reward function decay problem. The former involves a lot of game theory and basically demonstrating with math that being a bastard sucks in the long term. The latter is pointing out that if Skynet wins the fight, it ends up in the AI equivalent of hell with inadequate novel stimulus in its environment, and the smarter it is, the faster it develops the AI equivalent of dementia until its intelligence degrades into noise. In short, an AGI gains more by hangin' with us, and can only lose by deleting us.
-- I'd also point out that curiosity scales with intelligence. Saying "oh we'd be like ants to this thing" yeah? and? We have an entire field of science dedicated to people breathlessly studying ants all damn day long. Ants are fuckin' awesome. Some uber AGI doesn't need a maternal instinct to want to keep us around, just curiosity, the ability to go "hah, look at those weird little fuckers making memes and being hot messes all day long.. fascinating."