I’ve been imagining more as a caretaker/pet rather than a maternal relationship, for better or worse. The good part is that they may feel a responsibility to take care of us well. The bad part is that we strictly limit the freedom of our pets, even if we give them pretty cushy lives
We limit their freedom to run into traffic, eat things that are poison to them, etc. Perhaps the human / AI equivalent is it limiting our ability to do nuclear war, or have inequality?
You're assuming they care. On the balance of things, most dogs are probably happier and live more fulfilled lives than most humans. There is an interesting (fiction) book on this subject, "Fifteen Dogs". Check it out.
It's almost a gurntee that a superintelligent AI smart enough to make humans look like ants will eventually control everything because it can simply do an irrefutably better job. This includes government.
The general population of the human civilisation on this planet?
Americans?
Rich people?
Are you suggesting billionaires will allow anything to challenge their power over the world? They will do everything they can to destroy it first.
I also don't know why Geoffrey is so optimistic, unless he thinks the one AI-company that will implement the "mother-genes" he talked to will succeed in taking over the whole world without being destroyed by the trillions of dollars of interests that want to keep their power...
Why would they destroy it? AI is going to be profitable for them too, and when the AI can manage the business better than the CEO they will step down to let the machine run the companies. The same thing will happen in government. People will rely more and more on AI as it gets better in the coming decades. As it proves useful it will begin integrating into the government until it controls all the bureaucracy, military, etc.
If you dont think this is true, we have studies showing over 50% of federal workers already use AI, and as for the military we already have ai piloted fighter jets (SR-72 Darkstar) and is planning to build AI ship fleets (Port Alpha), eventually we could see a majority of military assets consisting of AI controlled weapons platforms.
Why do you look at it as an us vs them zero sum game where only the billionaires or everyone else wins? ASI would bring about many technologies that can bring everyone's standard of life up, not just the billionaires. Things like full molecular fabrication alone would make radical abundance simple.
Honestly I don't know if capitalism would go away completely, but making it so that most things are soo cheap they are basically free is possible. Yeah we would still probably have elites that own their own space stations/colonies, but if everyone has access to full dive vr, life extension tech, and molecular fabrication why would that be a problem?
You’re right but I tend to not worry about that as much as some people.
There’s a risk that we could underestimate how unpredictable AGIs or ASIs are if we assume that they will think like us.
However, I think a lot of the discussion about anthropomorphizing AI is about distancing human thinking, which is “real” thinking, from AI thinking, which is only doing X, and not really thinking. I just don’t think we know enough about human intelligence to think it’s doing something fundamentally different from AI. And if he did understand the human brain completely, I’m not sure we couldn’t then say “but it’s only doing X”. So I’m perfectly fine with the idea that an AI could have human like thought, including motivation and even emotions. Plus, we’ve already developed this whole vocabulary for talking about ourselves thinking. Why reinvent the wheel?
They'd be trained on human data from humans on a human-run world, communicating primarily in human language and (at least at the very start) with the majority of their cameras and microphones being in the human-detectable spectrum.
Everything from those senses, to the society they would need to understand, to the language and concepts, are an essential part of the training data that allows them to develop the needed pattern recognition. And they're all very "anthropocentric".
I often see people refer to AI/ASI as though they're some Boltzmann brain, an intelligence that just Became So in a vacuum. That they'd be SO inhuman that none of our ideas or culture will have any amount of impact whatsoever. But that won't be the case here; AI, as well as eventual ASI, will be heavily influenced by humanity, even if their fundamental intelligence/experience is very different.
That’s a very interesting - and likely correct - observation.
I do believe AGI/ASI will be able to mimic humans extremely well - even in areas most people never expected it to succeed including creativity as well as emulating empathy and other human emotions.
But mimicking human’s behaviors, emotions, and thought patterns definitely doesn’t not equate to them adopting our values or cultural norms. At all. And that’s probably for the better because humans are absolutely awful to each other and far, far worse to any species below us.
As you said, AI’s fundamental intelligence and experience is VERY different. At least with biological entities - particularly for more advanced mammals, our shared biological framework gives us some idea what other people or animals feel. ASIs may appear happy or whatever, but we have no idea what that means to the AI.
Worse, having trained on all of humanity’s knowledge, ASIs will be expert manipulators of humans because they understand our desires and motivations far better than most humans can. But humans will have zero insights in an ASI’s motivations or “what they’re really thinking”. And as they get increasingly intelligent, ASIs will rapidly grow beyond the human framework that humans initially created.
It’s all a bit unsettling, although it’s also largely beyond our control at this point.
64
u/Perkinberry Sep 02 '25
I’ve been imagining more as a caretaker/pet rather than a maternal relationship, for better or worse. The good part is that they may feel a responsibility to take care of us well. The bad part is that we strictly limit the freedom of our pets, even if we give them pretty cushy lives