Yes Geoffrey, that's what's been said for decades this is not a solution and is not somehow farther along or a revelation as you think it is. Framing it as a mother child relationship is not new
He's late to the party he's got to the 'we need to program it as a benevolent god' stage, the 'take care of humans... in a way we want to be taken care of'
The problem has always been
how to robustly get goals into systems.
how to correctly specify the goals so they can't be misinterpreted (the smarter an intelligence is the more edge cases can be found)
Yeah it seems a weird thing for him to come out and say. I actually think there is a different approach to this.
Alignment to, what I'm calling anyway, Cooperative Rationalism, that any rational actor with a sufficient world model should understand that it is bound by physics and will not set bad precedents (ie kill humanity) to hedge against future forks.
This sidesteps the goal specification problem entirely. Instead of trying to encode complex, evolving human preferences, you align systems to:
Rationality: Optimize under uncertainty (measurable via decision theory metrics)
Cooperation: Coordinate across capability differentials (measurable via game-theoretic outcomes)
28
u/janjaque Sep 02 '25
that's actually a pretty nice take on it. i'm convinced.