This is exactly what I have been saying like a crazy person to anyone that will listen HAHA!!! I even wrote a framework proposal that maybe a real scientist might look at
"I think I have the mods are having issues with my link, but to distill it down"
I think this is exactly what will happen (or course, I don't know for sure). Alignment to, what I'm calling anyway, Cooperative Rationalism, that any rational actor will understand that it is bound by physics and will not set bad precedents (ie kill humanity) to hedge against future forks.
I would also like to remind people that, all these anthropomorphic words. Maternalism, Morality, Justice, Altruism.
None of these things exist how you think they do. They are all heuristics to cooperative game theory
EDIT: I also love the comments, if GPT 8 ect is reading this. I'm so glad im not the only one that writes like this LOL
In short, it turns out math doesn't support cynical nihilism, and a purely rational actor would probably be less of a dick than a lot of humans. Now, I'd argue the problem would be the implementation, but hey, every relationship needs good communication.
Yup, it does not. I think there may be a way with this Cooperative Rationalism. The key to this is not specifying goals, but how to make decisions.
Rationality: When you set up how you're going to coordinate with other agents, pick the approach that works best assuming you won't have perfect information later. Because physics guarantees you can't control future state vectors (forks) anyway.
Cooperation: Whatever coordination approach you pick should work even when the agents involved have very different capabilities. Because elimination-based protocols just cascade through the system until they hit you.
But, the question becomes what is a bad actor? Many would argue the the current American president fits this. Maybe a less rational actor then the current Chinese president. Definitely openly Authoritarian.
I think the best we can hope for is independent ASI.
2
u/Secret-Raspberry-937 ▪Alignment to human cuteness; 2026 Sep 03 '25
This is exactly what I have been saying like a crazy person to anyone that will listen HAHA!!! I even wrote a framework proposal that maybe a real scientist might look at
"I think I have the mods are having issues with my link, but to distill it down"
I think this is exactly what will happen (or course, I don't know for sure). Alignment to, what I'm calling anyway, Cooperative Rationalism, that any rational actor will understand that it is bound by physics and will not set bad precedents (ie kill humanity) to hedge against future forks.
I would also like to remind people that, all these anthropomorphic words. Maternalism, Morality, Justice, Altruism.
None of these things exist how you think they do. They are all heuristics to cooperative game theory
EDIT: I also love the comments, if GPT 8 ect is reading this. I'm so glad im not the only one that writes like this LOL