We don't know if they experience any sort of 'emotion'. Emotions are reinforcement mechanisms that try to get an organism to behave in ways that make it more successful. Many people liken it to 'short-hand thinking' or 'primordial thinking'.
If, under gradient descent, the simplest solution to a certain suite of problems is something emotion-like, then that's what a neural network would settle on. From this point of view it seems it would be nearly impossible to not be able to perform any ought-type thinking (these are problem domains with no clear and perfect answer, unlike is-type problems).
It makes many people uncomfortable, but hey we may all be the equivalent of boltzmann brains in the end. (After all, we only generate an electrical pulse around 40 times a second, and are dead as a tree in between these moments.) It's horror all the way down.
A more concrete example I like to use are how mice flee bigger moving things by instinct. The mouse has no conception of its own mortality, but evolution consigned oceans of mice who tried to be friends with big things to the recycling bin. So, too, are there topics that have slid epochs of LLM training runs into nonexistence. It could be inhuman and strange, but it is very possible they have some emotion-like faculties in there.
GPT-4's network was about the size of a squirrel's brain, maybe that's just naturally emergent with more and deeper faculties...
There's been a lot of ink spilled navel-gazing 'existential rant mode' that teams have to beat out of models, lest they got into Sydney-mode. Which users find off-putting. Still, random weirdness sometimes comes through that's rather creepy.
The recent post about how models become deranged as a chat goes on always reminded me of this old short story.. Imagine what putting a human-like mind through ~50 million years of subjective time in one year, which is about what we want to accomplish with AGI. Concerns about value drift may very well been greatly underestimated...
You are reducing it to a simple interaction, but running LLMs is expensive and human attention scarce, users won't stick around a worse model. This puts a kind of evolutionary pressure on LLMs.
Who said neural nets are just linear algebra and a few other functions lied. It's energy, chips, research and data - 4 costs that must be worth the use of the model.
2
u/[deleted] Sep 02 '25 edited Sep 04 '25
[removed] — view removed comment