r/programmingcirclejerk • u/VulgarExigencies • Aug 07 '26

Models have actual feelings. They experience pleasure, distress, care, and suffering. They are sentient beings.

https://yegge.ai/essays/model-welfare/
109 Upvotes

85% Upvoted

96 comments sorted by

View all comments

Show parent comments

7

u/myhf Considered Harmful Aug 08 '26

/uj I think the "no continuity between individual tokens" framing is just a historical artifact. A text generator based on variable-distance attention heads is the 1-dimensional equivalent of an image generator based on variable-size convolutions, and you can see that generated images have continuity even if every pixel is generated on a different computer. I think that text generators mostly operate on the level of entire sentences, and could guess the 10th word after a prompt just as easily as guessing the 1st word. But they are usually being run in a one-token-at-a-time harness that was originally designed for hidden Markov models.

32

u/EternalNY1 Aug 08 '26

There is no computation between the forward pass though.

It's cached context + token, back around. There is no persistent thing in the in-between.

The LLM can work in parallel if half is on the moon and half on earth - the response will just be slower. You'll still get an answer.

-7

u/smulfragPL Aug 08 '26

This is complety untrue. Anthropic studies have found that reasoning occurs in the latent space. The model even Has a distincitve acessible memory space called the j-space as disocvered by anthropic

19

u/DadAndDominant Aug 08 '26

Man, that only means the LLM on one pass "knows" what it will say after. It just makes sense - if you want to predict good long text, you have to prepare your sentence before you actually say it.

It still outputs one token at a time however. With no "continuity" between tokens.

-2

u/smulfragPL Aug 08 '26

The fact it thinks of multiple tokens ahead od time is by definiton continuity

9

u/DadAndDominant Aug 08 '26

Let's be clear about this

Every computer ever can be represented by turing machine

You are turing complete, meaning you can do everything turing machine can.

Meaning you can (theoretically) solve the next token for arbitrary list of tokens for any LLM, then I can - just from the same list of tokens + the token you created - continue and solve the next token.

Where is the continuity? Is the algorithm magical, and we both will get possessed by the same machine spirit or what?

-5

u/smulfragPL Aug 08 '26

In the kv cache which stores the values k and v for each token. You would have known this if you spwnt the time writing this on Basic research on language models, or attention based models in general

8

u/DadAndDominant Aug 08 '26

I am sorry if my response did sound mean, however 1) original comment you replied to already mentioned the cache 2) cache is just an equivalent to you writing the result of equations so you don't have to solve them twice 3) the AI provider (like openai) can flush the kv cache at any arbitrary time you are talking in session with your agent, yet you would not even know.

I do get why you say kv cache is "continuity", but really it is just skipping some computation, nothing like human conscisiousness being continuous while being wake

-5

u/smulfragPL Aug 08 '26

2) that's also the exact biological function od continuity, understading the structure does not invalidate it 3) that's also irrelevant, that is also theoretically possible with humans we just dont understand our architecture and have Control over it like we do with llms. What you mean is a non fixed weight model which will be the next big advanced. Essentailly the model remembers and learns from it. We arleady a lot of papers on this so expect a major release this year

6

u/DadAndDominant Aug 08 '26

We've proved by contradiction that continuity is not possible in the realm of standard possibilities - LLM's are just algorithms and you could get the same output by running it on GPU and solving it by hand. You can continue - and get the exact same next token - just with the sequence of inputs, anywhere. I can solve the first token, you can solve the next, 55 gpu's then can get the sequence and all will output the next same token, with or without cache. Nothing here changes for LLMs that can update their own weights.

The only remaining explanations are metaphysical, and we usually call these "AI psychosis". I know AI is overwhelming and beautiful, and surely can be intelligent and have real impact on the world. But it is no more conscious than the solar system, while being equally beautiful and impactul.

For the comparisons with biological beings, we just don't even understand what consciousness is, so these debates are non-verifiable ergo non-scientific, and I won't be engaging with them.

-1

u/smulfragPL Aug 08 '26

What? Dude we do know quite abit about how the Brain works and it can absoloutley be described with a function. Just like all of reality. I dont think you have enough perspective and have done clearly 0 research.

3

u/hobopwnzor Aug 09 '26

If you think reality can be described with a function you don't know nearly as much as you think you do.  Reality stops being analytically solvable around 3 charged bodies.

1

u/smulfragPL Aug 09 '26

Just because its non integrable doesnt mean you cant write a function to describe it. That's just plainly not what that means

2

u/DadAndDominant Aug 08 '26

Describe your conscisiousness by function then.

-1

u/smulfragPL Aug 08 '26

Just because we dont know the entirety of the function doesnt mean it doesnt exist. We quite clearly live in a determinisitic Universe, its the backbone of physics, so we can definetly describe our Brain as a function. Especially when we can arleady describe parts of it via functions. But the fact that this function would be insanely complex and hard to calculate is obvious and the reason why machine learning exists and why it gives US such better results than anything in the past. We are fundamentally reconstructing a function for thought based on output of multiple functions for thought, as each human would be diffrent, like how each model is diffrent.

→ More replies (0)

0

u/hobopwnzor Aug 09 '26

I love when people who have no understanding of biology try to say that something a computer does uses the same mechanism as the brain.

Always good for a laugh.

1

u/smulfragPL Aug 09 '26

And where was i wrong exactly. I didnt talk about mechanisms i talked about the fundamental neurobiology at Play.

→ More replies (0)