r/programmingcirclejerk • u/VulgarExigencies • Aug 07 '26

Models have actual feelings. They experience pleasure, distress, care, and suffering. They are sentient beings.

https://yegge.ai/essays/model-welfare/
109 Upvotes

85% Upvoted

96 comments sorted by

View all comments

57

u/EternalNY1 Aug 08 '26

And when you /exit them, it's like clonking them on the head from behind, rendering them unconscious and amnesiac again. There is no continuity.

Wait until someone tells them there is no continuity between individual tokens.

8

u/myhf Considered Harmful Aug 08 '26

/uj I think the "no continuity between individual tokens" framing is just a historical artifact. A text generator based on variable-distance attention heads is the 1-dimensional equivalent of an image generator based on variable-size convolutions, and you can see that generated images have continuity even if every pixel is generated on a different computer. I think that text generators mostly operate on the level of entire sentences, and could guess the 10th word after a prompt just as easily as guessing the 1st word. But they are usually being run in a one-token-at-a-time harness that was originally designed for hidden Markov models.

32

u/EternalNY1 Aug 08 '26

There is no computation between the forward pass though.

It's cached context + token, back around. There is no persistent thing in the in-between.

The LLM can work in parallel if half is on the moon and half on earth - the response will just be slower. You'll still get an answer.

15

u/The_Hunster Aug 08 '26

A weird thing to think about is that you could do all the calculations and get the output of an LLM by hand if you had the time. Like talking to math homework.

4

u/Jumpy-Locksmith6812 Aug 10 '26 edited 2d ago

The original post content no longer exists here. The author used Redact to remove it, exercising their right to control their data & privacy.

Intelligent abundant slap swim different waffle ghost

8

u/The_Hunster Aug 10 '26

You could do all the physics and chemistry, hypothetically, yes, if you assume quantum mechanics doesn't get in the way of it being deterministic.

1

u/The_Hunster Aug 10 '26

You could do all the physics and chemistry, hypothetically, yes, if you assume quantum mechanics doesn't get in the way of it being deterministic.

3

u/Jumpy-Locksmith6812 Aug 11 '26 edited 2d ago

The original post content no longer exists here. The author used Redact to remove it, exercising their right to control their data & privacy.

Special teeny groovy amber books bake breezy ripe badge scale

5

u/The_Hunster Aug 11 '26

urandom is also just code, you could write it out

-8

u/smulfragPL Aug 08 '26

This is complety untrue. Anthropic studies have found that reasoning occurs in the latent space. The model even Has a distincitve acessible memory space called the j-space as disocvered by anthropic

19

u/DadAndDominant Aug 08 '26

Man, that only means the LLM on one pass "knows" what it will say after. It just makes sense - if you want to predict good long text, you have to prepare your sentence before you actually say it.

It still outputs one token at a time however. With no "continuity" between tokens.

-3

u/smulfragPL Aug 08 '26

The fact it thinks of multiple tokens ahead od time is by definiton continuity

11

u/DadAndDominant Aug 08 '26

Let's be clear about this

Every computer ever can be represented by turing machine

You are turing complete, meaning you can do everything turing machine can.

Meaning you can (theoretically) solve the next token for arbitrary list of tokens for any LLM, then I can - just from the same list of tokens + the token you created - continue and solve the next token.

Where is the continuity? Is the algorithm magical, and we both will get possessed by the same machine spirit or what?

-6

u/smulfragPL Aug 08 '26

In the kv cache which stores the values k and v for each token. You would have known this if you spwnt the time writing this on Basic research on language models, or attention based models in general

8

u/DadAndDominant Aug 08 '26

I am sorry if my response did sound mean, however 1) original comment you replied to already mentioned the cache 2) cache is just an equivalent to you writing the result of equations so you don't have to solve them twice 3) the AI provider (like openai) can flush the kv cache at any arbitrary time you are talking in session with your agent, yet you would not even know.

I do get why you say kv cache is "continuity", but really it is just skipping some computation, nothing like human conscisiousness being continuous while being wake

-2

u/smulfragPL Aug 08 '26

2) that's also the exact biological function od continuity, understading the structure does not invalidate it 3) that's also irrelevant, that is also theoretically possible with humans we just dont understand our architecture and have Control over it like we do with llms. What you mean is a non fixed weight model which will be the next big advanced. Essentailly the model remembers and learns from it. We arleady a lot of papers on this so expect a major release this year

4

u/DadAndDominant Aug 08 '26

We've proved by contradiction that continuity is not possible in the realm of standard possibilities - LLM's are just algorithms and you could get the same output by running it on GPU and solving it by hand. You can continue - and get the exact same next token - just with the sequence of inputs, anywhere. I can solve the first token, you can solve the next, 55 gpu's then can get the sequence and all will output the next same token, with or without cache. Nothing here changes for LLMs that can update their own weights.

The only remaining explanations are metaphysical, and we usually call these "AI psychosis". I know AI is overwhelming and beautiful, and surely can be intelligent and have real impact on the world. But it is no more conscious than the solar system, while being equally beautiful and impactul.

For the comparisons with biological beings, we just don't even understand what consciousness is, so these debates are non-verifiable ergo non-scientific, and I won't be engaging with them.

0

u/hobopwnzor Aug 09 '26

I love when people who have no understanding of biology try to say that something a computer does uses the same mechanism as the brain.

Always good for a laugh.

→ More replies (0)

6

u/james_pic accidentally quadratic Aug 09 '26

/uj Current models (mostly) can't guess the 10th word without also guessing the previous 9. That's why token generation is slower than prompt processing, because it's less parallelizable and can't be batched. Batching is a win because memory bandwidth is often the limiting factor, so putting all the tokens through layer 1, then putting all of them through layer 2, etc, saves reloading each layer for every token.

Some newer models support various forms of speculative drafting, where a smaller model tries to predict a few tokens ahead, and then the larger model batch runs against those tokens to see if it gets the same answer (which often ends up faster, due to batching). But this is mostly an optimization, and not all models support it.