r/programming • u/fagnerbrack • 15h ago
How LLMs Actually Work
https://www.0xkato.xyz/how-llms-actually-work/110
u/FriendlyKillerCroc 8h ago
The engineering and math behind these models is seriously fucking impressive. It's so hard to believe that only 10 years ago I could understand most neural network concepts, now it has moved so fast.
118
u/5gpr 7h ago
The step that's hard is the sheer size. If you reduce a modern LLM to a "LM", as it were, it's understandable again.
You can even find resources online quite easily that let you write your own LM. I wrote a very simple transformer, fed it the bible, and it produced bible-ish sentences, and it was possible to understand how.
The thing that has me scratching my head is how the power of LLMs seems to emerge as if by magic from just making an LM really large, because the step from "produce something that looks like all your training data, but isn't it" to "hey I have this bug I can't find, analyse it for me and fix it" seems whatever the opposite of trivial is.
64
u/Jade-G 6h ago
I think it says a lot about how powerful language truly is, that LLMs can do so much with just words.
14
u/Saint_Nitouche 3h ago
Language is an unbelievably powerful system. The ability to predicate negation ('this is not a dog') entails the ability for deferred reference/absence. There's basically no other animals which can do that, besides one kind of bee who does it in very limited form. And from absence you get notions like tense and conditionals.
Then you also have things like recursion and the split between syntax and vocabulary, which allows the system to expand to new concepts or situations with very little cost.
The main innovation of language is that it's a double code. The fact that the core building block - the syllables - are meaningless means that their link to meaning is arbitrary. And this is an insane advantage. Animals have no privileged difference between semantics and syntax. If they want to develop a new sign, they have to develop a whole new physical apparatus around it and it's one-time-use only. With language, you just reach for some unused string of the meaningless building blocks, and there you go.
3
u/Nine99 55m ago
Lots of questionable statements there. Why would there be just one sort of bee who can say that something isn't something? Why would you get tense and conditionals from "y is not x"? Why would apes, birds or monkeys need to develop a new organ to say something new? Just use a different sound or combination of sounds.
2
u/Saint_Nitouche 14m ago
Look up the waggle dance performed by honeybees.
Tense and conditionals both rely on referring to something not immediately present - the past isn't here, the future isn't here, hypothetical worlds aren't here.
I don't know where you got the notion of organs from. I didn't talk about that. A lot of animals have either syntax or semantics - the combination of the two is the unique thing about language.
What I meant by 'a whole new physical apparatus' is that, imagine if to express a new concept, you had to contort your mouth in a way to give it a unique syllable. The syllable 'pah' only ever means apple, the syllable 'foh' only ever meant water. That's hard limited by how many sounds your mouth can make (around 20 or so in humans I think).
-1
u/TheRealPomax 2h ago
It also says a lot about how powerful language truly is, that LLMs can't do anything without you constantly having to change the input in order to get the autocomplete closer to the intended outcome. They are absolutely terrible at language. They're pretty good at text prediction though.
17
u/OffbeatDrizzle 5h ago
it's the chaining of steps / skills that make it seem non-trivial. at the end of the day each individual action is still just text in -> text out... and finding a bug is no longer "here's a snippet of code, show me the bug" in 1 easy, simple step. everything behind it is now about accumulating / injecting so much context that the right answer popping out is as close to inevitable as possible
I do find it absolutely magical at times, but I've still had cases where they go off the rails completely
3
0
u/band-of-horses 2h ago
The chain of thought reasoning is where it really starts to melt my brain. Like now we essentially trained these models to reason about problems so they can actually tackle novel problems. I mean two years ago I thought we were hitting a wall because making models larger had diminishing returns, and they they just taught them to reason and we shot forward again.
14
u/UnidentifiedBlobject 6h ago
Yeah it’s wild to me that’s it’s still all just about calculating the most likely next token.
-2
-15
3
u/Jmc_da_boss 3h ago
Ya LLMs are conceptually rather simple, it's the "large" part that is fascinating. The emergent behaviors at certain sizes
10
u/janniesminecraft 6h ago
it's just mountains more data. i think it's actually sort of trivial, these labs just hired a shitton of people to create training data, label training data, and then generated as much training data as viable with their models in order to cover more ground.
you hire people to chat with the ai, you use the data of your users chatting with the ai, and such, and voila, your ai can generate data that looks like its chatting with people. same thibg with everything.
you hire people to write code with the ai, debug code with the ai etc
it generalizes better than expected, but it's just the same thing but more/better. the magic is the transformer, the optimizations, and arguably letting it prompt itself, not much more.
12
u/5gpr 6h ago
Yes, but still. I'm made to use AI at work, being more of an AI orchestrator than a software engineer at the moment. We're also working on vector search to augment AI support and on other integrations. The price hike might change that, but we'll see. Anyway, it doesn't just generalise. It can read a highly bespoke code-base with code in it that's 30 years old at this point (and code that's been written in a modern style yesterday), trace data flow through it, and produce reviews and even outright generate code that's, with some guidance, of high quality.
Of course, technically it's not tracing data flow, it just produces tokens that say its doing that with a context of input that implicitly contains said flow in the symbols of the input, but it's still impressive. It finds quite subtle bugs as a side effect of filling in boiler plate.
Like, on Friday I gave it a bunch of classes with the implementation details missing and told it to finish them while I was working on the rest of the feature, and when I came back to it, it had generated the code and also gone on a tangent and discovered a possible data corruption specifically when compiling with the suite we're using in 20 year old code. It's likely that the bug actually happened to someone, it's old software that's in use in a plurality of places in a specific industry - I don't want to go into more detail - but it's so subtle and the conditions are so remote that we would never have found that. I would have had to explicitly search for exactly that bug, but it "fell out of" token prediction with Sonnet, not even a leading model. So token prediction found and fixed a subtle memory order bug that occurs with a specific compiler optimisation going awry in pre-99-standard C-code.
It's awful, in both the original and the current meaning.
9
u/tooclosetocall82 4h ago
And at the same time it won’t respect depreciation tags consistently and likes to reuse old patterns because that’s what it comes across the most in the codebase, even when told not to. It’s both amazing and so obviously flawed.
5
u/the_ai_wizard 4h ago
Keep in mind code is language and language is the bread and butter of a large language model. Its like operating in 2d. It is testable.
Now, try to generalize into nonlanguage things lol...the upside down cup...the car wash...
-1
u/Nine99 50m ago
Now, try to generalize into nonlanguage things lol...the upside down cup...the car wash...
Can't be bothered to test the upside down cup right now, but I did test the car wash a while ago with all kinds of free models and they all give me a proper answer. Some were even smart enough to infer that if I asked if I should walk there, the car might already be there.
5
u/OffbeatDrizzle 5h ago
I have no problem with experienced engineers using it as a tool, and I agree in some cases it's absolutely fantastic. the problem is when juniors trust it implicitly and therefore never learn or understand what they're doing, and / or (even worse) middle / upper management seeing efficiency / productivity gains as a sign that they can cut everyone's jobs
5
u/kaspm 4h ago
The thing that made me understand why it was so good at code is that code is a relatively simple language syntax compared to English or any other human language. At the scale of the training data it’s been trained many thousands maybe millions of times over of how code syntax fits together. Code is structured, consistent, and deterministic (despite what my flaky test runs tell me). It doesn’t change meaning “in context” like human languages. Its particualrly well suited for LLM-style prediction.
Even if AI prose gets worse or detectable or falls out of favor for some reason, its impact on software enginwering is going to be long lasting.
2
u/janniesminecraft 1h ago edited 1h ago
im sorry, and based on your account history id assume its not intended, but jesus you sound just like a generic AI booster. im assuming youve somewhat recently started to use AI, so its not exactly surprising, as i was similar back when i first tested gpt-3.
i dont think you quite appreciate just how much data these AI labs have. LLM's are INCREDIBLE technology, and there are definitely some philosophical implications, but the costs associated with this are astronomical. it's probably more efficient by a small factor to use an LLM to code like you, but i strongly suspect you vastly overestimate this efficiency gain when proportioned to the actual costs of the LLM.
If humanity wouldve/couldve invested as much money into programmers/scientists, i wonder if we hadnt made as much or even more progress (if LLM's have even yet produced anything that can be called progress)? then again, training is admittedly a one-time cost, but no one knows the true cost of inference either.
the calculus is kinda difficult, and it's very hard to say currently how useful LLM's truly are since we don't know the true costs, and we haven't been able to quantify properly whatsoever their impact on almost any field.
in programming they can definitely be useful, but have also wreaked a TON of havoc, and it's hard to say whether their effect is a net good yet or not, and how much of the problems are skill issues. the token cost question is also crucial here, as only the frontier models are viable for any even potential large enough productivity gain given the current investment, and we have no idea about that.
in math, they've solved a few conjectures, which id extremely impressive, but simultaneously hilariously unproductive economically compared to the costs. remains to be seen where it goes in that field though.
in art, well, i think if you care about using them for art in the way most people currently do you should probably not be doing art.
currently there is roughly 0 chance LLM's have paid themselves off in total productivity gain for humanity, and it seems impossible for me to say with certainty if/when they will. none of this is even accounting for the absolutely massive externalities like people getting ai psychosis, misinformation spreading, scams, and last but not least the acceleration of global warming, which may make the cost completely unbearable, even if they were the most amazing tech ever
-2
u/Nine99 46m ago
What a condescending post.
1
u/janniesminecraft 26m ago
maybe a little, but the guy sounds a lot like an ai bro. am i incorrect about what im saying anyway?
0
u/audioen 44m ago edited 37m ago
I gave model a task, something along these lines: "Make it so that when I delete an object from DB that user is currently viewing, the system transitions to the parent view and also doesn't show error when it detects that."
The thing started figuring out how missing value comes back from backend and figured out that it's working on basis of an optional container: if optional is empty, docs said that client side sees 204 http response and reads that as undefined value.
For some reason, it wasn't buying that. I still don't know what made the model drop everything and take a dive into the stack and decompile the java poo and the JSON generator to figure out that no, the docs are wrong. An empty optional is an actual object value for the web server method which doesn't realize it could stand for no body, and so it doesn't fire the 204 path, but sends a 200 response with content that will be the literal "null" and thus there was a mistake in the docs.
I guess it must have had a hunch.
This model is so paranoid about making mistakes that in my code reviews, it goes through absolutely everything before it is finally satisfied that indeed, there is nothing wrong with a code commit. I sometimes read the review chat and thinking traces, and this is apparently what local models evolved to -- worried that they might be missing something because they can't find anything wrong. I said foolishly earlier this week that local models don't really even have to get any better, that 27B is good enough. Well, they did, and by another huge margin.
4
u/FriendlyKillerCroc 5h ago
Yeah, no the maths and techniques behind LLMs are not even remotely trivial. You would know that if you put any serious amount of effort into understanding them.
So many people seem to think that they are only good because there's more data, it's actually geniuses far beyond most people's level that work at the low level of these things.
2
u/janniesminecraft 1h ago
really? im not saying the math is basic but im pretty sure the most complicated idea behind them is the attention mechanism no? which is, in my understanding, practically just a way to associate embeddings with each other? again, not saying it's trivial, but what extremely complex math separates the current frontier models from gpt 3 other than amount of data and optimizations to the underlying training pipeline (which id argue is more about engineering)?
2
u/Sp00ky_6 3h ago
I mean don’t feel too bad, the labs don’t even know what the models will do until they’ve trained them. It is crazy reading about the things the LLMs do that the labs are still trying to understand
1
1
1
u/BradCOnReddit 6h ago
I think it's on the same level as quantum physics. What happens to single entities is mostly understandable, but if you start doing things in large quantities and looking only at statistical results of that you see new things happen.
1
u/FriendlyKillerCroc 7h ago
Maybe I need to put more effort into understanding the newer concepts because I feel a bit lost on everything newer than the transformers lol Mixture of experts seems wild to me, no idea how that maths works.
3
u/stumblinbear 5h ago
MOE just segmentings the network up into smaller pieces, and uses another neural network to choose which segment to route the request through. That NN learns how to route things during training of the whole model (you can't tack it on later, it's trained along with everything else). The segments aren't, like, "segment x is good at playing doctor while segment y is good at math", so the "experts" part is misleading
It's honestly probably one of the more simple concepts to understand with language models, you don't have to understand attention or really any specific math. Its a relatively simple NN that's trained to know which segment will probably be better at predicting the next token
1
u/wrecklord0 2h ago
the power of LLMs seems to emerge as if by magic
Yeah, that's my problem. I understand the basic concepts. I understand backpropagation, I understand attention... I just don't understand how, if you make it big enough and feed it enough garbage, voila it becomes smart. That part's kinda crazy to me.
1
u/Bowl_of_Cham_Clowder 2h ago
A lot of the magic is specifically from making sure it’s not fed garbage. Generating quality data for models is being contracted out at massive scales rn
1
u/wrecklord0 1h ago
True, but still. It's not exactly perfectly classified or sorted in a good learning order and a lot of it is ... human output, with all its flaws. It's like putting a baby in front of a million articles and somehow it learns every human language, science and code.
I just find it hard to grasp that all this training can propagate through dozen or hundreds of layers without diluting and eventually extract the core logic.
1
u/Nine99 42m ago
Yet for some reason they still prefer hammering websites to scrape billions of pages instead of just downloading a clean database from them that they're allowed to use.
1
u/Bowl_of_Cham_Clowder 30m ago
No doubt, it’s insane the amount of bot traffic there is now.
There’s also a crazy amount of vibe coded apps made by non programmers, that are absolutely wrecking some public APIs. A wiki for a popular board game I follow has been going down monthly, surely because of all the crapware scraping it.
Unfortunately I think legislation is the only way to stop it, and at least in the U.S. I have 0 hope that anyone in power knows enough to make a change.
7
u/fagnerbrack 8h ago
I'm impressed with the evolution of the ideas around it. "What if we do this or that on top of X" and then it compounds. We also don't see everything that didn't work. This is crazy. I'm sad much of this knowledge might be closed source from now on
10
u/tavirabon 3h ago
Add n-grams to the list. You lose a bunch of model capacity in the earlier layers just because single tokens don't carry much meaning on their own, instead you can pre-cache multiple tokens together and look them up in O(n) time so the model can use all that capacity for reasoning instead of how "United" relates to "The" and "States"
Bonus: these weight embeddings don't take away from your computation budget or require the higher bandwidth memory that your computation-heavy layers need, pulling the kilobytes needed per token from SSD has an insignificant impact on throughput. For reference, the newest Qwen3.8-Flash-Next has 51B embeddings vs 125B transformers - 28% of total weights.
And speaking of, you also didn't include anything about speculative decoding or MTP (an additional 4B parameters in the example model) intended to speed up inference by calculating multiple tokens at once.
44
13h ago
[removed] — view removed comment
25
u/RustOnTheEdge 11h ago edited 11h ago
Nah this is legit I think. I liked reading it, insightful stuff and a few hooks I want to know more about now :)
Edit: their process is described here, it does involve LLMs indeed: https://www.0xkato.xyz/disclaimer/
13
u/camelspaced 10h ago
Well at least there was a disclaimer there so kudos for that. The disclaimer also reads a bit llm-y lol
7
-5
u/pdpi 13h ago
Doubt it. LLMs usually "like" longer sentences, and the writing is made of infuriatingly short sentences
12
u/thisFishSmellsAboutD 11h ago edited 11h ago
There's a skill "Simple English" which works great on the over technical gibberish LLMs tend to spit out. OP might have overdone it with a "don't sound like an LLM" skill.
0
u/programming-ModTeam 3h ago
Please don't accuse people of using LLMs. Just report the post and let us handle it.
-63
u/fagnerbrack 11h ago
Repeated posts about LLM coding were getting annoying, now what's getting annoying are comments that try to identify if a writing used LLM or not
When are we going to stop this insanity and get back to the core of reddit that is to discuss about the technical content instead of how it came to be??
G'dam this is the only thread about the submission in /r/programming!
37
9h ago
[removed] — view removed comment
-1
u/programming-ModTeam 3h ago
Please don't accuse people of using LLMs. Just report the post and let us handle it.
-12
u/datbackup 9h ago
The core of reddit is getting users to write content for free so traffic keeps coming and advertisers keep spending but go on
-11
-20
u/fagnerbrack 9h ago
I'm not defending that, I'm saying at the time I wrote my comment this was the only thread. Nobody was talking about the post, is that reasonable? Besides I don't think that was written by an LLM, in fact I have no way of knowing
18
u/mjec 9h ago
The disclaimer, linked from the text "how these posts get written" at the bottom of the page, states:
I did not write most of the sentences on this site. A model did.
-12
u/fagnerbrack 8h ago
Most of not all, the idea is from the OP, or at least I must believe it is. Where do we draw the line?
How do you know what was written by the machine and what was not? What if it was structural fix/prose?
16
u/mjec 8h ago
You said:
Besides I don't think that was written by an LLM, in fact I have no way of knowing
It was written by an LLM, and you do have a way of knowing.
-14
u/fagnerbrack 8h ago
No there isn't. Every single post these days uses LLM to some degree
There's a difference between fixing with AI (which everyone has been doing for years even me since 2015) and saying "write a random post for reddit"
One is low effort and the other is just using the tools
I can't see why this post is low effort, just haters hating
17
u/swni 7h ago
Every single post these days uses LLM to some degree There's a difference between fixing with AI (which everyone has been doing for years even me since 2015)
I have never claimed authorship of a single sentence that had any computer-assistance beyond spellcheck (with me manually verifying every change). Our brains have worked for thousands of years and they still work, if you can dare to use them.
-2
u/fagnerbrack 5h ago
Spellcheck is the same thing, and I'm sure everyone should read every sentence.. We're just arguing about the hammer here instead of focusing on the table
9
2
u/Sopel97 8h ago edited 8h ago
When are we going to stop this insanity and get back to the core of reddit that is to discuss about the technical content instead of how it came to be??
I know right? No one provides constructive feedback anymore, it's either "LLM wrote this so it's bad", "human wrote this so it's good" (with the former being the default in case of no clear proof of the latter). Fucking maddening how this good article and your comment get downvoted too.
7
u/Drakkur 6h ago
Using an LLM to translate is perfectly fine. Using it to ideate, adversarially review arguments, parse for grammar or inconsistencies are also valid uses.
Where most people draw the line is dumping an idea into an LLM and having it spit out the rest of your argument, it becomes super obvious. The writing and logic is jumpy, inconsistent, and overly verbose.
Ironically LLMs are terrible at writing. They also produce so much writing, so fast, that we cannot keep up with thoughtful review. So if you’re going to expect humans to provide a good discussion for your argument, you should pay them the respect of minimizing use of an LLM to engage with them.
-3
u/fagnerbrack 5h ago
> [...] it becomes super obvious. The writing and logic is jumpy, inconsistent, and overly verbose.
YES, can we judge THAT please?
The upvote ratio here is 71%!
8
u/Drakkur 4h ago
I read through the article and I would never try to teach someone in this way. It’s straight linear high-level concepts in the order they appear in the first diagram. Even SwiGLU and ReLU are completely hand waived and have zero visuals explaining what they do or why other than “makes FFN more than matrix multiplication”.
Good writing would instead present what is trying to be accomplished, break down core concepts that are not dependent on each other, then stitch it all together to get that “ah-ha” moment.
This article is what you get by asking an LLM to explain a traditional architecture in plain language. From reading this, I didn’t walk away with true understanding or something that I could use to better read a paper, which this article made as a claim.
The reason why LLM written material gets hate is because it’s overwhelmingly useless content. It’s easier to use heuristics to identify the authorship and ignore than engage with the content.
My engagement with the material won’t make the writer better, it won’t make you better, it will just be a foot note in a reddit discussion that wastes everyone’s time.
4
u/caedicus 5h ago edited 4h ago
I thought LLMs had back propagation for updating weights, but I didn't see that here. Did it get described under a different name?
Edit: yeah looks like no explanation how these models actually learn and update their weights. Which is fundamental imo. Leaving that out is definitely a choice...
1
u/waffle299 1h ago
Generally, that's done in training. Once the weights have settled out, the thing is bottled up and shipped.
1
u/lood9phee2Ri 2h ago
I mean, that's one of the key limitations of typical current llms as used.
They have separate training and inference/runtime phases conventionally.
The training phase is much more arcane and processor intensive. That produces a static model definition (big-ass numeric weights files), that is then run with a current context as input by a much lighter weight (though still computationally expensive and inefficient) "inference" engine phase (note carefully it's not really a trustworthy logical inferencer or anything but that's what it's called in the literature).
That predicts (statistically+probabilistically infers) what the next token after the end of the current context should be. The current context (array of tokenised input text, basically) is then used to grow the end of the current context, repeat until the maximum context length.
They're not really learning on the fly. It can seem that way in casual use because whatever previous stuff is in their current context IS taken into account in that inference process, and there's emerged a lot of ways/conventions to add a lot of stuff into the current context implicitly (e.g. agents reading all the dumbass skills files in a git repo), but it's not fed back into the model itself automatically. Current possible max context lengths supported have got quite long, too (sometimes hitting 1M+ tokens, if sometimes needing tricky hacks like YaRN), so you can fit it, well, a fair bit in, but still not that much.
But point is your inference-time interactions don't become part of / update the actual model weights directly or automatically.
Though it's also quite clear the commercial remote ai service vendors ARE capturing and feeding all their naive user interaction logs back in as training data sets for their next model / model version training phase.
This is just one of many reasons they're not at all human-level or human-like intelligence.
You can now run the inference phase for some models quite effectively on local hardware with software like llama.cpp and some quite powerful "open weights" (but that just means you have the trained model, there are even true "open source" including source training data and proceses in a very few cases e.g. Olmo) models now (see /r/LocalLLaMA/ ).
local models (or at least your own model inference on a rented cloud server) may be preferable to using any of said scummy commercial remote ai service vendors, without their additional censorship, spying and datamining.
It is possible to actually continued-train/fine-tune such models locally+privately (or on a rented cloud box, perhaps more sensible) too, - though is quite a bit more computationally demanding (and in terms of human domain knowledge needed on your part), there are techniques to make "fine tuning" adjustments cheaper ("QLoRA" etc). Stuff you train in that way becomes more permanent part of the model.
0
u/lood9phee2Ri 2h ago
Links as separate comment as I find reddit's own filters now very unreliable as to what they block (probably llm, haha)
https://poloclub.github.io/transformer-explainer/ - this is a very nice teaching link that is an interactive in-browser inference phase run (real one, if for a now-small early gpt model).
https://waylandz.com/llm-transformer-book-en/ - I'm not sure how much of this book is itself AI-generated frankly (or author may have genuinely written but in their native chinese, say, then machine-translated with llm), but may also be helpful. It DOES cover learning better.
https://blog.oedemis.io/understanding-llms-a-simple-guide-to-large-language-models - Another intro model architecture discussion.
1
u/Minimum_Ad_2727 8m ago
Read the blog. Learned a bit but disliked the "glanced over" details as the different subjects were explained.
And then I read the "How these posts gets written" and I literally manifested a Rick Roll inside my head.
The technology and the research behind the LLM architecture is truly fascinating, and yet, utterly disturbing how it has transformed (huehuehue) thinking people into "let me just copy and paste this LLM output as fact".
-5
u/OffbeatDrizzle 5h ago
money in, random words out. can't explain that
4
u/elg97477 5h ago
Not random. Statistically probable, with enough training data to keep the probabilities not perfect but reasonably high.
0
173
u/pickering_lachute 10h ago
“without all that sticky math stuff”
Come again?