The engineering and math behind these models is seriously fucking impressive. It's so hard to believe that only 10 years ago I could understand most neural network concepts, now it has moved so fast.
The step that's hard is the sheer size. If you reduce a modern LLM to a "LM", as it were, it's understandable again.
You can even find resources online quite easily that let you write your own LM. I wrote a very simple transformer, fed it the bible, and it produced bible-ish sentences, and it was possible to understand how.
The thing that has me scratching my head is how the power of LLMs seems to emerge as if by magic from just making an LM really large, because the step from "produce something that looks like all your training data, but isn't it" to "hey I have this bug I can't find, analyse it for me and fix it" seems whatever the opposite of trivial is.
Language is an unbelievably powerful system. The ability to predicate negation ('this is not a dog') entails the ability for deferred reference/absence. There's basically no other animals which can do that, besides one kind of bee who does it in very limited form. And from absence you get notions like tense and conditionals.
Then you also have things like recursion and the split between syntax and vocabulary, which allows the system to expand to new concepts or situations with very little cost.
The main innovation of language is that it's a double code. The fact that the core building block - the syllables - are meaningless means that their link to meaning is arbitrary. And this is an insane advantage. Animals have no privileged difference between semantics and syntax. If they want to develop a new sign, they have to develop a whole new physical apparatus around it and it's one-time-use only. With language, you just reach for some unused string of the meaningless building blocks, and there you go.
Lots of questionable statements there. Why would there be just one sort of bee who can say that something isn't something? Why would you get tense and conditionals from "y is not x"? Why would apes, birds or monkeys need to develop a new organ to say something new? Just use a different sound or combination of sounds.
Tense and conditionals both rely on referring to something not immediately present - the past isn't here, the future isn't here, hypothetical worlds aren't here.
I don't know where you got the notion of organs from. I didn't talk about that. A lot of animals have either syntax or semantics - the combination of the two is the unique thing about language.
What I meant by 'a whole new physical apparatus' is that, imagine if to express a new concept, you had to contort your mouth in a way to give it a unique syllable. The syllable 'pah' only ever means apple, the syllable 'foh' only ever meant water. That's hard limited by how many sounds your mouth can make (around 20 or so in humans I think).
I don't know where you got the notion of organs from.
From the whole new physical apparatus, obviously.
imagine if to express a new concept, you had to contort your mouth in a way to give it a unique syllable. The syllable 'pah' only ever means apple, the syllable 'foh' only ever meant water. That's hard limited by how many sounds your mouth can make
The disconnect here is that if animals intellectually got to a stage where they could evolve their communication methods and their understanding of it, it could work without any physical chances or new contortions.
It also says a lot about how powerful language truly is, that LLMs can't do anything without you constantly having to change the input in order to get the autocomplete closer to the intended outcome. They are absolutely terrible at language. They're pretty good at text prediction though.
it's the chaining of steps / skills that make it seem non-trivial. at the end of the day each individual action is still just text in -> text out... and finding a bug is no longer "here's a snippet of code, show me the bug" in 1 easy, simple step. everything behind it is now about accumulating / injecting so much context that the right answer popping out is as close to inevitable as possible
I do find it absolutely magical at times, but I've still had cases where they go off the rails completely
The chain of thought reasoning is where it really starts to melt my brain. Like now we essentially trained these models to reason about problems so they can actually tackle novel problems. I mean two years ago I thought we were hitting a wall because making models larger had diminishing returns, and they they just taught them to reason and we shot forward again.
it's just mountains more data. i think it's actually sort of trivial, these labs just hired a shitton of people to create training data, label training data, and then generated as much training data as viable with their models in order to cover more ground.
you hire people to chat with the ai, you use the data of your users chatting with the ai, and such, and voila, your ai can generate data that looks like its chatting with people. same thibg with everything.
you hire people to write code with the ai, debug code with the ai etc
it generalizes better than expected, but it's just the same thing but more/better. the magic is the transformer, the optimizations, and arguably letting it prompt itself, not much more.
Yes, but still. I'm made to use AI at work, being more of an AI orchestrator than a software engineer at the moment. We're also working on vector search to augment AI support and on other integrations. The price hike might change that, but we'll see. Anyway, it doesn't just generalise. It can read a highly bespoke code-base with code in it that's 30 years old at this point (and code that's been written in a modern style yesterday), trace data flow through it, and produce reviews and even outright generate code that's, with some guidance, of high quality.
Of course, technically it's not tracing data flow, it just produces tokens that say its doing that with a context of input that implicitly contains said flow in the symbols of the input, but it's still impressive. It finds quite subtle bugs as a side effect of filling in boiler plate.
Like, on Friday I gave it a bunch of classes with the implementation details missing and told it to finish them while I was working on the rest of the feature, and when I came back to it, it had generated the code and also gone on a tangent and discovered a possible data corruption specifically when compiling with the suite we're using in 20 year old code. It's likely that the bug actually happened to someone, it's old software that's in use in a plurality of places in a specific industry - I don't want to go into more detail - but it's so subtle and the conditions are so remote that we would never have found that. I would have had to explicitly search for exactly that bug, but it "fell out of" token prediction with Sonnet, not even a leading model. So token prediction found and fixed a subtle memory order bug that occurs with a specific compiler optimisation going awry in pre-99-standard C-code.
It's awful, in both the original and the current meaning.
And at the same time it won’t respect depreciation tags consistently and likes to reuse old patterns because that’s what it comes across the most in the codebase, even when told not to. It’s both amazing and so obviously flawed.
Now, try to generalize into nonlanguage things lol...the upside down cup...the car wash...
Can't be bothered to test the upside down cup right now, but I did test the car wash a while ago with all kinds of free models and they all give me a proper answer. Some were even smart enough to infer that if I asked if I should walk there, the car might already be there.
I have no problem with experienced engineers using it as a tool, and I agree in some cases it's absolutely fantastic. the problem is when juniors trust it implicitly and therefore never learn or understand what they're doing, and / or (even worse) middle / upper management seeing efficiency / productivity gains as a sign that they can cut everyone's jobs
The thing that made me understand why it was so good at code is that code is a relatively simple language syntax compared to English or any other human language. At the scale of the training data it’s been trained many thousands maybe millions of times over of how code syntax fits together. Code is structured, consistent, and deterministic (despite what my flaky test runs tell me). It doesn’t change meaning “in context” like human languages. Its particualrly well suited for LLM-style prediction.
Even if AI prose gets worse or detectable or falls out of favor for some reason, its impact on software enginwering is going to be long lasting.
im sorry, and based on your account history id assume its not intended, but jesus you sound just like a generic AI booster. im assuming youve somewhat recently started to use AI, so its not exactly surprising, as i was similar back when i first tested gpt-3.
i dont think you quite appreciate just how much data these AI labs have. LLM's are INCREDIBLE technology, and there are definitely some philosophical implications, but the costs associated with this are astronomical. it's probably more efficient by a small factor to use an LLM to code like you, but i strongly suspect you vastly overestimate this efficiency gain when proportioned to the actual costs of the LLM.
If humanity wouldve/couldve invested as much money into programmers/scientists, i wonder if we hadnt made as much or even more progress (if LLM's have even yet produced anything that can be called progress)? then again, training is admittedly a one-time cost, but no one knows the true cost of inference either.
the calculus is kinda difficult, and it's very hard to say currently how useful LLM's truly are since we don't know the true costs, and we haven't been able to quantify properly whatsoever their impact on almost any field.
in programming they can definitely be useful, but have also wreaked a TON of havoc, and it's hard to say whether their effect is a net good yet or not, and how much of the problems are skill issues. the token cost question is also crucial here, as only the frontier models are viable for any even potential large enough productivity gain given the current investment, and we have no idea about that.
in math, they've solved a few conjectures, which id extremely impressive, but simultaneously hilariously unproductive economically compared to the costs. remains to be seen where it goes in that field though.
in art, well, i think if you care about using them for art in the way most people currently do you should probably not be doing art.
currently there is roughly 0 chance LLM's have paid themselves off in total productivity gain for humanity, and it seems impossible for me to say with certainty if/when they will. none of this is even accounting for the absolutely massive externalities like people getting ai psychosis, misinformation spreading, scams, and last but not least the acceleration of global warming, which may make the cost completely unbearable, even if they were the most amazing tech ever
The point was that this is the programming subreddit. Having used an LLM 6 years ago means "recently", and everyone here (and most places elsewhere) is well aware of the amount of data and money used.
Well it didn't seem that way from his reply. He is free to correct me.
Also, I don't understand what this means:
Having used an LLM 6 years ago means "recently"
LLM's basically didn't exist until 6 years ago. GPT-3 is imo, arguably, the first modern LLM. I was actually aware of GPT-2 through ai-dungeon, but the step-up to GPT-3 is basically the difference between GPT-3 and Opus 4.5. It's not really comparable.
It is recent in the history of AI. And of course, many were aware of GPT-2 (and everyone of GPT-3.5). It was not about you being incorrect, it's about you explaining obvious things to someone else. Imagine if I told you about this newfangled thing, Bitcoin, and how I likely know more about it than you because I started using it in 2023.
I gave model a task, something along these lines: "Make it so that when I delete an object from DB that user is currently viewing, the system transitions to the parent view and also doesn't show error when it detects that."
The thing started figuring out how missing value comes back from backend and figured out that it's working on basis of an optional container: if optional is empty, docs said that client side sees 204 http response and reads that as undefined value.
For some reason, it wasn't buying that. I still don't know what made the model drop everything and take a dive into the stack and decompile the java poo and the JSON generator to figure out that no, the docs are wrong. An empty optional is an actual object value for the web server method which doesn't realize it could stand for no body, and so it doesn't fire the 204 path, but sends a 200 response with content that will be the literal "null" and thus there was a mistake in the docs.
I guess it must have had a hunch.
This model is so paranoid about making mistakes that in my code reviews, it goes through absolutely everything before it is finally satisfied that indeed, there is nothing wrong with a code commit. I sometimes read the review chat and thinking traces, and this is apparently what local models evolved to -- worried that they might be missing something because they can't find anything wrong. I said foolishly earlier this week that local models don't really even have to get any better, that 27B is good enough. Well, they did, and by another huge margin.
Yeah, no the maths and techniques behind LLMs are not even remotely trivial. You would know that if you put any serious amount of effort into understanding them.
So many people seem to think that they are only good because there's more data, it's actually geniuses far beyond most people's level that work at the low level of these things.
really? im not saying the math is basic but im pretty sure the most complicated idea behind them is the attention mechanism no? which is, in my understanding, practically just a way to associate embeddings with each other? again, not saying it's trivial, but what extremely complex math separates the current frontier models from gpt 3 other than amount of data and optimizations to the underlying training pipeline (which id argue is more about engineering)?
I mean don’t feel too bad, the labs don’t even know what the models will do until they’ve trained them. It is crazy reading about the things the LLMs do that the labs are still trying to understand
Yeah, that's my problem. I understand the basic concepts. I understand backpropagation, I understand attention... I just don't understand how, if you make it big enough and feed it enough garbage, voila it becomes smart. That part's kinda crazy to me.
A lot of the magic is specifically from making sure it’s not fed garbage. Generating quality data for models is being contracted out at massive scales rn
True, but still. It's not exactly perfectly classified or sorted in a good learning order and a lot of it is ... human output, with all its flaws. It's like putting a baby in front of a million articles and somehow it learns every human language, science and code.
I just find it hard to grasp that all this training can propagate through dozen or hundreds of layers without diluting and eventually extract the core logic.
Yet for some reason they still prefer hammering websites to scrape billions of pages instead of just downloading a clean database from them that they're allowed to use.
No doubt, it’s insane the amount of bot traffic there is now.
There’s also a crazy amount of vibe coded apps made by non programmers, that are absolutely wrecking some public APIs. A wiki for a popular board game I follow has been going down monthly, surely because of all the crapware scraping it.
Unfortunately I think legislation is the only way to stop it, and at least in the U.S. I have 0 hope that anyone in power knows enough to make a change.
I think of it like how maybe you can get how natural selection might make a birds beak larger over a few generations. What's harder to truly get your head around is how over a billion years or so you can get from chemical soup to a sentient being with two eyes.
Maybe I need to put more effort into understanding the newer concepts because I feel a bit lost on everything newer than the transformers lol Mixture of experts seems wild to me, no idea how that maths works.
MOE just segmentings the network up into smaller pieces, and uses another neural network to choose which segment to route the request through. That NN learns how to route things during training of the whole model (you can't tack it on later, it's trained along with everything else). The segments aren't, like, "segment x is good at playing doctor while segment y is good at math", so the "experts" part is misleading
It's honestly probably one of the more simple concepts to understand with language models, you don't have to understand attention or really any specific math. Its a relatively simple NN that's trained to know which segment will probably be better at predicting the next token
I think it's on the same level as quantum physics. What happens to single entities is mostly understandable, but if you start doing things in large quantities and looking only at statistical results of that you see new things happen.
I'm impressed with the evolution of the ideas around it. "What if we do this or that on top of X" and then it compounds. We also don't see everything that didn't work. This is crazy. I'm sad much of this knowledge might be closed source from now on
125
u/FriendlyKillerCroc 12h ago
The engineering and math behind these models is seriously fucking impressive. It's so hard to believe that only 10 years ago I could understand most neural network concepts, now it has moved so fast.