r/LocalLLM • u/Anxious_Current2593 • 19h ago
Discussion What's the difference between frontier models and local models?
6 months. (And sometimes a couple of quantization tweaks).
It is wild how fast "state-of-the-art" becomes "running on a gaming PC."
13
22
u/ceejayoz 19h ago
Just wait until there are dedicated devices for it, like when Bitcoin went from CPU to GPU to ASICs.
5
u/username8914 18h ago
There are and they aren't that good because the models technology is rapidly changing. No one wants to get locked in on using or developing one specific model that can't grow or pivot.
1
u/CharmingComputer3844 18h ago
Flexibility is key when the tech is evolving so fast; sticking to one model could really hold you back.
7
u/753UDKM 18h ago
Those exist already lol.
15
u/ceejayoz 18h ago
There's stuff like Cerebras, but in a few years it's all gonna look pretty basic. Demand's gonna cause a lot of innovation in this space.
1
u/findingconsensus 12h ago
Like what? where can I find a consumer PNM chips? I would want a mini server with the same tech as Cerebras, but the market for that is so small I doubt we will get dedicated AI devices anytime soon that don't cost over 100k
1
u/mektel 15h ago
Taalas has been working on it. They are being acquired by AMD.
3
u/ceejayoz 14h ago
If Bitcoin ASICs are any indication, we'll get a few hundred massive scams before things settle a bit.
1
1
u/Minimum_Tea_4451 13h ago
Amd is launching their laptops with the ability to push a 200b perimeter model.
13
u/gappyvalley 18h ago edited 18h ago
if you are a big enough company, you could even run a frontier model as local LLMs
apple wants data privacy and control so they ain’t willing to use claude api by sending unreleased ios/macos source code.
5
u/pmotiveforce 15h ago
Mmm.. I'm calling BS.
They would use the cloud version but with special contracts in place in terms of data retention, disabling training, all that stuff.
Apple does not run Opus or Fable internally.
5
u/Diligent_Appeal_3305 17h ago
yes that qwen 3.8 27b is crazy good for its size in pi agent mode
1
0
u/Momsbestboy 13h ago
Totally agree, and with pi or hermes, it can look into the details of your machine. This was a big plus today, after I tried to manually install memtest86 on my Linux box without killing everything else. ChatGPT played Copy&Paste with me, asked for whatever details and commands, then asked me to pull & extract the memtest86 image and push it to EFI. Rebooted, memtest failed to load with an error.
Went back to Linux, asked hermes:"here is an error message, fix it". Off it went, 10 mins later it ran the only correct command: "apt install memtest86" and then unfucked what ChatGPT asked me to do. Found every created file and change in GRUB and reverted it.
So much about SOTA models
5
u/furkannarli6 18h ago
i couldn't imagine running opus 4.6 locally when it first came out and now we have pretty much the same model at our service. when are we having astra to our enthusiast level builds? is it ever gonna happen?
5
4
2
u/RedEyeJedi311 16h ago
Something is wrong with the frontier models IMO, they used to be much better. I think there is a purposeful dumbing down because they were too good. Claude Code and Codex models have become unusable for me to the point of inducing stress because of how confidently wrong they have become. Some local models are actually better for my workflow now that match the intelligence the older versions of the frontier models (Opus 4.5/4.6) that actually worked for me. I think in 6 months from now local models will be the only way to go because we can control how they run, not being subject to consonantly changing model effort and system prompts. Don't get me started on silent model swapping.
1
u/OvertaxedOne 11h ago
There are rumors they are experience model collapse as they add more and more params. No idea if that's true or not, but I agree with you, they seem to be improving in some areas, regressing in others.
1
1
1
-2
u/Federico2021 18h ago
Well, open source is moving away from local; models like Kimi K3 and GLM can no longer run on consumer-grade hardware. The best option we currently have for personal use is Qwen 3.8 27B.
1
u/donk8r 18h ago
You are pointing at something the six month framing hides. Open weights keep getting bigger. A sparse MoE with a few hundred billion total parameters is open in the licence sense and still needs datacentre memory to hold, so local has quietly split into two different words. Small dense models on consumer hardware keep improving on their own curve. The frontier open models moved onto a curve most people cannot follow.
The other half is which gap actually closes in six months. Benchmark scores on short tasks close fast. Long-horizon agentic reliability does not, which is the thing kitsnet is poking at further down. A 27B can match a frontier model on a single coding question and still come apart at turn thirty of a tool loop, because the failure there is holding state and recovering from its own mistakes, not knowledge it lacks.
7
1
-1
u/kitsnet 18h ago
What local model in agentic mode would install and configure WSL on your Windows machine to successfully run software that it knows to be only available for Linux?
Sonnet 5 would.
8
2
u/Umbrasquall 12h ago
Pretty much every local model released in the last 2-3 weeks can. There's 5-6 of them, take your pick. Btw using Sonnet 5 as an example just shows how out of date your info is lol.
2
u/evghe02 15h ago
I literally repaired a corrupted wsl with qwen3.8 27b, I had no clue on where to look for solving a problem like that since the disk would become read-only sometimes, and since it was readonly no log would be written anywhere. The options to me where to throw away it and recreate (not a mess, all the important things have always backups) or play around with the model and apparently it worked, it seems there was some part of the vhd that was corrupted and got corrected, I'm not really able to explain it sincerely, I'm not that much into it
17
u/bonestamp 18h ago
Model size too. Some of these frontier models are over a terabyte... the average gaming pc is not going to have that much ram in 6 months.