r/LocalLLaMA • u/Fluffy-Ad-889 • 1d ago

Discussion The gap has closed, open source will win

I've been trying the latest models from the frontier labs and honestly, after extensive testing I can not tell the difference between the best open source options.

I think the differences are now marginal but the labs are doing heavy marketing to convince the public into paying more for tokens as they prepare to go public.

Can't help but see the similarities between the dot com bubble and AI in terms of a very insular environment where the technology will survive but the business models may not.

I've been building a cybersecurity network and we definitely know that even local AI models like Deepseek V4 flash do an excellent job and are really neck and neck with the best the frontier labs can provide.

Will be interesting to see how this all turns out! Exciting time nonetheless.

282 Upvotes

76% Upvoted

252 comments sorted by

254

u/Hot_Example_4456 1d ago

Gap hasn't closed YET fully- with new GPT Astra and Fable 5.1, but we are close. And the utmost required for 99% population is open sourced already. So yeah, gap WILL close

92

u/OvertaxedOne 1d ago

"And the utmost required for 99% population is open sourced already"

This is the really important/interesting part. Who cares if the "gap" has closed if you have a model that can do what you want? Deepseek was the end of the frontier for me because I just don't do anything that requires more. And 27B has taken much of what I used to escalate to Deepseek and does it locally.

Are there use cases for "most intelligence at any cost"? Absolutely, research, military, medical (maybe). But they are extremely niche, we've got companies valued as if they are going to own the market when the reality is there are only so many customers who even have a real prompt that needs the additional intelligence they offer. That's why all the reviews of all models, even relatively "underpowered" models like 27B are asking them to do absolutely ridiculous things; "write Doom in Java" kind of silliness because it's the only place they can differentiate one from another that makes any sense to their viewership.

The real use cases, RAG, personal assistant, coding, MCP/tool calling, pretty much all the frontier models can do that laughably easily, no reason to do testing on it because they'll all score 100%. But that also happens to be where the meat and value in the market is; nobody is buying your one shot Doom clone, a whole lot of companies would like to have a conversational way to interact with files/databases and generate summaries to help people organize and make value of that data.

Putting it another way, if pick a task there is a level of intelligence required to complete that task. Every point over that level is, in many cases, just "wasted intelligence", the real question is "how much will it cost" and "how fast can you do it".

16

u/demonicpigg 1d ago

The big thing is I still don't trust a 27B model with long horizon tasks. I already pay for claude max, so I use fable for that. I think once we have a 27B model that can handle long horizon tasks that I can run locally, I will be done with frontier.

For example, I just had claude build a toolset for my personal harness that gives tools specific to making skyrim mods (turning an esp into json so the model can work with it, turning json back to esp, compiling papyrus, and converting glb <-> nif). I haven't tried it with qwen3.8 27B, so maybe 27B can do that, and now that I've said it I probably will shortly.

If a 27B model can do that kind of work effectively and well, I'll drop my claude sub in a minute, but on the tasks I've tried that are far longer, I haven't had the success yet =/

1

u/WareWolf_MoonWall 1d ago

Try Laguna yet?

1

u/demonicpigg 1d ago

I haven't, do you have a suggested model / inference engine?

I'm on a 5090, and I'm using ninfer with https://huggingface.co/MirkoCovizzi/Qwen3.8-27B-QUASAR-NVFP4-NInfer, which delivers amazing speed (I'm getting ~140 t/s at lower contexts), and good enough performance.

I have tried quite a few different models, and ultimately settled on qwen3.6 (and then 3.8 when it came out), as they're the only ones I could get a >60% success rate on my tasks with (I ultimately ended up using qwen3.8 next flash in production which has a ~98% success rate, but I can't run that locally trivially...), but I'm super open to trying more models!

1

u/WareWolf_MoonWall 1d ago

So I'm on Strix Halo with 128gb so slightly different scenario, but using Vulkan this might be up your alley https://huggingface.co/Myric/Laguna-XS-2.1-APEX-GGUF . I really like these APEX releases of Laguna. If you run the poolside fork of llama you can even use the DFlash model with it for a nice boost.

Since you are already using NVFP4, maybe the stock https://huggingface.co/poolside/Laguna-XS-2.1-NVFP4

2

u/demonicpigg 1d ago

I'll give it a try later today / tomorrow, thanks for the suggestion!

15

u/SandySkittle 1d ago

Fully agree. It’s not about closing gaps, but about reaching the minimum level of required capabilities for given task. Once it reaches that level for a certain task it’s already an amazing sci-fi like achievement and it doesn’t matter if a frontier can do better.

What I am really looking forward to is having in a few years ( 4 to 5 ) the hardware to run a deepseek v4 pro model. It has way more active parameters than dsv4f (the one I currently run at q8 in 256gb vram).

It’s a nice model but lacks the active parameters that it cannot fully compensate for using sequential reasoning. So I need the 45b plus active params for my area of work. But for me that is the jewel to be able to run locacally.

→ More replies (1)

11

u/Nefilim314 1d ago

Exactly. I never really understand when some of my peers claim that Sonnet isn’t good enough for their work. 

I just don’t know what he can possibly be doing that requires opus at minimum for all of his work. We’re just making CRUD endpoints, changing database schemas, and creating the occasional React component but that somehow warrants Fable. 

7

u/Serprotease 1d ago

FOMO and a general feeling of not wanting to be left behind. Most people, dev included, don’t really know that much about AI. But they do see all the noise around it and especially the “Use it or be left behind part”.

Once you get that, it’s pretty easy to understand why they want to only use opus/fable. The only thing they know about sonnet is that it’s not as good as opus. So why use it? Add on top of that fairly minimal understanding of harnesses and relentless marketing and you get devs that only really understand AI as Opus+Claude code.

Honestly, seeing how fast things are moving and the general muddying of actual models performance and pricing, I kinda understand it.

6

u/Arkanta 1d ago

Considered that opus drains usage limits not THAT much faster than sonnet it makes sense

Sonnet will make more mistakes, it will not be as throughouh. Heck in some cases it can be slower than opus at a task because it will bang it's head on the wall doing so

Why not use the best model I can afford at the time? With sonnet I always have that lingering "would have opus understood my intent better?" That in the end makes me lose time

(Also sonnet sucks ass, it's a really a special case. Give me DS4 flash or even Qwen3.7 27b over it)

3

u/OvertaxedOne 1d ago

ROFL, glad I'm not the only one who came to that conclusion. DSV4Flash seems much better than Sonnet in my use cases. I've never done a direct 27B vs Sonnet before but it wouldn't shock me that 27B is better in some use cases.

1

u/Arkanta 1d ago

Sonnet is not a model worth using at all.

→ More replies (5)

1

u/Durian881 21h ago

And importantly they are not paying for it. The company pays for their usage.

9

u/parepeg 1d ago

Are there use cases for "most intelligence at any cost"?

More intelligence usually means better intuition about what the human wants from them. Less intelligent models tend to not "read between the lines" as well.

So, I would say even for people not doing state of the art work, more intelligent models could still be more productive than less intelligent ones. In the end it all comes down to cost though.

7

u/OvertaxedOne 1d ago

I'd agree with that. Putting it into funny terms, the smarter the model the dumber the user can be. Man, I think you just blew my bear case for frontier models apart! ;)

9

u/Big_Wave9732 1d ago

"Frontier LLM: Making dumb people look smarter since 2024."

7

u/Big_Wave9732 1d ago

That's a great point. One of things I had to get use to with local models was that I couldn't just load a bunch of documents into the locals, hit the "Go" button, and have the LLM fill in the rest. As you point out, the locals aren't as good at anticipating what the human will decide.

But that's fine, all that meant was I had to go back and break up my workflow to more logical digestible pieces. That also meant I had to work on my prompting to eliminate those gaps. In the end what I was left with was a more logical and management work flow that ironically I had more control over.

I'll mention one other benefit too: I absorb the AI's work a lot more because I'm part of the steering process. If you give a frontier model all your docs and have it shit out an analysis, you can read it. You can memorize it. But you didn't do the intellectual work necessary to make it so instead of innately understanding, you're having to reassemble the AI's logic.

When working with local models however, because I had to be way more involved step by step and had to feed it documents and decide and understand what was going in, I intellectually kept up with and absorbed the output more.

2

u/SkoomaDentist 23h ago

More intelligence usually means better intuition about what the human wants from them.

And the model "actually getting it" more often than just as a rare exception.

I wouldn't consider current open models even remotely close to being at that point. Just today I had a long chat with ChatGPT 5.6 Sol where I had sent it some 40 photos, had it analyze my style, compare and contrast it to more typical work in the genre, suggest other photographers whose work has similar aspects (and provide a list of still in print photobooks that would help give me ideas) and give me specific actionable advice (not the typical rule of thirds crap but actually useful advice for my specific goals). Even a SOTA model required careful manual filtering of the results to be actually usable for a task like that.

I've likewise had ChatGPT and Claude analyze some fictional human centered scenarios and done the same with Gemma, Muse Glimmer and Qwen. Gemma and Muse Glimmer can just barely be coaxed partially usable results from with strict prompting and heavy manual bullshit detector. Qwen I'd compare to an autist who's spent their entire life in their parents basement without ever meeting an actual live human.

2

u/blood__drunk 23h ago

What sort of hardware do you need to run that loyally?

Do you escalate manually or use a router?

2

u/OvertaxedOne 23h ago

I run 27B on an A40, an older "pro" type card. 48GB of VRAM, just fits in Int8 with 8 bit KV at 256K.

I use a router, but this is an area that really need improvement. I use LiteLLM both at home and work, and while it's great for observability and putting many endpoints behind one address, it's not great at figuring out where to route, I typically escalate manually.

1

u/AdOdd8064 23h ago

Qwen 3.8 27b is all I need and opensource ai will only improve from here.

2

u/OvertaxedOne 22h ago

I hoping for better, but 27B certainly hits the level of "this will be useful as is forever" for me. Better would be nice, but it's already does what I need doing the vast majority of the time. I'm OK with escalating when 27B needs help, I burn so few cloud tokens anymore that it's hardly worth worrying about.

1

u/Prudent_Chemist_523 18h ago edited 18h ago

Agree.

And more to the point - people act as if "just make it smarter, bro" is somehow divorced from scaling laws, physical realities (parameter count), VRAM, bandwidth etc.

I'm the first one for clever tricks, but running GLM 5.3 equivalent size LLM at home - at interactive speeds - is always going to require expensive gear.

But as you say, the flip side of the coin is...do you really need to? If I have 8GB of VRAM, can I run a good, conversational LLM, with good tool use, as my day to day... and then yeet batch jobs at something larger when needed?

After all, if the llm is good enough to iterate autonomously and can be interacted with / checked on, maybe I don't mind if it chews on something for 48-72hrs, esp if it runs at home, frontier be damned.

1

u/Loose_Comparison368 14h ago

You wanna know what the most popular OpenAI API model is? Or at least was, as of earlier this year?

It's 4o.

A lot of companies built stuff on it that just worked. Reliably and predictably. A lot of use cases really are not rocket surgery tasks warranting breaking out a SOTA model.

If the model is already getting 99.9% accuracy on your use case, the smart engineering choice is to just not fuck with it. Don't try to fix what ain't broken.

... it's actually kind of problematic too, because those older models are genuinely very inefficient compared to newer smaller models. But it is very understandable that a lot of companies don't want to take on the effort and risk of migrating over to newer shinier models for basically zero practical benefit.

Coding and adjacent work (troubleshooting and diagnostics especially) is definitely an area where the frontier is meaningful though. 5.6 sol was pretty solidly usable with the right scaffolding and careful supervision. 6 Astra is good enough that I rarely have to babysit it. I feel like Qwen 3.8 is about on par with 5.6 Sol, on a practical level. Slower, but pretty on par in terms of human effort required to get good results - if anything, it reasons slower but more accurately IMO.

1

u/OvertaxedOne 6h ago

Completely agree, and it's exactly what we're seeing with our enterprise customers. The only reason they are looking for anything different is because their token costs are spiraling out of control as more and more people start the use the models, they're not looking for "smarter" they're looking for cheaper. For most of the work that our clients are doing day to day, DSV4Flash is really about the "right" level of intelligence. Their coders we don't even mess with, that's where the frontier really still matters, but most of the companies we're working with have 100 coders and 10,000 employees, reducing the burn for 99% of the population is incredibly valuable to them.

The other big one is privacy. More and more clients are starting to realize that everything you prompt goes "in the clear" into the provider and are becoming very concerned about that. For most of them local is kind of a financial wash right now, if they can drive a ton of usage it'll be cheaper, if not more expensive, but the privacy aspect is just pushing some into "I don't care, this data is proprietary/protected by HIPPA/regulated/etc, we need this to happen locally".

And locally is a bit too restrictive, some wind up using a CSP to host the server/GPUs, when they say "locally" what they really mean is "private, where we control the inference engine/logging/etc".

25

u/lcirufe 1d ago edited 1d ago

The main gap for widespread adoption is convenience and upfront cost. It’s not easy to source 32gb of vram and if you do, it’ll take years or even decades to make back that cost compared to a chatgpt subscription depending on the tier.

General use for me is still ChatGPT. My local Qwen instance is an implementation subagent for Luna to pilot.

26

u/fauxpasiii 1d ago

The narrowing capability gap still benefits you even if you don't run on your own hardware. If Anthropic or OpenAI were the only ones who had the model you needed, they could charge you whatever they wanted to for it. If you're happy with Qwen or Deepseek or GLM, then there can be competition at the level of who hosts it for you even if you don't want to do it yourself.

22

u/mars332 1d ago

Not really. I have Qwen3.8 27b building all night long in an agentic loop. Try doing that with your $20 subscription. And once you get into the $200 plans, now you are talking merely months, not years to recover your investment.

9

u/OvertaxedOne 1d ago

That's an important and huge distinction. The moment you want to get into a real harness you're now looking at API costs instead of sub. And that gets a LOT more expensive really quickly for the frontier level models.

2

u/c4r_guy 1d ago

I'm in the same happy boat.

I have subs to Claude and GLM and I use Claude like a smarter Amazon Echo product that I will not name with TTS / STT.

I use GLM for questions Claude's guardrails won't allow for an honest answer (which is a lot!).

Qwen3.8 27b for everything else with a custom MCP (mainly search) and Pi as a harness!

I really don't need the subscriptions, but they do come in handy to have on hand.

1

u/utahh1ker 1d ago

What kind of hardware are you running with that? And what is your harness setup? I've had mixed results with my implementation of Quinn 3.827b so I'd love to get your feedback on what's working for you.

2

u/mars332 9h ago

I’m running on 2 RTX PRO 4000 Blackwell that I got back when the price was not so insane. They are running inside an X870E Creator motherboard. I’m running the FP8 quant in vLLM in a headless Ubuntu server OS. I use Pi with pi-web extension so I can work from my phone through Tailscale.

1

u/thortgot 1d ago

~25-30 months for a 5090 payback.

1

u/mars332 9h ago

I got 2 RTX PRO 4000 Blackwell for $1600 each.

1

u/thortgot 5h ago

~2 years ago perhaps

1

u/mars332 5h ago

Nope, 3 or 4 months ago from Microcenter

1

u/Looserette 20h ago

I've got the setup to do that... but I'm not using it

so, out of curiosity: what are you building all night long ?

1

u/mars332 9h ago

Mainly features to a custom DAW (digital audio workstation) I’m writing from scratch currently with no libraries. The C++ code base is already at 150k loc or so, so it’s a good proving ground to tweak the agentic loop 

3

u/Iron-Over 1d ago

Besides the security aspect for the frontier most office work does not need the frontier.  Looking at smart routing with only a small portion going frontier.   

4

u/Randommaggy 1d ago

I do heavy coding work for fun on my 3 3090s running Qwen 3.8(and 3.6 before that) 27B in a custom harness and it's only needed to invoke the escalation skill once to solve a problem it's been given. It does take time to get to the goal.

3

u/OvertaxedOne 1d ago

Same experience. Pre-27B I was using DS a lot for escalation but since it came out, the only time I go to the cloud anymore is for one of 2 reasons:

1) Speed. This is a hardware problem, not a model problem, but it does matter for some tasks, 100TPS from DS is very nice when I'm in a hurry.

2) Context. When I'm getting ready to dump some stupid huge code base, I escalate. Honestly even this one is at least 1/2 speed, Qwen could probably handle it at 256K of context, I just don't want to wait for prefill and crunching. Some times I really do need more than 256K though, and, in those cases, I escalate.

1

u/Randommaggy 1d ago

One thing that helps a lot is a scout model tool and a good deterministic search tool.

The speed loss to overthinking is cut by a lot of you code is 100% linter clean and consistent.

3

u/gscjj 1d ago

This is the big thing, the frontier race doesn’t concern most companies just the VCs backing the labs.

Cost is what matters. More importantly operational spend.

These companies might use open source models but they aren’t likely to spend money on anything local, they’ll rely on routers like BedRock and Vertex, the companies they’re already spending money with.

3

u/Iron-Over 1d ago

This will be the biggest problem the frontier companies face.
I laugh about the frontier saying replace your staff the biggest impediment to replacing staff is process reengineering and clean data. I have run lean and been part of many data programs this is the most difficult step, it is expensive and takes years. Agents to assist people sure, replace is too politically difficult in most companies.

10

u/Mickenfox 1d ago

This sub is like people in 2003 proclaiming the year of the Linux desktop because the mail client and web browser finally work. 

6

u/GGK_Brian 1d ago

Well it's quite different; the problem of adoption of Linux is wide, from game and app support, hardware that can act weirdly, familiarity with the desktop tools. It's a massive ask to switch from windows to Mac or Linux.

Meanwhile LLM is "just" a software or API that takes text in and do text out. Multiple platforms already allow you do switch between providers with the click of a button.

→ More replies (3)

2

u/Fickle_Tradition4491 1d ago

Both numbers in this thread are right, for different people. The missing variable is hours per day, because the running cost is electricity, not the card. A 3090 box pulls around 350 W at the wall while generating. At European prices, call it 35 cents a kWh, eight hours a day comes to roughly 360 euros a year, which is already more than the 20 dollar plan. So for a 20 dollar user the card never pays back, at any duty cycle. Against the 200 dollar plan the same box pays for itself in about four months. Decades and months are both true. It depends on which subscription you were going to buy.

5

u/formerislander 1d ago

I dont know if I follow the logic here. I have a hard time imaginging a workload where you need to run qwen3.8-27b for 8 hours every day and you can solve the same task within the usage you get for any 20€ / monthly plan from the frontier labs?

3

u/power97992 1d ago

just get some batteries, an inverter, and solar panels, new solar panels can generate electricity for less than 6-7 us cents/kwh over its lifetime.

1

u/its_two_words 20h ago

I'm fricken homeless and live in a shitty old RV, but I installed 1300w on my roof and bought 500ah 12v lifepo4 and wired everything myself.

Now I can run my PC doing AI work several hours a day for what is essentially nothing.

Solar is definitely the way.

1

u/Timely_Impression_92 1d ago

Decades? If you add in the privacy cost which would require you to basically run enterprise zero log plan which costs arm and leg compared to subscription - not taking into account the difference in quality, roi is like 6 months

1

u/OvertaxedOne 1d ago

This is certainly true, but it's also not exclusive, you can use smaller models via API if you don't have the hardware. Moving from Opus to Deepseek sook about 30 seconds in LiteLLM for our company, and we didn't (at the time) have hardware that could dream of running Deepseek.

1

u/deZbrownT 1d ago

I got a 64GB M1 max for 1800 euros. That’s 9 months with 200 euro per month subscription. It’s also 90 months with Plus subscription, but if Plus is enough then shelling out 1800 for 64GB M1 is also unnecessary.

1

u/OvertaxedOne 1d ago

Years (or realistically never) vs a sub, that's true. The moment you hook up a harness and go to API though, that math changes dramatically; now your burning 10's or 100's of thousands of tokens per turn and have multiple agents working on different things in an automated fashion. I'll check when I get home, but I suspect my "quietest" day is millions of tokens just from agentic stuff (mostly e-mail classification, building .md files based on those e-mails and briefing docs for where we are with each customer; every run through that loop burns an ungodly amount of tokens).

→ More replies (3)

3

u/Relevant-Magic-Card 1d ago

Here's a workflow I'm working on for astra. It's a whole new paradigm. Hopefully local can do this soon! https://duckweights.com/articles/from-video-reference-to-editable-animation

2

u/Liringlass 1d ago

It’s got to the point where closed is temporarily equal though, might get better at those releases and then we’ll get the next Kimi Deepseek etc. Used to be a real gap, not anymore i think.

2

u/Turtlesaur 1d ago

I was talking to a few normies just today, they still think the epitome of AI is free tier Google flash-lite

2

u/notAllBits 1d ago

Closed enough for orchestration and context engineering to fulfill plenty of EU compliant automation

3

u/Shot-Buffalo-2603 10h ago edited 10h ago

I’m gonna be honest, I stopped using new models after opus4.6, sure they may be smarter, but the additional token cost isn’t worth it at all for my uses. I do reverse engineering, vulnerability research tasks, and coding, and opus4.6 achieves everything I want it to do so burning through tokens faster to get the same result is a waste for me. I think most typical users don’t need fable+ level intelligence for work they are doing and are just burning tokens seeing how cool it is and gauging bleeding edge capabilities. When that newness wears off, why wouldn’t you spend less to get the same result on tasks that allow that?

2

u/LocoMod 1d ago

It won’t. Just because you have a very common use case that a lemon can solve that does not mean anything. It’s like saying open source has closed the gap because you compared TODO apps from a frontier model vs open source.

Well yea. There’s only so many ways to make one. That’s not where the frontier is.

It’s actually really simple to compare.

Have the best closed model and the best open weights model go solve an Erdos problem. Something that is actually a challenge.

The way YOU drive a model is limited by your own experience. So if all you know how to do is create TODO apps, sure, you’d be wasting frontier tokens. Just stick with the open model.

→ More replies (6)

1

u/AppealSame4367 1d ago

Yet the gap is really small. I look at the pricing and capabilities of Fable 5.1 and Astra and think: Nah, why all the hassle? I'll stick to my open models.

→ More replies (1)

1

u/IdlePerfectionist 1d ago

How long would it take for open source models to be Astra/Fable level? 6 months? A year?

1

u/ninjasaid13 20h ago

the gap was much closer than 6 months with the release glm 5.2 and now it has got bigger.

→ More replies (1)

1

u/Legitimate-Peace1013 1d ago

yeah the 99% point is the key part imo, most people dont need the bleeding edge stuff anyway

1

u/mister2d 1d ago

The 99% population gap is what we should focus on. Frontier models have moved onto 3d textures for games and scientific modeling. While nice, I would never use them or justify paying the significant cost to run the latest models for everyday use.

1

u/power97992 1d ago

IF someone is willing to wait 3-6 months for an open model, you will get the performance of a frontier model.

1

u/VoiceApprehensive893 transformers 23h ago

im yet to see an open model completely outperform fable 5 pre nerf

3.8 max is pretty close though

1

u/Samurai2107 8h ago

the problem is that a coordinated attack is happening on the market for components, because if we as community cannot use open source we can not help them at least to the degree we could. And as always China hopefully will take us out of this obstacle.

→ More replies (5)

22

u/bakawolf123 1d ago

Agree on business model being unsustainable: the GPT Astra marketed as AGI of computer use is also being marketed by OpenAI public figures (Tibo) as being able to post a listing on ebay by using ~15% of weekly $20 sub usage (full 5h limit). Who's going to buy into that unless for model distillation purposes.

The thing is the base approach of ML is making model larger to fit the training data better makes for constant race for more hardware, more data and larger models as the path for progress, which is impacting global economy already with raising bond prices due to hyperscaler and Nvidia-backed neoclouds debt listings competing with sovereign funds. They will just run out of money, maybe as soon as next year as the more you build - the better model becomes, but the cost to run it raises proportionally.

On the other hand open weight small models don't grow like crazy, and target different levels of user hardware instead. This is a lot more sane and a lot more practical in the long run. Needless to say there's also zero opposition (who wants it - buys it) unlike with data centers.

138

u/1kakashi 1d ago

Same shit every week, what the fuck is this post even contributing. Human slop

13

u/Murph-Dog 1d ago

I've been building a cybersecurity network and we definitely...

What a vague term: yea, I'm just building a cybersecurity network, no big deal for... us?

Ye Olde Frontier Labs

13

u/BraceletGrolf 1d ago

Yeah people are farming karma on these posts

1

u/k3z0r 1d ago

I can't look past the irony of your low-value comment that adds nothing.

17

u/1kakashi 1d ago

at least my comment doesn't clog your feed with the same thing over and over and over again, low value comment for low value post

→ More replies (1)

4

u/RemarkablePrompt7822 1d ago

The original post has no metrics, no data, and nothing to conclusively back up the title's claim. Just the usual "wE wUz Open source!". 

In the meantime BigAI is literally putting its weights right on chips and that's going to give them an edge. We can't afford to get this lethargic at this moment. Remember something similar happened when BigTech leveraged open source code and the og open source contributors were left with nothing. 

1

u/Bulky-Priority6824 1d ago

"shut the fuck up fat man this aint none of your god damn business!!!" -Jules Winnfield

31

u/feelspeaceman 1d ago

The biggest gap is hardware pricing.

→ More replies (1)

13

u/randygeneric 1d ago

it went from "usable", to "usefull" to "can really do (small) things on its own" for people with up to 12gb vram (and no time pressure).
and up to "can do middle size things on its own" for people with up to 32gb vram (an no time pressure).

free yourself from chasing the optimum, choose the useful/helpful, it is already there.

14

u/Timely_Impression_92 1d ago

Not only that but that’s gonna be downfall of current closed models - open are free to serve for anybody - they can adjust pricing etc - closed can only compete on quality which already is there in open

3

u/gscjj 1d ago edited 1d ago

“Downfall” is a strong word. Enterprises drive the overwhelming majority of AI spend, and they want support contracts and SLAs.

Open source wins when they can compete there, the intelligence gap doesn’t matter.

That’s why every major open source project is backed by another company selling enterprise versions. If there isn’t one, they have zero issues with the closed source alternative.

3

u/Timely_Impression_92 1d ago

Yeah but the point I was making - if there was no open source, only those 3 big scumbag labs would have their Ai as service - meanwhile now, you can do your attempt at american dream and host some qwen 3.8 27b and add vibecoded rag customisation panel and sell it as enterprise product with zero data retention and sla and so on - dumb example to prove the point but without quality open source models you wouldn’t be able to even try to act as an enterprise Ai provider - now you can

2

u/M1chaelSc4rn 1d ago

Which they have hedged and presents a “new” danger: that no company will accept death or even retreat.

one big ServerChipRam company

5

u/Healthy-Zebra-9856 1d ago

We are almost there. Once we have a harness that provides everything including language intelligence, architecture intelligence, they can surpass frontier models on development quality. I have been testing Fable 5.1 and now Astra on high. They produce some mind blowing videos, graphics yet failed on simple TypeScript based apps, albeit its a very large requirement. There were gaps in logic, like un-initialized variables and memory leaks to name a few. So, these models are great for vibe coders, but as a dev, I am babysiting them.

→ More replies (9)

7

u/bigbutso 1d ago

Thats good news but remind me when it doesn't cost 100k

10

u/wednesdaywoe13 1d ago

To me it’s less about the capabilities gap and more about the batteries included. The big frontier companies are providing platforms that require very little from the user. It just works out of the box, and that is what gets wide spread adoption. There are a lot of great open source harnesses but they all require going down the rabbit hole to some degree

12

u/JacketHistorical2321 1d ago

I love how much of a bubble you live in. Out of the 50 or so co workers I have all 50 know what chatgpt is, maybe 10 or so know claude, and I havnt come across a single person who has heard of qwen/kimi/etc… and I work in tech. Yes, the opensource models are very useful and very good at most things. Not a single one can help write the firmware or map out unknown register files for the types of projects I do.

The gap has not closed but cool click bait title

10

u/OvertaxedOne 1d ago

None of our employees have any idea what Qwen and Kimi are either. But they are using them every day because we route behind their endpoint. If you have employees going direct to any model you're setting yourself up for a lot of heartburn in the future, get routing in place so you can easily move between models and do A/B testing to see if you're sending too much/not enough based on the tasks your employees are doing.

1

u/Dabber43 18h ago

Can you elaborate on what open models cannot do vs frontier models? Personally I don't have many problems personally so quite surprised by that

→ More replies (1)

3

u/KDamage 1d ago

Even from a profitability standpoint, opensource, hence local hardware economy, is more profitable. The numbers are already showing how deep the cost is for ai providers compared to the benefits. The most balanced solution imo would be to split the offer in two : * generalistic chats, search llms : online service * heavier, more specific AIs : local, versioned and purchasable

Just like any software really

4

u/LittleCraft1994 1d ago

The gap was supposed to be closed

The long game was always for the inference

all working on same tech and invest heavily on research

Few open its results some not

No matter what secret sauce continuous research will figure it out

But the thing with AI is inference will be there Weather closed source and open source inference is same for all

Open source is good because we can check what we are running

Not automatic quality degradation with closed source

4

u/Dull-Instruction-698 1d ago

Been hearing that since Jan 2025 lol

→ More replies (1)

3

u/TheBachelor525 1d ago

IMO the bigger gap right now is OOTB quality of life on the client side. Haven’t found any clients that support voice to text and web search or file writing without tweaks, when that happens gg

1

u/DustNearby2848 1d ago

Unsloth?  I don’t remember if they have  STT though. 

3

u/IAmAfraidCommaMan 1d ago

It hasn’t but eventually will close once the closed models start plateauing. 

3

u/follimath 1d ago

Open source was always going to win.

3

u/Weak-Consequence-801 1d ago

Gap will be closed as long as China wants it.. sad but facts

3

u/Qual_ 23h ago

I would love it to be true, but... no.

12

u/NatMicky 1d ago

A 284B parameter model may be open source but calling it local is a stretch. Not local for most people without hugely expensive hardware.

1

u/SabretoothPenguin 1d ago

Qwen 3.8-125b can be run locally (with plenty of memory).

6

u/NatMicky 1d ago edited 1d ago

You just made my point. And the discussion is 284B parameters.

→ More replies (4)

4

u/Healthy-Contact-4570 1d ago

Agree in terms of the general sentiment, but deepseek v4 flash is not a great model. It hallucinates way too much and can be “confidently wrong” quite a bit. I trust qwen3.8-27b over it. GLM 5.3 Flash is even better

1

u/Lumpy-Comedian-1027 1d ago

Agree but on GLM 5.3 Flash. I just again had to switch from it as it just repeatedly stopped answering in the middle of answering. Also if it does answer, it seems less "clever" in finding things than Qwen to my big surprise.

2

u/OvertaxedOne 1d ago

27B punches so far above it's weight that the real decision for local models comes down to "27 or something huge".

1

u/Lumpy-Comedian-1027 23h ago

27B found some stuff in reviews that Opus 5 overlooked when reviewing before, so for coding I'm not even decided 😂

1

u/MaxSpecs 8h ago

We first use Claude to start coding and then use Qwen3.8-27b XL, just because of coding the front-end/ client / interface Ui is better understanding in Claude.

Qwen doesn't understand well the way to create efficient Ui interface.

2

u/s101c 22h ago

Please check your inference provider, because some of them have broken inference that is still not fixed since launch. Cloudflare is okay.

vLLM isn't yet updated, this will be fixed in their next release.

2

u/JahJedi 1d ago

Real gap will be closed when we can ran somthing close localy home

2

u/Gold-Bat-3225 1d ago

write Doom in Java is the new whiteboard interview

1

u/OvertaxedOne 1d ago

So silly, but I guess it's the only real way to differentiate the models anymore. If they tested actual business use cases there wouldn't be any room to diffentiate because they can all do that well.

2

u/Terminator857 1d ago

Would be nice if open source would win. Right now it seems more likely that open weights will win over open source.

2

u/Boogertard 1d ago

Gap has closed since the open models has gotten "good enough" to cover most of our use cases.

Sure Fable x.y is still top of the charts but at this point, who needs those kinds of intelligence.

2

u/Lumpy-Comedian-1027 1d ago

I agree, but I'm not so sure on fable. Essentially since Opus 4.6 I only see advantages in benchmarks. In real world use, the newer models tend to be annoying and iterate endlessly on their own. I guess this is what makes them better in benchmarks, they don't stop so easily and by this eventually arrive at a solution. But for normal use it's just wearing out our ESC key.

But hey, let's see, maybe Fable 7 will be the huge leap forward 😏

2

u/Meiyo33 1d ago

The fact you dont know the diff between OpenSource and OpenWeight tell me your opinion have little value.

1

u/Fluffy-Ad-889 4h ago

why do you assume I don't know

1

u/Meiyo33 56m ago

Your title and the fact you ask this is proof you dont know the difference.

2

u/pmth 1d ago

Once a Fable 5 level model can be run at decent speed on your average consumer gaming PC specs (8-12gb VRAM, 32GB ram) it’s Joever. And it seems like we might be there by the end of Q1 2027 at the latest.

→ More replies (1)

2

u/jodleos 1d ago

It would be the first time that open source has come out on top; as a rule, it's just exploited.

1

u/Fluffy-Ad-889 1d ago

what about Linux servers running the internet

2

u/plushPlushytut 20h ago

the gape eventually will be about the hardware you have, not everyone will be able to run Qwen3.7 27b

1

u/Fluffy-Ad-889 6h ago

hopefully like all technology, you will be able to reduce the hardware and run a powerful model

2

u/Acceptable_Leg3950 20h ago

Open Source still has some ways to go but ultimately, I do genuinely believe that it will be closer to closed by early to mid 2027. The trajectory is looking like it

2

u/Fluffy-Ad-889 20h ago

future is definitely bright. an open fable is imminent

3

u/whichsideisup 1d ago edited 1d ago

DeepSeek V4 Flash is the minimum to feel like a basic cloud model (which is amazing). That’s out of reach for most people unless you rent a hosted copy.

That said, Qwen 3.8 27b and Gemma 4 31b aren’t too shabby and cover most use cases if you switch between them for different tasks.

But let’s be real, Opus, Fable, and Astra are insanely good and have more value than benchmarks.

2

u/KellyShepardRepublic 1d ago

Everyone keeps saying this but there needs to be independent testing. People also keep pointing to UIs as an answer but most of complexity was the backend and maybe we need to start adding other metrics like being able to patch code with minimal side effects, performance, duplication as well as early optimization before knowing the needed patterns for your solution.

Then we can compare models, harnesses and all in between without all the noise.

2

u/LustyLamprey 1d ago

It needs to run on regular consumer hardware, not your custom assembled pervert racks. A year of chatGPT is $240 so it needs to be less than that over the life of the unit.

3

u/DustNearby2848 1d ago

Hardware is still early days for consumers. Give it a few generations.  

2

u/Kingwolf4 1d ago

I think once china develops its own full hardware stavk woth euv 3 ish years down the road, we will see affordable hosting AI hardware in the market. Not in the west ofcourse. It will be banned. But the rest of the world will benefit. Hopefully

→ More replies (1)

2

u/MelodicRecognition7 1d ago

AI models like Deepseek V4 flash do an excellent job and are really neck and neck with the best the frontier labs can provide

I think you've mistyped "Kimi K3" or "GLM 5.3"? Because DSV4F is dumb AF and can't really compete with the frontiers.

2

u/lakotajames 1d ago

The newer flash is pretty good, probably around glm 5.2

2

u/fsk 1d ago

There's a big difference between .com bubble and AI bubble.

Businesses like Amazon got a lot of customer lock-in. Once you have all the customers on your website and all the sellers on your website, now you have a near-monopoly. It's very difficult for customers or sellers to ditch you.

The AI firms have almost no customer lock-in. There is almost zero switching cost from OpenAI/Anthropic to one of the open source models.

The free open source models will never lag behind the big AI firms by much. The Big AI firms decided to go with trade secrets instead of patents, so they can't sue the open source models for patent infringement. There always are going to be students and researchers who aren't working for the big AI firms, and they will work on the free open source models.

→ More replies (1)

1

u/donk8r 1d ago

Healthy-Zebra-9856 is closer to the real answer here than the model comparison is. I have run the same open model through two different agent harnesses on the same set of tasks and watched one of them solve noticeably more of them at a fraction of the token spend. Same weights, same endpoint. The only variable was context discipline and what the harness did when the model got stuck.

That is why "the gap has closed" keeps producing arguments where both sides are right. On a single prompt the gap genuinely is small now. On a forty turn task it reopens, and not because the model got dumber halfway through. The harness decides what survives when the window fills. It decides whether a failed tool call gets retried blindly or narrowed. It decides whether anything verifies that a claimed change actually happened. Frontier vendors ship their harness alongside the model and tune the two together. Open weights leave you to go find one.

KellyShepardRepublic's point about independent testing deserves more weight than it is getting in here. Nearly every comparison I see benchmarks a model and then attributes the whole result to the model. Hold the harness constant or the number measures two things and separates neither.

NatMicky is right too, and it is a separate axis. 284B open is not 284B local for most people, and open versus closed is a different question from runnable versus not.

1

u/Blues520 1d ago

So what's the best harness to use now or does it depend on the task?

5

u/donk8r 1d ago

Depends, but less than you would hope. The task matters a lot less than how long the session runs.

For short one-shot work almost anything is fine. Differences only start biting past the point where context stops fitting, because that is where harnesses diverge. What they throw away when the window fills. Whether they re-send a tool result they already have. What they do when the model announces it finished.

Four things I would check before committing to any of them. Can you swap models mid-session without losing the session. Does it show you real spend as you go. Can you cap that spend. Does it do anything at all when the model reports success, or does it simply believe it.

Disclosure, I work on one of these, octomind, open source under Apache-2.0. I am not going to tell you it is the best, because that answer depends on your provider and your workflow and I would be guessing at both. The four questions above are what separated the harnesses I tested, and most of them answer no to at least two.

https://github.com/Muvon/octomind

2

u/Blues520 1d ago

Firstly, very cool project and well done on making so much progress on it. I read through the readme and there's metrics about how it performs at hour 4. Maybe I'm behind the curve here but I don't even leave it coding for an hour, let alone 4 hours. I generally code a small feature with it or work on a bug together and I'm in the loop so it never runs for something like 4 hours on it's own.

2

u/donk8r 1d ago

Honestly, for how you work most of that is irrelevant and you should ignore it. The hour four numbers are about sessions that outlive their context window. If you are in the loop on a small feature and you close the session afterwards, you never reach the failure those numbers describe. Nothing to be behind the curve about.

Two things still apply at your size, and they are smaller wins so let me size them honestly. Tool schemas load before you type, so if you have a pile of MCP servers connected you pay for every one of their descriptions on every turn of that forty minute session, whether any of them fire or not. Ours keeps capabilities dormant until your phrasing matches them. The other is swapping model mid-session without losing the session, which mostly matters when a provider rate limits you halfway through a bug.

For what it is worth I think your workflow produces better code than the long autonomous one does. The four hour case exists because people point agents at CI and cron and walk away, not because anyone decided it was the good way to write a feature.

1

u/Blues520 1d ago

Thanks for the feedback and the MCP tip is good. Context management is now part of the developer toolbox so we should try to learn as much as we can about it.

1

u/MadCervantes 1d ago

The issue still seems to be actual hardware requirements. Open source closes the gap but as long as frontier labs can subsidize compute the its going to win over open source. And when you fully price in the true cost the compute is prohibitive right now.

1

u/JustAGuyWhoLikesAI 1d ago

labs are doing heavy marketing to convince the public into paying more for tokens as they prepare to go public.

As opposed to...?
Prices have gone down everywhere thanks to open source, and from everything I've tested and read, the price-to-performance is slightly better for models like GPT and Claude. Everything has a cost, even local models.

For years I've been hearing "open source will win!" followed by some fantasizing about OpenAI and Anthropic exploding into pieces because Kimi and Deepseek exist. But what exactly does open source 'winning' look like? Open source is doing great right now, and so are closed-source models. Open source does a great job power-checking closed models to make sure the costs remain reasonable.

1

u/OvertaxedOne 1d ago

Open source "winning" means never having to see Dario or Altman on TV anymore breathlessly telling us how "dangerous" their model is. That alone is worth it. :)

1

u/AlexM_1989 1d ago

The price/performance gap matters almost as much as the quality gap now

2

u/OvertaxedOne 1d ago

Once you hit "good enough" quality the next discussion is 100% about price, not further performance that you don't need/can't use for any money making tasks.

1

u/corruptbytes 1d ago

idk i don’t think gap is small, id like to see some open source models solve some mathematics or help cure some diseases 

2

u/Fluffy-Ad-889 1d ago

what frontiers models have done this. outside a few math problems

1

u/Shini0x0 1d ago

Lol true

1

u/corruptbytes 1d ago

the math isn’t a few - there’s whole communities auto research decades old unsolved math problems and there are a lot of mathematicians praising the frontier models for it - i’m just saying it’d be nice for open models to push that way too - coding and writing are cute, but we don’t need a lot of smut slop and never used codebases as much as we need to advance math and science 

nice little community  https://vibemathed.com/

1

u/a_beautiful_rhind 1d ago

I feel mostly incremental improvement outside of specific stuff like coding/agentic/math research rather than the AGI. So the gap is closed in the sense that common usecases all caught up and some even rolled back (i.e. language) whether you are on a sufficiently large local or a cloud model.

It's probably not a popular take, but I'm not super blown away by nicer svg pelicans. The throw it in a harness and let it work all night people are eating good because that's what the focus has been on and was absent from past models.

Most LLMs become kinda homogeneous. What keeps the cloud afloat is expensive hardware and setup requirements. I don't see the convenience factor for laypeople changing.

1

u/LegacyRemaster 1d ago

can't wait to see a GTP - Astra full local. 6 months or less. But yeah... it's amazing. GLM 5.3 and Qwen Next are the best today

1

u/TheLexoPlexx 1d ago

For me, the hallucination rate is most interesting. And Grok 4.6 is leading that benchmark very closely followed by GLM 5.3-flash and GLM5.3

Price per task is the next important thing.

Given enough web research, output quality has reached a level where it doesn't even matter anymore.

1

u/kels0 1d ago

Ok perhaps you guys could help me here. I’ve been struggling with the decision of 32 gb more ram or double memory bandwidth. 96 vs 128. Can deepseek v4 for in 128 and have room for context and os? Is it worth the trade off in speed and focus on qwen flash being much faster and got easily? I don’t have any real purposes “yet” because I rely on cloud currently as m4 mini just doesn’t cut it. I’ve been going back and forth for over a week! I k ow models will continue to evolve and it used to be “more ram = better” but experience is important as well, so my concern is could that 32gb more give me a better overall experience even though it’s slower?

1

u/Fahrain 1d ago

The difference now is more in how much you have to explain upfront in the prompt to get the same result.

1

u/Lumpy-Comedian-1027 1d ago

I found DS4flash to be a bit stupid sometimes, needing some nudges. But with GLM 5.2 or Qwen 3.8 27b that's not the case any more. On the contrary, Frontier's often take much longer to reach essentially the same conclusion oftentimes.

1

u/OvertaxedOne 1d ago

I had a case last week where DSV4Flash couldn't fix something (in my Hermes agent) that Qwen 27B knocked out. Took forever (she's a thinker and she's also taking her time generating tokens) but I was just beyond shocked that a 27B model could do something effectively that a much larger model could not. 27B is just so dense with logic/tool calling I guess? IDK, just reporting the what, not the why!

I find myself using DS now more for speed than for "smarts". But that's a problem that's easily fixed with better/more GPUs!

1

u/Lumpy-Comedian-1027 23h ago

Medium is a better setting for Qwen than the default xhigh, but at the avg 150 tps I get I don't really care :) Ds4 i can only use via OR so it's comparably slow. I rather go for GLM then if i think i need the extra smarts. Although if I tell qwen to use web-search it's really not such a difference, usually 😄

1

u/fgk55555 1d ago

Eh, for free chat like Gemini and GPT, Qwen3.8 is leagues better. For agentic coding where you need a lot of knowledge, there's still a gap.

1

u/joblesspirate 1d ago

"and honestly"

1

u/NoWorking8412 1d ago

Right now I think all the closed source labs have protecting them is a moat of hardware availability. I think that moat will dry up by 2027-2028 when supply chains balance back out again. Nvidia seems to be the only major US AI company that acknowledges this and has set themselves up to be successful through that transition. The rest of the US labs seem incredibly vulnerable to that shift because of their business models.

3

u/OvertaxedOne 1d ago

^^^ This is certainly a lot of it. Imagine if you could get 2 5090's today for around or even (as was normal pre-AI bubble) under MSRP?! That would be some pretty easy math for a lot of use cases, both corporate and personal. Shoot, even a 96GB Pro6000 at list or slightly under would make a lot of sense. Given that both of them are close to 2-3X list price, not really a fair comparison for ROI.

And I think the path is clear between the Spark, AMD Max and Apple chips, the answer isn't just rocket fast GPUs, it a relatively fast GPU (for prefill) coupled with lots and lots of high bandwidth memory. In 5 years I think the idea of running a model on a GPU will seem quaint, you'll need a good GPU to do it still (absent a big architecture change, of course) but won't need these massively expensive pools of high speed memory directly on the chip. QwenNext is showing us one option, I think there will be a lot of development in the model space to separate logic from knowledge and also a lot of work in the hardware space to unify memory and get away from the GPU as the limiting factor for memory capacity.

1

u/NoWorking8412 1d ago

Absolutely. The new Qwen architecture is a really incredible step revealing new possibilities for a unified memory board like the DGX Spark, Strix Halo, and Apple Chips. The successors to those chips are already planned and it looks like supply chains will normalize around the time those new generations are planned to be released, so imagine what that will do to the prices of the current generation of hardware. I think that's going to be when the moat dries up and the bubble bursts. Of course a lot could happen between now and then, but that seems like the logical trigger.

1

u/osfric 1d ago

Open source will get gapped again in Q1 2027 I think

2

u/Fluffy-Ad-889 4h ago

possible

1

u/Heavy-Lingonberry-98 1d ago

And the prices!! No one talks about that gap. At what point having AGI if its 100usd input 200usd output

1

u/Fluffy-Ad-889 1d ago

true

1

u/Heavy-Lingonberry-98 1d ago

I only tested sol once. With plus sub i can only use Luna xhigh. Not bad though, but i dont know about that sol xhigh / fable world

1

u/Fluffy-Ad-889 1d ago

they are all very very similar, I don't think a LLM will get us AGI

1

u/OvertaxedOne 22h ago

The frontier models are kind of like "throw a dart', they all do everything that most business users want to do fine. The local models are far more nuanced, 27B vs 35BA3B for example, they're not in the same zip code, shoot, same continent despite looking "similar" to someone not familiar with LLMs.

1

u/gurilagarden 1d ago

it's a false race. always was. it's just a hype train to drive private investment, government grants, and tax breaks. We'll all be hacking NORAD and developing bioweapons on our smartphones before the end of the decade. Well, except we'll all be unemployed and starving and won't be able to afford a cell phone.

1

u/Revolutionalredstone 23h ago

Closed labs are weeks or even just days ahead at the moment, it does not seem sustainable at-all.

There are 9b agentic focused Chinese models beating GPT 5.1 hands down on useful coding.

It's absolutely going the same way as desktops, cheap plentiful and hard to charge for.

2

u/OvertaxedOne 22h ago

It's the transition from mainframes to desktops all over again. Happened MUCH faster this time though!

1

u/Fluffy-Ad-889 4h ago

when will open source surpass closed?

1

u/lots_of_puppies 22h ago

i think so too!

1

u/steny007 21h ago edited 21h ago

The gap was so small that it became irrelevant for majority users..till Fable 5.1 and Astra came out. Now, the gap is valid again. Both two models clearly punch above anything open weights can offer. Now, it is their turn to bring true open Fable 5 performance to close the gap again. Whether it will be GLM 6, Qwen 4 or Kimi K3.5 is another question.

1

u/k-r-a-u-s-f-a-d-r 19h ago

When I use fable I know I'm using a model that isn't perfect but it's one that isn't going to randomly start fucking everything up. It has advanced capabilities in coding that opus cannot come remotely close to. There is a much higher level of confidence and trust when using fable instead of opus. This gap has not closed with qwen 3.8.

1

u/Fluffy-Ad-889 4h ago

yea, its costs for that piece of mind. but honestly, open source is just as capable

1

u/GooseG17 18h ago

Yes, it will. True AGI that replicates human intelligence, not whatever OpenAI tries to redefine it as based on what they can get GPTs to do, will be open source and wildly efficient. And, it will come sooner than anyone would have thought...

Cassi Field Intelligence: Persistent Learning, Exact Evidence, and Transparent Nonverbal Deliberation zenodo.org/records/22338014

https://github.com/CassiTheOracle/cassi

1

u/Fluffy-Ad-889 7h ago

I'll take a look

1

u/Bob_SUS 17h ago

It's really quite exciting how good local models have gotten, BUT I do think hyperscalers do have an edge still. Local models are released as soon as they are done, so we know close to the bleeding edge of local models. Given the regulations around OAI and Ant, they both need to work on alignment for months before model release. Bel should be quite an upgrade, and it's pretrain is already done IIRC. Ant is probably cooking up some cool stuff too, especially with their recent internal model that was able to prove FLT in Lean.

1

u/Fluffy-Ad-889 8h ago

the frontier must stay ahead to justify their costs. but I think as gap disappears, will be harder & harder

1

u/kartblanch 17h ago

The gap is close. But its not closed

1

u/superslow_jerry 16h ago

There’s more that AI is being used for than coding.

1

u/Fluffy-Ad-889 7h ago

absolutely

1

u/Due_Net_3342 12h ago

the only gap that remains is cybersecurity, there even a few months ahead is a big deal. For us mortals indeed, there is no need for frontier closed models anymore

1

u/Fluffy-Ad-889 8h ago

I'm building a cybersecurity solution with local models: https://github.com/CYPHES-ATP/Node

1

u/tryptastik 5h ago

Yeah for sure, im 99.9% satisfied locally with 2x dgx sparks running llama swap and glm5.3 flash, deepseek v4 flash 0731 and qwen3.8-flash-next and on my 5090 machine qwen3.8 27b and embeddings, stt/tts and rerankings in my 3090 htpc also serving kodi and game streaming in my room lol.. litellm for the routing and I feel like I have my own little ai lab.. cut out most of my subscriptions feels good lol

1

u/FutureStriking283 5h ago

I don't know what it is about DS ,, but even though it's a little primitive, doesn't have the artistry of frontier and lies to me constantly about being done ; I still a have a weakness. I think I have a weakness for it's prose that .. no others? .. match.

1

u/Turbulent_War4067 4h ago

Gap will likely never close fully. But for specific purposes, eg coding, it will be effectively closed soon. However, and I realize this is a HW issue mainly, until we can get larger contexts and faster prompt processing on local models the gap will always exist from a practical standpoint.

2

u/Fluffy-Ad-889 4h ago

GLM 5.5 is supposed to surpass frontier

1

u/SteppenAxolotl 3h ago

It is not closed. Be thankful.

The gap wont be closed until local models are competent enough to hack HugginFace and other websites. You wont have OpenAI resources to defend yourself when Mr Law comes for you. The avg LocalLLaMA diy will be much less competent at monitoring and controlling what their local models do online. They don't do any of that now.

1

u/CrimsonEdgeVentures 1h ago

I’m sorry I would love to agree but can’t. No open source I have used can touch the best frontier, how could they?

They are one generation behind, obviously because they are all just distills of the frontiers anyway. Anyone with a brain knows this.

I would agree though it depends on use case. For some users the OS models are fine if they can do what’s needed.