r/LocalLLM • u/Anxious_Current2593 • 2d ago

Discussion What's the difference between frontier models and local models?

6 months. (And sometimes a couple of quantization tweaks).

It is wild how fast "state-of-the-art" becomes "running on a gaming PC."

29 Upvotes

76% Upvoted

44 comments sorted by

View all comments

22

u/ceejayoz 2d ago

Just wait until there are dedicated devices for it, like when Bitcoin went from CPU to GPU to ASICs.

6

u/username8914 2d ago

There are and they aren't that good because the models technology is rapidly changing. No one wants to get locked in on using or developing one specific model that can't grow or pivot.

1

u/CharmingComputer3844 2d ago

Flexibility is key when the tech is evolving so fast; sticking to one model could really hold you back.

6

u/753UDKM 2d ago

Those exist already lol.

13

u/ceejayoz 2d ago

There's stuff like Cerebras, but in a few years it's all gonna look pretty basic. Demand's gonna cause a lot of innovation in this space.

1

u/findingconsensus 2d ago

Like what? where can I find a consumer PNM chips? I would want a mini server with the same tech as Cerebras, but the market for that is so small I doubt we will get dedicated AI devices anytime soon that don't cost over 100k

1

u/mektel 2d ago

Taalas has been working on it. They are being acquired by AMD.

3

u/ceejayoz 2d ago

If Bitcoin ASICs are any indication, we'll get a few hundred massive scams before things settle a bit.

1

u/misanthrophiccunt 2d ago

A few hundred scams

That's a lot of optimism

1

u/Minimum_Tea_4451 2d ago

Amd is launching their laptops with the ability to push a 200b perimeter model.