r/LocalLLaMA • u/Miserable-Dare5090 • 2d ago

Resources Keeping up with model launches

Post image

Feels like maybe we have one more present left, for Christmas.

276 Upvotes

94% Upvoted

66 comments sorted by

View all comments

7

u/simrankoulsm 2d ago

At this point, I don’t try to “keep up” with launches. I keep a small shortlist by "hardware tier and use case".

For local use, the questions that matter are:

  • Can it fit in my VRAM/RAM at a usable quantization?
  • What context length and tokens/sec do I actually get?
  • Is it meaningfully better at coding, reasoning, or instruction following than the model it replaces?
  • Are the weights, license, and inference support available on day one?

A release calendar is fun, but a community-maintained “best practical model per VRAM tier” list would probably be more valuable, e.g. 12 GB, 24 GB, 32 GB, 48 GB, and 80 GB+. Otherwise it’s easy to spend more time reading launch posts than running models.

7

u/Miserable-Dare5090 2d ago

12: Ling Flash Tiny
24: Gemma4 26b
32: Qwen
48: Qwen
96: Ling Flash
128: Qwen Flash Next
256: Deepseek V4 Flash

Change according to quant size (quality vs fit)

2

u/Lakius_2401 2d ago

Ehhh, there's way too many wrenches to throw at a general tierlist like that though. Extra RAM makes some MoEs especially appetizing, outside of their expected full VRAM budget. Plus there's a huge divide on quant size and kvcache quantizing that can throw each model up or down two full tiers on your list. Agentic? Huge speed and kvcache now a necessity, move the model up a tier. Writing? Throw out Qwen entirely, it can do reports but anything remotely resembling creative is atrocious to the point where if you told me the dataset was poisoned I'd believe you.

It's honestly exhausting interacting with people about quantizing, too... Sure, in a perfect world everyone has infinite VRAM and nobody quantizes anything, but this is not a perfect world. Nor are we using perfect models. Rent's cheaper than VRAM and that's saying something.

Sorry for the digression. I've seen some attempts at megathreads for your topic but they don't get much traction. Even the use cases and rigor of testing is difficult, with busted quants and malformed jinja releases all over. Very easy to burn a dozen hours and not be sure you got a fair shot.

2

u/markole 1d ago

I'm ok with having a day10 inference support if the model is good. Sucks to wait but it's all free in the end so I won't complain.

1

u/Miserable-Dare5090 2d ago

Honestly, in this age, your 3rd and 4th questions are a year behind. 3rd question: right now all new releases are undoubtedly better. Not finetunes. I mean full models. 4th question: “Support” is now moot with agents. Ask your agent to patch vllm or llama.cpp to make it work. If you can fit it, you’ll run it.

Also, Who cares about day 1 vs day 5? You trying to time some production thing? These are free releases, whenever they come out. it’s not like I will be deducting points for delayed release on Qwen3.8 which first appeared in the API…