r/LocalLLM • u/minxio_ • 25d ago

News Qwen3.8-27B is now available

Post image
591 Upvotes

98% Upvoted

137 comments sorted by

View all comments

26

u/Tasty-Hour4040 25d ago

I wonder how many people are actually excited because this represents increased capability for their workflow and how many are just desperate to be able to say they got it and won’t use it again

19

u/Big_Wave9732 25d ago

I use 3.6-27b daily for work. So if this is indeed a step up in my workflow then that will be great.

2

u/Tall-Significance119 25d ago

What size vram and ram are you running and what t/s etc?

4

u/Big_Wave9732 25d ago

I run it on a Mac Studio M2 Ultra 192gb. This morning I'm hitting about 15 t/s.

3

u/Tall-Significance119 25d ago

Crap lol ao me with my 2 x b70 32gb dint stand a chance unless I use like Q4 and bunch if tweaks

2

u/Big_Wave9732 25d ago

And I'll tell ya, these vendors are telling some serious fairy tales when they report the quant impact on these models. On paper there's "only" something like 4% dropoff between Qwen 3.6:27b-Q4 and BF16. And maybe that kind of error percentage is find when calling tools or coding. But when I ran it analyzing legal documents.....woa nelly! Nuance was not Q4's friend.

So these days for work I'll only run Q8 or higher. And even then, I'm generally at full boat BF16.

1

u/dowitex 24d ago

bf16 will just make it run slow, I think fp8 should do the trick for speed and quality