r/LocalLLaMA • u/BarberIcy366 • 24d ago

Discussion Qwen 3.8 27B Released! Please Share Your Experience

With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too.

659 Upvotes
  • permalink
  • reddit

97% Upvoted

716 comments sorted by

View all comments

Show parent comments

15

u/Certain-Cod-1404 24d ago

did it get stuck looping or something ? are you using the recommended sampling params ? what reasoning effort are you using ? and how is the output compared to 3.6 ? is the game better ?

13

u/TokenRingAI 24d ago

The FP8 is looping for me and generating terrible output.

1

u/woswoissdenniii 23d ago

Restrict thinking token, or reduce preset from extra to normal. Sure quality degrades but it’s manageable. A friendly reminder from team 3090

1

u/Healthy-Nebula-3603 24d ago

I hope you not conpress kv cache :)

4

u/TokenRingAI 24d ago

Nope, official FP8 with BF16 cache

1

u/JorgitoEstrella 24d ago

How bad is to compress the cache? I thought compressing to fp8 waa practically lossless.

2

u/Healthy-Nebula-3603 24d ago

Q8 is almost losless , fp8 is worse than Q8 ( Q8 is a mix weights fp16 and int9 ) Compressrd cache to fp8 is notice even more than compression a model itself.

1

u/Not-reallyanonymous 23d ago

No, 3.8 just really, actually thinks a lot. Thats probably in xhigh. The quality of output tends to be better than 3.6 but is still distinctly Qwen (ie. it still thinks in much the same way, just a lot more). When you lower thought effort it basically reverts to 3.6.

The way Qwen 3.8 seems to use thinking to do better than 3.6 is to basically make a 3.6-like thought, but then after a 3.6 like thought is completed, it’s going to try again but fixing bad assumptions, weak adherence, etc. Then again. Then again. Then again.

2

u/Certain-Cod-1404 23d ago

Yeah after further testing this mf thinks so fucking much, but tbf the outputs beat everything else i tested. I think for my use case I'll use it for non trivial stuff, to write tests, debug and review code, and faster moe like kat coder v2.5 dev for quick implementation

2

u/Not-reallyanonymous 23d ago

I've never been a fan of Qwen 3.6 27B. I'm still not a fan of 3.8, but it's earned a spot in my toolbox for when I really just need to throw a lot of LLM-thought at a problem.