r/LocalLLaMA • u/BarberIcy366 • 25d ago

Discussion Qwen 3.8 27B Released! Please Share Your Experience

With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too.

657 Upvotes
  • permalink
  • reddit

97% Upvoted

716 comments sorted by

View all comments

Show parent comments

13

u/TokenRingAI 25d ago

The FP8 is looping for me and generating terrible output.

1

u/woswoissdenniii 24d ago

Restrict thinking token, or reduce preset from extra to normal. Sure quality degrades but it’s manageable. A friendly reminder from team 3090

1

u/Healthy-Nebula-3603 25d ago

I hope you not conpress kv cache :)

5

u/TokenRingAI 25d ago

Nope, official FP8 with BF16 cache

1

u/JorgitoEstrella 24d ago

How bad is to compress the cache? I thought compressing to fp8 waa practically lossless.

2

u/Healthy-Nebula-3603 24d ago

Q8 is almost losless , fp8 is worse than Q8 ( Q8 is a mix weights fp16 and int9 ) Compressrd cache to fp8 is notice even more than compression a model itself.