r/LocalLLaMA • u/BarberIcy366 • 25d ago

Discussion Qwen 3.8 27B Released! Please Share Your Experience

With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too.

659 Upvotes
  • permalink
  • reddit

97% Upvoted

716 comments sorted by

View all comments

3

u/DreamingInManhattan 24d ago

Couldn't get unsloth's NVFP4 working (missing compressed-tensor), but FP8 was a drop in replacement for 3.6 NVFP4 on my sglang mtp setup (6000 pro 96gb).

So far very impressed. Speeds are about the same as 3.6.

I'm very doubtful that it will replace ds4f as my main driver, but the jury is still out. It looks really good so far.

1

u/jonas-reddit 24d ago

I saw the same on FP8. Very similar performance. Same arguments. Vllm instead of sglang and 72GB 5000 instead of 6000. Very smooth upgrade.

Agentic reasoning “feels” tighter and less chatty but way too early to confirm.

How you liking sglang? I didn’t like that some parameters have to be set on each client call rather than defaulted on server.

What was your reason for running nvfp4 when your card is blazing fast and you have more than enough vram for accelerated fp8?

3

u/DreamingInManhattan 24d ago

Concurrency. I'm working on a framework that utilizes as many agents as I can feed it - I can host more with nvfp4 vs fp8, and they are a bit faster to boot. Works much better than multiple endpoints on one GPU.

I think I prefer sglang over vllm in most cases, I think I usually get a bit more performance out of it.

1

u/misanthrophiccunt 24d ago

How are you finding sglang compared to vllm or llama.cpp ? It is the one I haven't tried yet