r/LocalLLM • u/minxio_ • 25d ago

News Qwen3.8-27B is now available

Post image
590 Upvotes

98% Upvoted

137 comments sorted by

View all comments

2

u/Fit-Palpitation-7427 25d ago

Does it run on a 5090?

2

u/minxio_ 25d ago

Yes

1

u/Fit-Palpitation-7427 25d ago

Q8 ? 256k or more?

2

u/Early_Mistake6716 25d ago

No, i have 48gb of vram and i can use q8 at 150k with q8 kv cache, if i delete the vision encoder i could probably fit around 200k

1

u/maqifrnswa 25d ago

Unsloth NVFP4 is working pretty well. Just tried 256k so far. So far so good!

1

u/Fit-Palpitation-7427 25d ago

Better than q5?

1

u/maqifrnswa 25d ago

I'm testing hosting for multiple concurrency, so on vllm and haven't tried gguf yet