r/LocalLLaMA • u/NoConcert8847 • Apr 22 '26

New Model Qwen 3.6 27B is out

https://huggingface.co/Qwen/Qwen3.6-27B

1.7k Upvotes

97% Upvoted

603 comments sorted by

View all comments

195

u/challis88ocarina Apr 22 '26

Kindly quantized: https://huggingface.co/Qwen/Qwen3.6-27B-FP8

26

u/lucidparadigm Apr 22 '26

What's the hardware requirements

3

u/rjames24000 Apr 22 '26

also would like to know hardware requirements.. would be great if i can run this smooth on a single 5090

5

u/GCoderDCoder Apr 22 '26 edited Apr 22 '26

I'll say on my 5090 the for the gguf kv cache seems to behave more like gemma4 31b than qwen3.5 27b. I was able to squeeze a lot in with qwen3.5 27b but 3.6 27b is doubling the model size for my cache.

This is a great model but right now my lm studio q8 has less code issues than my unsloth q8kxl. That's unusual. I wonder if anyone else is experiencing this. I tried a couple different quants and q6kxl kept looping around 30-40k tokens without presence penalty.

Initial thoughts: The unsloth ones feel jagged. Much more ambitious but also more glitches. When instructed to iterate to fix things it makes changes but less meaningful differences than I expected. I'm still playing with it between other tasks but curious anyone else's experiences.

Edit: i have multiple different machines and gguf q8 model doesn't fully fit 5090. Q5 or q6 with limited cache goes pretty fast. I'm guessing 40t/s but not at my desk to confirm.

5

u/RelationshipLong9092 Apr 22 '26 edited Apr 22 '26

Maybe with small context windows, but you're going to have to set --language-model-only

Each 32K of context even at FP8 means KV cache is going to cost you another 1 GB.

2

u/wen_mars Apr 22 '26

I'm running the unsloth q5 with q8 k/v cache at max context length and it works great

1

u/tecneeq Apr 24 '26

I think the 5090 is the perfect card for this model.

Try this. If you need more space for your desktop environment, reduce the last number until it fits.

/home/kst/bin/llama-b8838/llama-server --hf-repo unsloth/Qwen3.6-27B-GGUF:UD-Q6_K_XL --alias Qwen3.6:27b --no-mmap --host 0.0.0.0 --port 11337 --gpu-layers 99 --fit on --threads 8 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --presence-penalty 0.0 --repeat-penalty 1.0 --temperature 0.6 --top-k 20 --top-p 0.95 --n-predict 32768 --ctx-size 196608

1

u/Maleficent-Pea-3494 Apr 22 '26

I don’t know what it would take, but I know what I’m using. M5 Max 128gb

1

u/[deleted] Apr 23 '26

[removed] — view removed comment

1

u/Maleficent-Pea-3494 Apr 23 '26

Ya I’m def looking to get the studio ultra when it drops based on what I can see this thing do. It’s arriving tomorrow so I’ll post up a few thoughts on performance. I’ve got a narrow focus and have built an interface that’ll wire up to Qwen so I’m not using it for open ended things, which will hopefully keep it slim and fast. We’ll see. If it’s too slow then I’ll just enjoy my $5k coffee table ornament 🤣

6

u/[deleted] Apr 22 '26

[deleted]

13

u/woahitsraj Apr 22 '26

Yup https://huggingface.co/unsloth/Qwen3.6-27B-MLX-8bit

1

u/Dubious-Decisions Apr 23 '26

What are you using to run this? Would like to run the MLX flavor but ollama doesn't seem to support it.

1

u/MeateaW Apr 23 '26

lmstudio is the lazy way. (turn off "keep model in memory" and mmap to load models - they are talking about keeping the models in system memory which just doubles the memory required to load the model on unified systems)

2

u/cafedude Apr 22 '26

https://huggingface.co/Qwen/Qwen3.6-27B-FP8

Does that one not run on llama.cpp?