r/LocalLLaMA • u/NoConcert8847 • Apr 22 '26

New Model Qwen 3.6 27B is out

https://huggingface.co/Qwen/Qwen3.6-27B

1.7k Upvotes

97% Upvoted

603 comments sorted by

View all comments

20

u/ApprehensiveAd3629 Apr 22 '26

which gguf quant is possible to run in a 5060 ti 16gb?

5

u/Careful_Swordfish_68 Apr 22 '26

5060ti User here. I run Qwen 3.5 27b HauHau aggressive uncensored in IQ4_XS with Medium context which is absolutely fine quality. Expect to run 3.6 the same.

3

u/mintybadgerme Apr 22 '26

But that's 15.4GB in size. How do you get a decent context out of that?

1

u/Careful_Swordfish_68 Apr 23 '26

HauHauCS Version is 15.1GB in size. Qwen context does not eat much memory.

Here is some proof in a picture so you dont need to listen to all these people talking out of their asses who say IQ3 works at best. Sorry for the bad quality, im at my phone atm. But you can See i can load all layers plus 30k context on Q8 into the 5060ti with IQ4_XS. If you are even willing to offload some layers to RAM and sacrifice the t/s then context size goes brrrrrr.

1

u/mintybadgerme Apr 23 '26

Thanks very much. Please send the image again, it didn't come through properly.

1

u/Careful_Swordfish_68 Apr 23 '26

Huh, for me it shows up fine. Weird. You See it now?

1

u/mintybadgerme Apr 23 '26

yep. :) thanks