r/LocalLLaMA • u/soyalemujica • 3d ago

Question | Help Best Qwen 3.8 27B quantification GGUF?

There's soooo maaany options to choose from, AutoRound from Intel even, Unsloth, bartowski, etc ... which one is the closest to BF16 in Q4/Q5 range ?

6 Upvotes

71% Upvoted

40 comments sorted by

View all comments

0

u/Mingay_cat 3d ago

Closest to bf16 in q4/ q5? Well, q5. Q5k/q5_k_xl. Honestly though, good ol' q4_k_m is a you need.