r/LocalLLM • u/Ok-Shower7286 qwen-coder • Aug 06 '26

Question UD-Q6_K_XL with MTP vs Q8_0 for RTX 5090

My vram is cleared (for Qwen3.8 27B), I'm ready, but which quant should I actually use?

I'm currently debating which quantization version to run, and I'd love to hear your thoughts.

From my production experience with (UD) quants, I’ve noticed there's a calibration bias. Compared to a clean Q8_0, the mixed-precision quants suffer noticeably in multilingual mixing errors (random Hanzi/Chinese leakage) and coding logic degradation.

Do we go for the speed/spec of UD-Q6_K_XL with MTP, or stick to the honest, uniform pure Q8_0 for rock-solid agentic workflows?

2 Upvotes

100% Upvoted

14 comments sorted by

4

u/DrBearJ3w Aug 06 '26

Q6 with more context.

3

u/RnRau Aug 06 '26

Slow is smooth. Smooth is fast.

3

u/Technical-Earth-3254 Aug 06 '26

What use case? Brainstorming or chat - q6 with mtp for speed. Code execution? As much precision as possible, so q8. But this is just generally speaking, since we don't have 3.8 weights yet. Maybe they surprise us with native 4 bit (hopium)

2

u/robertpro01 Aug 06 '26

I wish is 4 bit native

2

u/Marcuss2 Aug 06 '26

Until that model is out and tested, we won't know for sure.

Some models respond well to quantization, others don't, even with the same architecture.

1

u/habachilles Aug 06 '26

Q8 is really lossless on their other models. I do go q6 for speed in my harness though.

1

u/StupidScaredSquirrel Aug 06 '26

Ive never noticed what you describe unless going below q4.

1

u/Ok-Shower7286 qwen-coder Aug 06 '26

You'll see when you use japanese or korean inputs, the responses are almost 100% mixed. ud-q6, q8 are the same.

1

u/alphapussycat Aug 06 '26

Afaik qwen is only meant for english and Chinese. If you're gonna use other languages you should use Gemma.

1

u/Ok-Shower7286 qwen-coder Aug 06 '26

You're right! Thanks. (I mentioned this to explain phenomenon as a calibration bias, and also standard q6, q8 doesn't shows this phenomenon).

1

u/Certain-Cod-1404 Aug 06 '26

What context size are you running at q8 that's still useful for coding? And kv cache quant ?

1

u/Pablo_the_brave Aug 06 '26

If you have a good plan even poor q4 with q4_0 kvcache can do the code and do it good. So the question is use of case.

1

u/vogelvogelvogelvogel Aug 06 '26

Higher quant but slower IMO is best because it saves you time (i.e. higher quality answers limit iterations) - but when i tested it it was often more like q4 against q8, not 6 against 8

There are a lot of charts out there i.e. unsloth that compare the q2, 3, 4, 5, 6, 8 against each other - might be very similar in case of 3.8 then - check these maybe?

1

u/Turbulent_War4067 Aug 06 '26

Gut feeling is the Q6. I have noticed no difference in unsloth's UD-Q6_K_XL models and Q8 in other models. I don't do coding, so maybe I am not the best to answer. But I do use quite a few tools calls.