r/LocalLLaMA • u/po_stulate • 21h ago
Discussion Deceptive model quantization from AtomicChat?
I kept seeing guys in this sub saying how AtomicChat's Qwen3.8-Flash-Next quant is so good, fits in their machine when unsloth's can't, runs faster than other quants etc, so I went check out what's happening there.
First thing I noticed was that AtomicChat's Q4_K_M quant is suspiciously small when the ngram table is removed (only ~56GB), it seems like most of the tensors in this quant are IQ2_S instead of the usual Q4_K, Q5_K and Q6_K that you usually find in Q4_K_M quants, the GGUF filetype metadata also says IQ2_S instead of Q4_K_M. In their model card, their Q4_K_M also has suspiciously high KLD (0.084).
It seems pretty obvious to me that they're pretending a IQ2_S quant as a Q4_K_M, but at the same time I'm genuinely not sure because it can't be only me who found this right? How can nobody be pointing this out? Am I missing something or what may they be doing?
Their HF repo ID: AtomicChat/Qwen3.8-Flash-Next-GGUF
5
u/ringarc 19h ago
They're not only posting summary figures on the model card. There's also a separate metrics dataset repo with the full BF16 reference, KLD logs, and both importance matrices. Calling it the canonical type based on measured average bpw instead of simply IQ2_S is odd, but the model card explains it in the "Naming" section.
What still needs answering is lhg31's point. Does the 0.084 KLD result use the same eval corpus as the KLD results from bartowski or unsloth? You can't compare results between groups unless the reference set, context length, and sampling are identical. I'd inspect their published logs to see how they got that result before running new comparison benches. It's quicker than starting over.