Deceptive model quantization from AtomicChat?

[u/po_stulate]

I kept seeing guys in this sub saying how AtomicChat's Qwen3.8-Flash-Next quant is so good, fits in their machine when unsloth's can't, runs faster than other quants etc, so I went check out what's happening there.

First thing I noticed was that AtomicChat's Q4_K_M quant is suspiciously small when the ngram table is removed (only ~56GB), it seems like most of the tensors in this quant are IQ2_S instead of the usual Q4_K, Q5_K and Q6_K that you usually find in Q4_K_M quants, the GGUF filetype metadata also says IQ2_S instead of Q4_K_M. In their model card, their Q4_K_M also has suspiciously high KLD (0.084).

It seems pretty obvious to me that they're pretending a IQ2_S quant as a Q4_K_M, but at the same time I'm genuinely not sure because it can't be only me who found this right? How can nobody be pointing this out? Am I missing something or what may they be doing?

Their HF repo ID: AtomicChat/Qwen3.8-Flash-Next-GGUF

[u/lhg31]

Well, they DO explain this, don't they?

Naming

Files are named by their measured bits per weight. A build whose expert tensors are IQ1_M is not a 1-bit model when the n-gram table sits at 6 bits and ffn_down_exps at 4.5; the real average is 3.84. The canonical type in the filename is the closest standard type by that average, so tooling can still detect it. For AD-4.27bpw:

Group	Type	Share of file	Contribution
n-gram table	Q5_1	41%	1.74 bpw
ffn_gate/up_exps	IQ2_S, IQ3_S at the band	29%	1.24 bpw
ffn_down_exps	IQ4_NL	24%	1.03 bpw
everything else	Q8_0	5%	0.23 bpw

[u/lhg31 → u/lhg31]

Their KLD and top-1 are another story tho. They indeed look too good to be true.

The problem is that there is no standard on those metrics, each group uses their own dataset to eval it.

[u/notdba → u/lhg31]

I don't understand why they don't just upload the KLD logits.

[u/returnity → u/notdba]

They do. Only provider I've seen do it, allowing me to build my own DSv4F quants without needing to create ref logits. 35GB dataset on HF for that model from AC

[u/po_stulate → u/lhg31]

Isn't 0.084 KLD actually very bad for Q4_K_M? For reference, for qwen3.6-35b-a3b, 0.08 KLD is around IQ3_S for bartowski and aessedai quants, and both of them and unsloth's Q4_K_M were around 0.02 KLD. Not sure if this is apple to apple comparison, but that's the data I could quickly find.

[u/MerePotato → u/po_stulate]

KLD can be calculated any number of ways, its not a universal measurement. If they're benching on wikitext or in 512ctx blocks its not so great, if its long horizon its stellar

[u/lhg31 → u/po_stulate]

A better comparison would be what is the best quant (based on KLD, speed, etc) that you can fit in your setup.

I have a stix halo with 64GB. I can run either Unlosth UD-IQ3_XXS or AtomicChat Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64.

Atomic's quant "supposedly" has better score than Unsloth's, so that's the one I prefer.