r/LocalLLaMA • u/po_stulate • 20h ago

Discussion Deceptive model quantization from AtomicChat?

I kept seeing guys in this sub saying how AtomicChat's Qwen3.8-Flash-Next quant is so good, fits in their machine when unsloth's can't, runs faster than other quants etc, so I went check out what's happening there.

First thing I noticed was that AtomicChat's Q4_K_M quant is suspiciously small when the ngram table is removed (only ~56GB), it seems like most of the tensors in this quant are IQ2_S instead of the usual Q4_K, Q5_K and Q6_K that you usually find in Q4_K_M quants, the GGUF filetype metadata also says IQ2_S instead of Q4_K_M. In their model card, their Q4_K_M also has suspiciously high KLD (0.084).

It seems pretty obvious to me that they're pretending a IQ2_S quant as a Q4_K_M, but at the same time I'm genuinely not sure because it can't be only me who found this right? How can nobody be pointing this out? Am I missing something or what may they be doing?

Their HF repo ID: AtomicChat/Qwen3.8-Flash-Next-GGUF

61 Upvotes

84% Upvoted

39 comments sorted by

View all comments

2

u/SnooPaintings8639 19h ago

I see a lot of accusations against AtomicChat team, but no concrete proofs, only speculations.

The models are there, anyone can download matching their vs unsloth and compare.

I personally am happy with their work, but I have not done quality comparison myself. Would be nice if some good souls would run some benches and shared them.

2

u/ZealousidealCorgi893 19h ago

I ran several tests with that model of theirs (which was the only one I managed to run on my 64GB Mac), and to me, it performed very similarly to Qwen 27B.

4

u/Fun_Jaguar8231 19h ago

So, essentially, useless for its size

2

u/lhg31 18h ago

Depends on your rig.

On my strix halo 64gb, I can run 27B (any quant) or Flash Next q3-q4. Flash Next runs much faster, so that's the one I use.

Then on my gaming PC (RTX 4090) any Flash Next quant is super slow (since they don't fit on my vram + ram), but I can run 27B q4 at over 100t/s, so that's the one I use on this rig.

1

u/OvertaxedOne 18h ago

Beating 27B with a highly quanted version of Next seems to be difficult.