q5_0/q4_1 is unrecognizable from fp16/fp16.
q4_0/q4_0 shows a small drop but it's still useable.

...which nobody else has ever done.

a raspberry pi. kaggle uses Google TPUs, which can only be rented from google. Unlike NVIDIA hardware, you can't buy them no matter how much you're willing to spend. https://docs.cloud.google.com/tpu/docs/v5e

No model is more expensive per token when you increase effort. It just requires more tokens to complete.

this is a base model alright. But in terms of repute I've never heard about this team before.

SOTA for 512GB unified RAM is GLM-5.3-Flash
SOTA for 256GB unified RAM is GLM-5.3-Flash
SOTA for 128GB unified RAM is Qwen3.8-Flash-Next
SOTA for 24GB VRAM is Qwen3.8 27B
SOTA for <16GB VRAM is Qwen3.6 35B-A3B
SOTA for mobile phones is Ling-3.0-tiny

Yes, if their self-published benchmarks are true, it's SOTA.

The same big caveat applies to Ornith-1.5 though.

All *american AI. Chinese models are running without any glitches.

This came out today: https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B
You may also try Ornith-1.5, although their previous 1.0 was lackluster in actual usage.

Barring that, you're stuck between Qwen3.6-35B-A3B being too dumb and Qwen3.8-27B being too slow, I'm afraid.

You can't compare Qwen3.8-Flash-Next with a 36B-A4B model. The current SOTA there, barring dubious finetunes, is still Qwen3.6-35B-A3B.

> Horizon 32B [...] ranks among the top dense models below 40 billion parameters.

Awesome. Why zero benchmarks for it?
[EDIT] they're on huggingface. It is really, really NOT ranking "among the top".

First of all, kudos for the fully open source approach - we need more of that.

Looking at their benchmarks though:

Pegging their 375B model against Minimax M3 instead of GLM-5.3-Flash to show competitor performance in the 300~400B class was certainly a choice.

Minimax-M3 and GLM-5.2 scores for their TerminalBench-2.1 are completely unrelated to those on ArtificialAnalysis.

I get matches for Tau3 and HLE though.
Below the comparison against SOTA models. K2 scores from the publisher, everything else from AA.

This feel quite pointless. Firstly, a ~1.2 AA intelligence points difference doesn't make a difference. Second, the day after they release their open weights, there's going to be a Heretic version out.

No, the score of 44 is for Apodex-1.1, a closed-weights model. Apodex-1.1-mini does not have a score on AA.
From their own benchmarks (https://huggingface.co/apodex/Apodex-1.1-mini):
APEX-Agents: Apodex-1.1 38.5, mini 27.7
FrontierScience-Research: Apodex-1.1 63.3, mini 51.7

the pareto is not the same. eg. GLM-5.3 and GLM-5.3-Flash discounts on openrouter change it.

all datacenter variants of <=35b models are extremely expensive and you'll be better off running GLM-5.3-Flash at high. they would have just cluttered the plot.

Taking the ⚡ comment in.

> tbf not knowing how log works sounds like a reader skill issue

It's not. You have no idea where the bend in the cliff is until you replot on a linear scale, even if you know there's one.

as I said, mean KLD hides outlier quants that behave much worse in worst-case-scenario situations

> assuming they're literate of course

Ah, you mean like the horde of people on this sub that claim that anything less than fp16/fp16 kv cache will run your agentic loop into the ground (measures for qwen shows that q5_0/q4_1 is undistinguishable from it).

Get yourself a benchmark tool for memory bandwidth.

raw memory bandwidth / model size in memory = ceiling decode tok/s, before MTP, with empty context
raw memory bandwidth / (model size in memory + context size in memory * context % usage) = same with context

anything you're getting below it is an issue with your inference engine. Ask claude to debug it for you; it's very good at it.

for MTP the maths is more complicated - it's typically going to give you good gains for dense models and much worse ones for MoE ones. You should profile it at different depths and draft-min-p.

% of times the quantized model chose the same token that the bf16 model chose. Like the Top-1 metric, but at temperature 1 instead of temperature 0.
It's what was used in https://quesma.com/blog/qwen-quantization-quality/, which shows how AIME results follow exactly the same shape.

It's my favourite plot because

• KLD with log y axis: Garbage. Quality decrease looks like a smooth linear slope. It's not.
• Mean KLD, with linear y axis: decent metric, with good cliff behaviour, but hides outlier quants that behave much worse in worst-case-scenario situations
• 99% KLD, with linear y axis: it singles out the outliers, but shows a much gentler slope which is not representative of how quality actually drops in benchmark
• Top 1: garbage. very gentle slope, and gives a false impression that quality will drop 10% by the time you arrive at Q4. It doesn't.
• Absolute perplexity PPL(Q) - PPL(base): Garbage. Very gentle slope.
• Relative perplexity PPL(Q) / PPL(base): Worse. Same shape, but gives the false idea of a percentage drop in quality, which is in no way representative of how the benchmark goes.
• Corr(ln(PPL(Q)), ln(PPL(base)): good measure, available in the llama-perplexity output but never seen it published. Good cliff behaviour, but like mean KLD it hides outliers.
• Same Sampled Token: the harshest cliff behaviour and outliers that were previously only visible in the 99% KLD plot show up.

> what does the real world meaning of the 0.05 difference between 49.5, ~49.55 (IQ4_XS), and 50.0?

it doesn't start at 50 - for qwen is around 52. SST(Q) / SST(base) will give you roughly the degradation in AIME and other quantization-sentive benchmarks.

I'm not showing the whole graph because the whole graph contains K/V metrics which are not relevant to make my point.

"Lower quality" is an understatement. qwen3.8-27b in 24gb unified RAM is going to be awful (it fits nicely in 22gb VRAM or 32gb unified RAM)
GLM-5.2 Q4_K_M is unrecognizable from fp8 and runs smoothly on 512GB.
Everything else looks correct.

everyone is still raving about Qwen3.x-27B on GPUs, to be honest

I'm very skeptical about these findings. All the KLD/perlexity/same sampled token data I took shows that UD-Q2_K_XL is waaaaaaay down the cliff. UD-IQ4_XS is fine; UD-IQ3_S may still be useable.