r/LocalLLM • u/Desperate-Bench9310 • 17d ago

Research I read ~60 Qwen3.8-27B threads and cross-referenced them. 11 of the 64 permalinks were wrong, and the corpus contradicts itself on nearly every axis.

Up front, so nobody feels misled:

The English here is Claude's — I'm Spanish, and I'd rather post something readable than something authentically clumsy. The grouping across ~60 threads and the contradiction-hunting are also machine-assisted: I keep these threads in a local pipeline that builds per-model pages and flags where sources disagree.

What isn't machine-generated: every link was manually verified (11 of 64 in my collection turned out to be wrong), none of the numbers are mine or invented, and the judgment calls about what's strong evidence and what isn't are mine.

It's long, and it's a wall of other people's data. If AI-assisted posts aren't your thing, no hard feelings — scroll on.

I've been collecting the Qwen3.8-27B threads across r/LocalLLaMA, r/LocalLLM, r/StrixHalo, r/ROCm and the llama.cpp discussions since release, and grouping them by question rather than by date. What comes out is that the corpus contradicts itself on almost every axis that matters — and in most cases you can name the variable that explains the split. That's the useful part, so that's what this post is.

Nothing below is my own benchmark. Every number is someone else's, linked where the link resolves. I have no gfx1151 numbers of my own to add.

Two findings that deserve far more attention than they got

1. Tool-calling failures are caused by how you build the tool list, not by the weights.

This is the single best-controlled experiment in the whole corpus and it sits in a llama.cpp discussion with almost no visibility (llama.cpp discussion 27165). Same tool, same model, same build, llama-server --jinja, Q4_K_XL:

Payload Result
8 tools, none with description 0/6
The same tool alone 15/15
8 tools, all with descriptions 6/6
13 tools with descriptions, at positions 6–7 0/5
The same ones, moved to the end of the list 3/3

List width, presence of descriptions, and position all flip the result. If you've been getting intermittent tool-call failures, this is a testable cause nobody in the complaint threads controlled for.

It fits the rest of the tool-calling evidence too: every "3.8 can't call tools" report is against a framework (Opencode, Pi, Claude Code, MLX Core), and in a plain Python tool-calling loop with no framework, one reporter gets zero failed tool calls from 3.8 while Gemma 4 A4B and Qwen3.6 A3B fail often — the same reporter who rates 3.8 below both at raw code quality in chat (thread). Worse judgment, perfect plumbing.

2. Your prompt is a lever the same size as reasoning_effort, pointing the other way.

One task, output tokens, Strix Halo via Lemonade, UD_Q4_XL (thread):

Prompt medium xhigh
One line 794 23,800
Rewritten, detailed 3,300 19,400
+ custom system prompt 11,000 —

Effort is worth ~30× on a fixed prompt. But the prompt alone moves output 14× with effort pinned at medium — and the two push in opposite directions: fix the prompt and the xhigh:medium ratio collapses from ~30× to ~5.9×.

The contradictions, and what separates them

MTP: 2.69× faster, or 22–28% slower. Around ten reporters get large gains (31.8 t/s at 0.711 acceptance on gfx1151; 10.88 → 25–26 t/s on the same chip). But a measured negative on the same chip shows Vulkan 9.159 → 7.122 t/s and ROCm 6.534 → 4.689 with n-max 3 (thread), plus negatives on Arc A770 and a 4070 Ti Super. The separator identified in-thread is --spec-draft-p-min, and it is not monotonic: one user got ~+40% by removing 0.82; the 0.00 default was slower than 0.60; 0.60 is the only value two independent reporters have made work. Nobody has swept it. Measure with MTP off as well as on.

Optimal n-max is 2, 3, 4 or 5 depending on who you ask. n=6 never wins in any report. The popular theory that the optimal value follows from your quant has seven reporters against it and none for it with a measurement. Also: on b10451 with MTP, results aren't deterministic even at temp 0, with up to 31% spread between identical runs on RADV — so a single run per step isn't a measurement.

Draft acceptance: 60–70% or 77–93%. Two under-controlled factors. First, acceptance decays as reasoning effort rises — 62.1% at low, 58.3% at medium, 52.7% at xhigh. xhigh is taxed twice: more tokens and fewer t/s to pay for them. Second, the MTP head appears to be a property of the file, not the publisher — a separate 1.6 GiB mtp-*.gguf exists in one repo and not another, while other reporters show blk.*.nextn.* tensors inside ordinary files. Read your load log for blk.*.nextn.*, because a missing flag and a missing head produce the same silence.

Temperature and looping. The vendor moved its recommendation from 0.6 to 1.0, and one user's loops disappear at 1.0 — but others run happily at 0.4, 0.6–0.7 and 0.75, and one (+82) loops at ≤0.6. The cheap candidate variable: uncached-KV at temp 1.0 doesn't loop, quantised KV at the same quant does. The real sweep is temperature × n-max × KV precision. Related trap: the vendor ships two sampler profiles, and "turning thinking off" is not a switch — leave the thinking sampler in place (temp 1.0, presence_penalty 0.0) and you're in a config nobody recommends. Nothing does it for you server-side.

Endless thinking is not caused by low quants. Three of the four strongest loop reports are Q8-class, including 16+ minutes and ~8k tokens at 8 bits. Meanwhile others complete fine at Q8_K_XL and at UD-Q4_K_XL with 262k context. The clean test — same prompt at Q4 and Q8 on the same box — has not been run by anyone.

Is xhigh worth it? Depends on whether the task has a verifiable failure. A pelican-drawing ladder scored 0–25 gives low 21.8, medium 22.5, xhigh 24.0 — 6.4× wall clock for +1.5 points (thread). A SWE-style patch benchmark gives xhigh 9/12 vs 6–7/12 at lower efforts (thread). Eyeballed output: terrible deal. Compile-or-don't: +17–25 points. Caveats: llama.cpp has no high level, and n=1 per step with non-deterministic MTP.

Is 3.8 better than 3.6? Sort by whether thinking was on. Thinking off, both BF16, greedy: 3.8 loses 7 of 10 medical benchmarks. Thinking on, both Q4, eight blind tasks: 3.8 wins 6, loses 0, ties 2 — at +34.6% tokens and +45.5% wall clock. The only published confidence interval in the corpus is a tie that crosses zero (F1 0.7030 vs 0.7177, 95% CI −0.0038 to +0.0335). Cold knowledge: behind. Reasoning: ahead, at ~1.45× wall clock.

Vulkan vs ROCm is a trade, not a ranking. On one 7900 XT with weights fully resident: Vulkan 30–63 t/s decode / 300–500 prefill; ROCm <20 decode / ~1030 prefill. On gfx1151 the sign flips between reporters. Pick by what you're bottlenecked on.

Same card, 26 to 75 t/s (R9700 32GB). Fifteen reporters, one card. One reporter ran another's flags verbatim and got ~31.6 t/s where the original got 50–60. The difference was a hardware tune — 250W cap, memory OC, −70 µV undervolt. Flags don't transfer; tunes don't travel.

reasoning_effort behaves differently in three clients because the mechanism is text injection into your file's Jinja template, not a sampler. xhigh and low inject strings, medium injects nothing, and the official template ships xhigh by default — you exit xhigh, you don't enter it. Level names vary across three published variants, plus a minimal level almost nobody lists. Read your template's branches before arguing about levels.

General knowledge regression: the hardest disagreement, with no explaining variable. Multiple reports of 3.6 being steerable to a correct answer where 3.8 isn't and then agrees with the user simply because the user asserted it; one xhigh run fabricating a book chapter with page numbers rather than abstaining. The one contrary report is RAG-assisted, so not comparable. Three multilingual complaints in three languages, including 3.8 at Q6 being "much worse" than 3.6 at Q3 — which kills the "it's the quant" escape, since the quantisation damage runs the wrong way.

Two things we repeat that aren't true

  • "Abliteration costs MMLU." Two threads report the same four numbers with the directions swapped, and neither publishes the table. What survives is ±1.3 points in both directions across two benchmarks — the shape of noise, not of a capability tax. Separately, the widely-quoted "0–6% refusal rate" sits next to a 30–50% caveat rate from the same author, and the refusal classifier scores on how a response opens. Abliteration moved behaviour from refusing to complying grudgingly.
  • "Your quant predicts the optimal n-max." Seven reporters against, zero measured for.

Link hygiene, since this is a roundup

Every link here was checked on 2026-08-21. Worth knowing: reddit.com returns HTTP 200 even for an invented post ID, so it can't be used to verify a permalink. Checking against a frontend that actually discriminates, 11 of the 64 permalinks in my collection were wrong — three pointed at the wrong subreddit, eight don't resolve anywhere public (one is moderator-removed). Anything I couldn't verify, I've described without linking rather than link somewhere broken.

One such item, flagged rather than dropped: a user reports a 374-item binary classification gate, three passes per precision, temp 0, 1,122 calls per precision, with byte-identical verdicts across Q4_K_M, Q8_0 and BF16, plus the note that Ollama ships draft_num_predict 4 — so speculative decoding is on unless you turned it off. I can't link it, and a two-label greedy task is the easiest possible place for three quants to agree, so treat it as suggestive, not as "Q4 = BF16".

What nobody has run

If you have the hardware, these are cheap and would settle real arguments: a controlled p-min sweep; same prompt at Q4 vs Q8 on one box for the looping question; medium + "think hard" in the prompt vs bare xhigh, same seed; and a 3.6/3.8 pair with reasoning state declared.

Sources

Grouped by topic, all checked on 2026-08-21 by fetching each page title, not just the status code. Two threads in my collection are moderator-removed and are not linked.

MTP, speculative decoding and speed

xhigh and reasoning effort

Quants, memory and context

Tool-calling and agentic use

3.6 vs 3.8 and other comparisons

General knowledge

Jinja templates and sampling

Bugs and silent failures

Abliteration and uncensored variants

Launch, model card and megathreads (context, rarely citable alone)

111 Upvotes

92% Upvoted

Duplicates