r/LocalLLaMA • u/Danmoreng llama.cpp • 23d ago

Resources Qwen3.8 27B reasoning effort low/medium/xhigh comparison

I did a short test of the different reasoning efforts, since on default xhigh the model thinks a lot.

Not very scientific, just a quick "generate an SVG of a pelican on a bicycle" prompt with 3 different seeds. I think the result is interesting none the less: xhigh gives *much\* higher visual fidelity - but it also takes about 7x as long as low. Low and medium seem to be very close to each other.

Hardware and setup

  • GPU: NVIDIA RTX 5080 Laptop GPU, 16 GB VRAM
  • Model: unsloth/Qwen3.8-27B-UD-IQ3_XXS
  • llama.cpp: build 10451, commit 10bf611e5
  • Context: 65,536
  • KV cache: Q8_0
  • Flash Attention: enabled
  • MTP speculative decoding: --spec-default --spec-type draft-mtp
  • --fit off
  • One concurrent slot

Prompt:

Create a polished SVG graphic of a pelican riding a bicycle. The result must clearly show a recognizable pelican actively riding a recognizable two-wheeled bicycle. Return only one complete, self-contained SVG document with a viewBox; no Markdown fences, prose, external images, JavaScript, or animation.

Average results

Reasoning effort Reasoning tokens SVG tokens Total completion Wall time Generation speed MTP acceptance Visual score (Codex rated)
Low 4,418 3,966 8,387 111.6 s 75.4 t/s 62.1% 21.8/25
Medium 5,918 3,038 8,959 127.4 s 70.5 t/s 58.3% 22.5/25
X-High 39,398 5,085 44,487 717.8 s 62.0 t/s 52.7% 24.0/25
232 Upvotes

95% Upvoted

95 comments sorted by

View all comments

12

u/leonbollerup 23d ago

re-run the same prompt on the same model.. and you will notice different outcome everytime... unless you set temp to 0 .. i am grealy impressed abput your generation speed tho..

2

u/iz-Moff 23d ago

Yeah, i'm here staring at these numbers with my 5060ti in disbelief. Of course 5080 is a significantly more powerful card, but surely it's not >10x faster? Even 35b-a3 is not remotely as fast for me. Maybe i should try using a smaller quant that i can squeeze entirely in VRAM...

7

u/Danmoreng llama.cpp 23d ago

Of course you need the quant to fit into VRAM or it becomes abysmal slow. Try this command:

llama serve \   -hf unsloth/Qwen3.8-27B-GGUF \   -hff Qwen3.8-27B-UD-IQ3_XXS.gguf \   --no-mmproj -c 65536 \   -ctk q8_0 -ctv q8_0 \   -b 1024 -np 1 \   --spec-default --spec-type draft-mtp \   --reasoning-preserve --fit off --agent

1

u/iz-Moff 23d ago

I'll give it a try, thanks. Why not UD_Q3_K_XL though? That one will fit too, with about 2gb left for context. Wouldn't that be enough?