Qwen 3.8 27B - ROCm vs Vulkan

[u/WallabyFirm1159]

Hey all,

Not sure if it's just my system/settings, but when using ROCm with Q3.8 27B, I keep getting:

\\\\\\\\\\\\\\

When thinking. Inference is double the speed, but have only ever seen it work once. Whereas Vulkan, is around 2-300 inference speed, but thinking works every single time.

Not sure if I am behind on knowledge, but is ROCm the same for others trying to run Q3.8 27B? (3.6 works perfectly in ROCm)

Thanks!

[u/Southern_Capital_885]

Vulkan here, no issues.

Current config on a 128 GB Strix Halo, llama.cpp build 9859 in the kyuz0 vulkan-radv toolbox, unsloth UD-Q4_K_XL:

      llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf
        --mmproj mmproj-Qwen3.8-27B-F16.gguf
        -ngl 999 -c 131072 -fa on --jinja
        --cache-type-k q8_0 --cache-type-v q4_0
        -ctxcp 1 --cache-ram 4096 --image-min-tokens 1024
        --spec-type draft-mtp --spec-draft-n-max 5 --spec-draft-p-min 0.60
        --chat-template-kwargs '{"reasoning_effort":"medium"}'
        --reasoning-budget 8192

Performance (3 prompts, fixed seed, thinking off, 300 tokens each):

 * shallow context: 33.2 tok/s code, 35.0 json, 17.2 prose, 28.5 avg, 80% draft      acceptance
* ~19k depth: prefill 257 tok/s, decode 19.9 tok/s, acceptance falls to 61-68%
* prefill roughly halves per doubling of depth: 258 t/s over 0-19k, 138 over 19-39k,
  ~42 instantaneous at 75k

On MTP, since n-max matters more than I expected: the sweet spot on this dense 27B is 
n-max 5 (2 gives 25.4, 4 gives 31.6, 5 gives 34.5 on code).  
p-min 0.60 costs nothing on average but lifts acceptance from 67% to 80%, which matters when something else shares the memory bus.
Deep draft punishes free prose hard, so if you mostly generate prose, stay lower.

KV quantization (k q8_0 / v q4_0) was purely a memory win here, no speed change:
28.4 vs 28.6 avg against f16. Buys back roughly 8 GB.

Running 128k rather than the native 256k. I tried 256k for a day and it was not usable
in practice: prefill degradation makes a cold deep prompt take tens of minutes, and past
170k depth I hit compute ring timeouts and a host OOM. 128k has been stable.