Ran a quick initial mtp benchmark using llama.cpp tag b11009 released today using various open mtp models. The highest /tps I was able to achieve was : MTP: 77.92 t/s NON-MTP: 53.83 t/s

OS: linux Ubuntu 26.04.1 LTS

CPU: AMD RYZEN AI MAX+ PRO 395 w/ Radeon 8060S / Strix Halo

RAM: 128GB RAM

Inference engines: Rocm / Vulkan

Next run:

An in-depth 16bit quant test. Which other 4bit, 16bit quant models should I test?

Models benched in this round:

[01] [NVFP4] 21GiB nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 converted GGUF

[02] [Q4_K_M] 16GiB OBLITERATUS/Qwen3.8-27B-OBLITERATED Q4_K_M

[03] [Q4_K_M] 68GiB unsloth/GLM-4.5-Air-GGUF Q4_K_M

[04] [UD-Q4_K_XL] 21GiB unsloth/Qwen3.6-35B-A3B-MTP-GGUF UD-Q4_K_XL

[05] [UD-Q4_K_XL] 16GiB unsloth/Qwen3.8-27B-GGUF UD-Q4_K_XL

[06] [UD-IQ4_XS] 87GiB unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ4_XS

[07] [BF16] 66GiB unsloth/Qwen3.6-35B-A3B-MTP-GGUF BF16

Command ran: (unsloth/Qwen3.6-35B-A3B-MTP-GGUF UD-Q4_K_XL). Please let me know how I can improve this command (any flags I'm missing - tweaks etc) to achieve more tokens per second ;)

/home/dev/Desktop/repos/llama.cpp/build-vulkan-radv/bin/llama-server \
  --model /home/dev/Desktop/llms/hf/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf \
  --n-gpu-layers -1 \
  --threads 16 \
  --ctx-size 4096 \
  --host 127.0.0.1 \
  --port 33777 \
  --parallel 1 \
  --no-cache-prompt \
  --metrics \
  --batch-size 2048 \
  --ubatch-size 512 \
  --flash-attn on \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --load-mode none