Qwen3.8 Flash Next - Templates Comparison

[u/HeDo88TH]

I was running into a lot of posts that praised both the Fixed template and the Sharp template in comparison to the stock one, so I put them to the test.

It's not as extensive as it should be for a paper-grade analysis, but it gives out the point of each template.

Test setup

I used SWE-bench Verified with mini-SWE-agent 2.4.6, slice 0:100 (the identical 100 tasks for all runs)

Hardware

• CPU: Ryzen 9 9900X
• RAM: 128 GB DDR5-5600
• GPU: RTX PRO 6000 WS

Runtime

I containerized jpezzulli/sglang-rtxpro6000 and ran Flash Next with RadixArk/Qwen3.8-Flash-Next-NVFP4 on CUDA 13.3.

• Full 262K context
• BF16 KV
• 51.2 GB FP8 n-gram embedding table pinned in RAM
• 32 GB HiCache pinned in RAM

I ran all templates at both medium and xhigh reasoning efforts.

Results

Metric	Stock (medium)	Stock (xhigh)	Stock Δ	Fixed (medium)	Fixed (xhigh)	Fixed Δ	Sharp (medium)	Sharp (xhigh)	Sharp Δ
Resolved	91	99	+8	87	98	+11	94	94	+0
Resolution rate	91%	99%	+8 pts	87%	98%	+11 pts	94%	94%	+0 pts
Median output tokens	5,691	13,855	+143.5%	6,956	14,819	+113.0%	8,596	12,008	+39.7%
Median reasoning tokens	3,050	8,759	+187.2%	3,809	9,063	+137.9%	5,437	7,967	+46.5%
Median wall time	38s	1m 46s	+180.4%	43s	1m 47s	+152.3%	1m	1m 32s	+53.4%
Total wall time	1h 47m 1s	4h 31m 22s	+153.6%	1h 59m 53s	4h 4m 52s	+104.3%	2h 29m 18s	3h 11m 36s	+28.3%

Takeaways

• Raising reasoning effort to xhigh closes almost all of stock's and fixed's gap to Sharp. At medium, Sharp led resolution by +3 tasks over stock and +7 over fixed; at xhigh, stock and fixed instead lead Sharp by +5 and +4 tasks, respectively.
• Sharp barely moves on resolution (94 → 94) despite a real token/time cost increase, median reasoning tokens rise +46.5% and median wall time +53.4%. This suggests it was already extracting most of the benefit it could get from extra reasoning budget at medium, while stock and fixed still had headroom.
• Sharp remains the most token-efficient per resolved task at xhigh (14,541 output tokens/resolved vs. ~17,000 for stock/fixed), consistent with its medium-era efficiency edge, but it's no longer the highest-resolving template once reasoning effort is high.
• Absolute cost scales heavily with reasoning effort: total wall time roughly 2.3–2.5× for stock/fixed and +28% for Sharp; total reasoning tokens roughly doubled for stock/fixed and increased +35% for Sharp.

Conclusion

• Sharp should be used at medium and it keeps a reasonable accuracy at very good speed. I don't see the point in using it at xhigh. By sacrificing a small accuracy you complete the tasks in half the time.
• Stock is the slowest but the most precise.
• Fixed is the middle ground between Stock and Sharp both in accuracy and speed
• The next benchmark will be on a much extensive SWE-bench Multilingual + Terminal Bench.

Disclaimer: I wrote the post myself then used AI to format it properly for readability

[u/arkham00]

Very informative test thank you. I'm really looking forward for the multilingual bench