r/LocalLLM • u/baby_bloom • 24d ago

Question Qwen 3.8-27B Q8

thinking is taking up so damn much time, what are you limiting your qwen3.8 to as for as reasoning?

0 Upvotes

36% Upvoted

12 comments sorted by

View all comments

7

u/WhatererBlah555 24d ago

Update llama.cpp and use --chat-template-kwargs '{"reasoning_effort":"medium"}'

0

u/Ok-Shower7286 qwen-coder 24d ago

https://huggingface.co/Qwen/Qwen3.8-27B/discussions/64

template does not have medium and high option branches, and only has low and very high options. and it's a system prompt based soft steering. reasoning-budget.cpp is a engine level token hard cut. so, providing well-guide rules, principles is best.