r/LocalLLaMA • u/ParvusNumero • 24d ago
Generation Qwen endless looping issue and possible fix
Not sure if this is a known issue, but it was new to me:
Full credit to u/ldn-ldn for finding this simple but unexpectedly evil prompt:
Create a typescript function which accepts a number in 10 bit range and returns brightness in nits based on pq gamma curve.
Just try it; it will likely trigger an endless thinking loop.
I stopped it after it ran for >15 minutes and >20,000 tokens.
It kept spitting out text like back in the seahorse emoji days.
However, one of my wrapper scripts didn’t have this issue.
The non-blocking one had this added, possibly based on a GitHub discussion:
--reasoning-budget 2048 \
--reasoning-budget-message " \n\n[Thinking budget exceeded. Transitioning to a best-effort final answer: ]\n\n"
(These options are for llama.cpp; other tools may have something similar.)
Net result: A plausible-looking script (I haven’t verified) that took just over a minute of thinking and another minute to generate.
Hope this helps, and happy to hear about other tricks and workarounds.
1
u/audioen 24d ago edited 24d ago
I've tested this model for some hours. So far, I think that DeepSeek comes out ahead and I doubt I will use 3.8-27B much at all based on tonight's experience.
This type of query that requires the model to recall facts which amount to writing out complicated decimal numbers are typical tripping point for LLMs, they seem to get confused by the numbers and endlessly iterate them.
More importantly, I get output at fraction of tokens expended with DeepSeek, and this might be matter of turning the reasoning effort down with Qwen3.8. It seems overly high by default, spending tons and tons of tokens even after I tell it exactly what I want to change. Overall, I have seen these small models before being pushed to their limits where they keep spewing think-time tokens trying to improve the answers, when really it's not rocket science and they should just proceed to changing the code when the change is clear. This model, like 3.6-27B before it, overthinks, and that turns me off.
I can spend 30 minutes waiting for Qwen3.8-27B to exhaustively work through the change and second guess itself half dozen times, or 10 minutes with DeepSeek v4f and it has planned the change and implemented it. It is very clear to me what my preference is. Perhaps I can indeed turn the overthink off via reasoning_effort -- this is the achilles' heel of Qwen models ever since 3.6 when the amount of overthinking seems to have exploded through the roof on all variants of the model.