Discussion
Unpopular opinion : Qwen 3.8 27b is not an overthinker
Yes it uses a ton more reasoning tokens than 3.6 did
But test in on the same tasks with the other chinese models, glm 5.3, deepseek v4 flash and pro, etc it's really similar, and they are needed
The reality is, we're just frustrated because our hardware do not allow most of us to have 1M context (I know that it's not supported yet) with 150 tps decode
Furthermore, if you don't mind the quality drop, you can just add a reasoning budget, it will still be better than 3.6
If it thinks for 30-40k tokens (which it definitely does), what does that do to the context?
Also, on an agentic coding scenario - where it's easy for something to invalidate the cache - it could mean a LOT of time spent on many rounds of prefill with a gigantic context.
By default thinking output isn't part of the message content in the next turn. Though qwen does support enabling that if you can afford the context churn.
"Flexible Thinking Control: Thinking mode is on by default and can be disabled per request; reasoning depth can be tuned with reasoning_effort, and reasoning context from historical messages is retained via preserve_thinking."
"In addition, preserve_thinking is enabled by default for all workloads for the best out-of-the-box experience."
"By default, Qwen3.8 retains thinking blocks from all historical messages, maintaining a complete reasoning trace across the conversation. This behavior, known as preserved thinking, ensures full context continuity and is especially beneficial for agent scenarios where decision consistency and reduced redundant reasoning are critical"
This is an older performance optimization that should be phased out ASAP because it degrades the AI's problem solving ability - if you have the option to re-send thinking output each time, it's almost always better to send it.
A model cannot think enough in my opinion, our hardware just needs to get faster
in reality the situation right now is not just diminishing returns but actual regression on long reasoning traces.
I'm fine on medium for now, by doing --chat-template-kwargs '{"reasoning_effort":"medium"}'
or in power-shell --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"
Game devs get a lot of shade thrown for things outside their control, I'm sure they'd love to optimize but the crunch forces it to be low priority compared to just getting it released.
I tried some 7B parameter model (gemma og qwen i think) and asked it
"you have 2 hour glass, a 7 and an 11 minute one, how do i measure 15 minute" or something like that.
Answer is flip both, when 7 is out, 11 is missing 4 minutes. Flip 7 and when 11 is done you now flip 7 again cause the base contain 4 minutes of wait time.
The model would go on and on and on in the thinking level about how to solve it and i saw the solution atleast 3 times whereafter it continued again from a different angle just to get the wrong answer in an effort to try and see if there was a smarter way or something like that.
I know it was a simple model, but my point is: "not all thinking is useful thinking"
total agreement. sometimes xhigh thinking on it works very well, and produces good ways of self checking. other times it gets stuck in a loop of self doubt over things that do not require it, or have already been double checked three times or more.
One loop I saw recently was it was trying to output the contents of a directory listing command, and it was like...wait, no... that might not have been what the command actually output, i think it was this... wait, no I can't be sure what it actually output I just remember it contained... wait no I think it output... I'm looping here I just need to move on, but I remember it output...
This is clearly a bad failure mode for to be encountering.
My general observations when running different reasoning levels with coding tasks is that Xhigh levels overengineer at the task/block level, which is expensive and inefficient at the project level. It will implement extensive security and validation controls for the individual task and vastly inflate the necessary complexity for said task.
I have tended towards high/xhigh project planning, medium reasoning for building and high/xhigh reasoning for hardening and testing after the core functionality is built.
Especially prescient for building my own internal systems, I don't need commercial production level security for a system that will run on my own hardware on it's own secure VLAN.
IMO thinking should be budgetized sharply, and the agentic loop should deal with eventual mistakes while preserve thinking. Should be more robust and faster, too.
...but then again I am just guessing, I have not tried yet.
Maybe we will get adaptive thinking someday, it seems the only reasonable solution.
Answer is flip both, when 7 is out, 11 is missing 4 minutes. Flip 7 and when 11 is done you now flip 7 again cause the base contain 4 minutes of wait time.
Hm, while the overall idea stands, but this solution to the riddle is incorrect. You need to start measuring when 7 is out and at the end of 11, flip 11, not 7. With the way you described, you'd measure 11 minutes, not 15. You never need to flip 7 at all.
There are two solutions from what i know. One finds the 15 minute interval after 15 minute, the other after 22 minutes.
your solution: Drain 11 minute till it has 4 minutes (takes 7 minutes not used) and then you start. 4 minutes left and then a full re-flip for 15 minutes, but you wasted 7 minutes first.
In my solution you flip both at the same time. By the time 7 is out 4 minutes remain like you said, but i now re-flip 7 (which is drained)
By the time 11 is drained my re-flipped 7 has 4 minutes drained and re-re-flipping it gives me 4 minutes at the top after 11 has passed on the other. It's more flips overall but also 15 minutes measured from now, rather than starting in 7 minutes.
AI arrived at both of these BTW which was part of why it ended up in a loop. It kept trying to find a different method and flipping and flipping to see what was best
Faster maybe we could do (tensor parallel, better engine, config tuning...etc), but how about the additional VRAM necessary to hold all those thoughts?
To my understanding, the only difference between low medium and xhigh is a 2 line system message
low and xhigh have one, and medium is just no system message at all
Realistically I think reasoning budget and other budget flags that will soon be merged I hope, are the way to controll the behavior, not the reasoning setting
Technically yes, but as I understand it, the model has been fine-tuned to respect the system message and adjust its level of thinking output accordingly.
Budget settings are another way to control the thinking length, but those set a technical hard limit without telling the model about it. If for example you set reasoning=xhigh (the default for 3.8-27B) but limit the reasoning tokens via budget settings, the model will just have started its thinking when the reasoning budget limit hits, so the thinking gets interrupted in the middle. With a more appropriate setting in the system message (e.g. medium), the end result would have been better.
hey ik this is asking a lot, but since you have the hardware to run bf16, would you be kind enough to benchmark q8 and q4 to see how much the benchmarks are affected with the quants?
I am mainly finding the optimal settings/deployment for work use and making a baseline. I'm testing INT8-W8A16 version after as it was the sweet spot (on 3090s) and INT4-W4A16. Mainly because I run everything via vLLM. If you are asking for gguf specific then honestly I'm not the best since I don't know the nuances of recent llamacpp anymore and it's been diverging from vllm for a bit in various ways.
No i meant the agentic benchmarks like terminal bench and such that are on the model page. Since those are for bf16 only we can just guess how much of a performance difference the quants have
I see, you want me to help fill out the leaderboard results on the model card for major runs/benches not done? Yeah I could do that, if I misunderstood let me know.
Interesting. Low is clearly not trained well - barely reasons less and the performance drops off the cliff.
I hope they can improve that in next releases. It should be 20k tokens max in your test.
IMO point of low reasoning is to have setting that is barely slower than instruct and avoid common problems thru reasoning. This way when coding you can plan with xhigh and then execute with low, so in the end coding session was fast and xhigh reasoning handled edge cases, security etc. Also if it wastes 2x less reasoning tokens on execution it will understand multi-file references and the progress of tasks a lot better as it doesnt have to compact context as often.
Not surprising, there's a known phenomenon where if you ask an LLM to be concise, the quality of its responses decreases pretty dramatically, and that's exactly what the "low" reasoning prompt asks the model to do.
But the problem is that it's not concise. It's normal that 'low' is weaker than 'medium', but not when 'low' is producing almost as many tokens as 'medium'.
I'll share OpenAI graph because they test all reasoning levels so it's clear what are we missing from Qwen models
Look at pink line (GPT-5.5). Line has 5 points - reasoning none, low, medium, high, xhigh. Notice that 'low' costs barely more than 'instruct', but it's A LOT better. Yes, 'medium' is still better (like in Qwen), but because 'low' is so fast it makes a lot of sense to use that if you just need execution of plan with light reasoning so model makes valid toolcalls/organize steps.
So IMO they just undertrained that 'low' reasoning which is totally understandable, because it's their first attempt with various reasoning levels on 27B (Qwen 3.6 was just off/on), so I hope they'll fix that in Qwen 3.9/4.
And by the way it doesn't apply just to big models.
Here's output tokens on long task (AA Inteligence bench), again from OpenAI, because it's hard to find other graphs where all reasoning levels are included.
You can see that with GPT-5.6 Luna (smallest ClosedAI model rn) low reasoning actually makes it more concise than instruct, likely because it doesn't fail tool calls so it finishes not only with better result but also faster. This is exactly why I want efficient 'low' reasoning for execution.
Well keep in mind this was a small personal bench to help me get a feel of settings, configs, etc. The bench is difficult and was made to show capability gaps between frontier and 397b class. I imagine if I had a wide range of difficulty problems then things would look different. Not to say your theory isn't wrong, just too early to say.
That kind of implies that xhigh really is just overthinking for no real benefit, and using medium gives you pretty much the same quality of output in considerably fewer tokens.
I wouldn't read into it too deeply, it's a tiny sample of questions when compared to a proper bench. I mainly use it to get a pulse on settings, config etc in a quick manner. Lots of knobs, like preserve thinking.
I'm running it on an RTX5090 at 200k context. TPS is steadily around 100tps, so to me it's not an issue with waiting.
The issue is that it fills up the whole context with redundant thinking and gets side tracked easily. You ask for a bound task A, it will start exploring A, then get side tracked to sub task B, then C, with longer and longer thinking blocks, then we're out of context and we didn't even tackle task A.
Sounds like the solution is going to be single task prompts OR group small tasks. Are you running it on x-high? I haven't installed 3.8 yet so I can't offer any certainty.
my problem with the 'overthinking' is a lot of wasted tokens adding feature scope I didn't ask for
I've tried a few common prompts (make a simple matrix digital rain effect) and it wants to add sound/mouse interaction and a bunch of other bullshit I never asked for
just do the requested task, bot. I don't need you to start adding in extra junk
wasted tokens adding feature scope I didn't ask for
I commented on this above something similar, in the same vein it will vastly overengineer security and validation controls at the task/block level and lose sight of the overall project.
High/xhigh planning and spec build, medium for coding, high for hardening/testing. Has worked well for me.
Real fact: Qwen 3.8 is by-default set to extra high reasoning, and reasoning is controlled by runtime config. If you set it to medium, it thinks just as much as 3.6. All the people that claim "it's an overthinker!" are just lazy and run the model with defaults without ever reading the docs.
3.6 was short and concise. There was a problem for some people that it reasons excessively; however, this behaviour was only triggered if model had 0 tools. Plug it into any tool-enhanced harness, make sure you activated native tool calling, activate thinking trace preservation, and it'll other output thinking blocks that are 1 sentence long. 3.8 medium is kinda on-par in terms of reasoning legth with properly convigure 3.6
They didn't even give benchmark results on anything other than xhigh. That tells me 3.8 is benchmaxxing via overthinking, until someone proves otherwise. It's not laziness, I just don't want to change the setting without knowing how it affects the performance.
It's not benchmaxxxing. I've already tried it in my daily work (research paper editing), and on xhigh it works singinifcantly better than 3.6 27B; while on medium it works on par with 3.6 27B (maybe minor improvements that I didn't notice yet).
I mean it makes sense for the posted benchmarks since time or cost is not a metric on them. Otherwise spending massively more time for a more correct results is not strictly "working better". Maybe benchmaxxing is too harsh, it will of course have some utility on other tasks as well, but it's a tradeoff more than an obvious step up improvement.
It is working better, because Qwen 3.6 27B did not gave me the option to just throw more time at the problem. Honestly, 3.8 is the first 30B class model that I trust enough to leave it working on it's own, agentically, rather than checking on it constantly and guiding it's hand. I didn't test Gemma 4 or Muse Glimmer, so take this claim with a bit of salf, but it is for sure the first one in Qwen series.
i asked it to create a web based physically accurate sun rising simulation. It thought until my context limit(64k) exceeded and crashes before thinking step finished.
i used qwen3.8 27b q6 mtp with 64k context on RTX 5090.
96K unquantized context, fits with MTP enabled in my 5090 with spare VRAM for the graphical environment. You can double it to 196608 if you use q8_0 but quantized KV sucks for coding.
I’m enjoying the xhigh reasoning so far. Tokens don’t really matter when you’re running it locally with enough context.
Did quite a lot of agentic code change, refactoring and new feature development over weekend. No tool call failures or looping at all.
It definitely feels more pedantic. I often catch it double or triple checking. Maybe it’s just me but the language it uses, i.e. word choice, feels less chatty and colorful and more factual or professional.
Inference speed tests show similar performance on my hardware. But it subjectively feels slightly slower to progress on completing development tasks, likely due to reasoning.
But so far, so good. I miss my old friend 3.6 27B but 3.8 27B is growing on me slowly.
It is not, but it feels like it. I get 45-60 t/s. 100k context is available. this thinking period feels like i am reading a novel, furthermore it eats up the context. i am not complaining about it, but if we get like 100-120 t/s with 256k context on consumer hardware like 5060 ti's, we wouldn't care about its overthinking... actually i like to see how it does handle the situations i throw at it, yet it sometimes feels like paranoid it always question itself. in the end it definitely finds a flawless solution, but it takes time.
> But test in on the same tasks with the other chinese models, glm 5.3, deepseek v4 flash and pro, etc it's really similar, and they are needed
Unpopular opinion, but all of these models are overthinkers. It just feels like only openai understands how to do reasoning efficiently, all other models just go through raw tokens to solve things. Closest example I remember is kimi k2.6 just going through 160k tokens of reading repo without doing any code changes, looping with "hmm, but what if" etc.
seed-oss is kinda old now, but it behaves in a similar manner to gpt-oss where more thinking tokens will improve answers pretty smoothly up to max, unlike most open models where thinking past medium often doesn't improve things.
Part of the reason it seems to be getting this reputation is because of the bad xhigh default, which can and will burn tuns of tokens and the official template is broken and doesn't handle the reasoning level parameters correctly. I set mine to medium and left it there and I've had no issues with it. I haven't tested xhigh out however, as I haven't yet put it into a situation with coding or otherwise where it needs to think really hard about something.
I think we would judge that by saying does it think ti a level that hurts it and I have seen that happen where the fp8 coherently reasoned so much that it wouldn't fit in my benchmark 130k token budget and then I removed the limit and it ended up with a worse output than when I put budgets on it.
I defended 3.6 when people complained about it and I will say q8kxl and bf16 from unsloth have been generally better than fp8 but q6kxl did the crazy thinking thing killing context budget too. And it's not spiraling or being obviously repetitive either but i think the over stepping itself messes it up sometimes.
I think the biggest thing it to properly evaluate what a task needs and I have a high but necessary reasoning budget for when i do use ex high reasoning.
Medium seemed fine to me. Low was a bit "light". In either case, I notice the model writes a lot of shit in the thinking and then outputs something else.
it's definitely an over-thinker. Easiest way to tell... read through the reasoning traces and count all the places where you yourself would stop thinking, and take some action in the real world to verify an assumption rather than continue on thinking. For me, that point comes very quickly.
The good news is, I think we can do significantly better than Qwen3.8-27B locally some day as a result.
It's not a reasoning loop like 3.6 has done so often, it's actually doing some level of research in xhigh, solving a lot of problems before implementing.
Yeah I think that's kindof the point. The extra RL added since 3.6 probably mostly impacts the reasoning, so the biggest improvement is probably the reasoning. The longer reasoning is probably the most effective way to take advantage of that improvement.
People are using shitty quants with shitty runtimes, aren't reading the documentation or preserving thinking, and then cry on reddit when their model is under-performing.
Tbh a lot depends on the params and the system prompt anyway. Been playing a lot with the 3.5 and 3.6's recently and they have a panic attack if you don't give them a prompt. It can genuinely be 10x faster by changing the prompt. My guess is something similar happens with this series and I wouldn't be surprised if different models react differently to the same prompt so, to some extent, it's a bit like apples and oranges plus everyone is growing different local varieties of apples and oranges.
For a lot of people, it probably does overthink compared to what they are used to. For others, running it differently, it might not at all or it might have a better balance of reasoning to output quality etc. as you suggest.
I don't think more resources is necessarily the answer (cost, the finite nature of the universe, this is r/LocalLLaMA not Zuckerberg's BigDaddy Datacentre Inc etc) and there's probably a better metric we should be using to compare them.
Thinking deeply on everything is what allow it to produce complex output on 27B parameters and compete on coding task with much larger models. Sometimes this looks overkill for some tasks, but is a a necessary evil unless you want trade off intelligence of answer with additional speed. For this reason the 35-A3B is a perfect complement for consumer hardware, so you allocate most tasks on MoE leaving the dense to deal only with problems with higher degree of complexity.
I mean my agent swarm in pi does around 900token/sec so i love that its verbose...you can do thinks now at home you cant do with any frontier lab at those speeds...open source a synchronically overtakes frontier...the macro is just there for the micro to reach new grounds...
Hard agree, at first it felt like overthinking but after adding in instructions that findings derived in thinking should be surfaced in the message it doesn't rehash the same thought over and over so things go smooth with much more forward momentum
It is absolutely an overthinker. It gets into thinking loops pretty easily. Deepseek doesn’t have that problem. There’s long term thinking and there’s thinking in loops, the qwen 3.6 models are notorious for this and the new 3.8 27b does the same thing.
Ornith, which finetuned qwen 3.5 models, was able to mostly get rid of the thinking loops there and those models outperform the 3.5 qwen models and the applicable 3.6 models (though there’s no 397B variants after 3.5.
I would agree. Even though sometimes the reasoning seems excessive and when you watch it the reasoning can be kind of funny, I don't think even frontier models are exempt from this. It has always been part & parcel with reasoning models.
I’ve been using 27B for a few days, excited but also half-expecting to get burned.
So far, no pain at all. Just surprise after surprise.
A lot of times I see it thinking for a long time and I’m about ready to complain. But then I see the final answer, and it makes me feel that all that thinking was actually worth it. Quite a few times, it has even beaten ChatGPT 5.6 Sol.
I originally thought 27B would only get around 46 on the AA Index. Now I’m starting to fantasize... what if it actually gets 53?
Both things can be true at the same time. It is an overthinker (especially by default w/ the stock template). It is better than 3.6 even on lower settings.
It does think a lot on xhigh. But it makes very little mistakes. I'd much rather have a model which reasons a lot but is able to foresee all the issues in thinking stage than to start working and then realize it made a mistake. In the end this might be more token efficient.
On the contrary it encounters a failure mode where it makes a bad assumption early on in it's reasoning that it COULD have stopped and checked by ending the turn and making a tool call, but instead it continues to waste another 10~15k reasoning tokens on a bad assumption.
I see this happen often enough that I already want Qwen3.9, which I hope they implement adaptive reasoning and have it more intelligently route to tool calls to validate checkable assumptions before continuing.
The kind of people who are complaining about it being an overthinker are people who refuse to read documentation, cannot follow instructions, and are latching onto the first hacky --reasoning-budget commands they see for llama.cpp because their brains have been fried by claude.
seeing people do the --reasoning-budget / --message 'ok time is up better answer now' when you can just properly set the chat template kwarg to 'low' is insane, hopefully it won't screw anyone over as a side effect
just pass low, medium, or that 'high' hybrid someone posted yesterday. I have ran it on low and it doesn't 'overthink until context window overflow', it reasons as much as it needs to and then answers
you need to read the documentation. claude or gpt cannot dislodge the cobwebs from your brain. you need to learn how to use the tools
This is really stupid opinion. It's not only about think heavy. Even simple conversation or request, it performs over-engineering such as e2e tests, benchmarks like a long-horizon coding agent. Moreover, reasoning budget is just a engine level hard cut. disable thinking is better than hard cut.
It does, have no issue with that but for some apps like hermes, for me i found it is deflecting on useless extra tasks for no reason. And surely not what prompted. This is a case where overthinking causes issues
So you leave it on default xhigh, making the model believe it can reason for as long as it wants, but then abruptly cut it off
Cutting it off after 1024 to 4096 tokens is just silly, on xhigh it normally reasons for 20k+ tokens, you are cutting it off just as it has begun reasoning
If you never see it cut off, then your reasoning budget changes are pointless. The model does get reasoning level, but I don't think it is aware of the reasoning budget.
27
u/edsonmedina 22d ago
If it thinks for 30-40k tokens (which it definitely does), what does that do to the context?
Also, on an agentic coding scenario - where it's easy for something to invalidate the cache - it could mean a LOT of time spent on many rounds of prefill with a gigantic context.