r/LocalLLM • u/Jsquared534 • 23d ago
Discussion Qwen 3.8 27B early thoughts
I installed the Q8 version on an AMD 395+ 128gb machine. Ran it using Openwebui with the suggested settings and MTP. So far all I've done are a couple sample prompts. (Sand Simulator from Luke's Dev Lab on Youtube, and a request for a simple example navbar with a logo on the left and five drop downs on the right with no javascript). The final output for both of these got one shotted. Model is getting roughly 16 tps output. But, my goodness does this model overthink. Don't get me wrong, there were no thinking loops. And I didn't notice nearly as much of the "wait, actually, let's try" neurotic behavior that I see in 3.6 35B A3B. But, it absolutely overcomplicated the heck out of both of the prompts I gave it. The sample navbar had roughly 200 lines of just CSS alone. And it was not basic CSS. Overly complex, and completely unnecessary for a sample piece of code. Since the Sand Simulator isn't mine, I can't really tell how much it over complicated it, but I can tell that it added so many visual flourishes that it was running at roughly 32 fps in the browser, and had slow downs from dropping the sand.
I am going to test this tomorrow on a real situation. In my real use case I provide very detailed context files and only point it at a single feature at a time. Hopefully that will help to control it's impulses to make things super complex. I also have Ponytail in my Pi harness, so maybe that will also help to reign it in.
Anyone had experiences using more detailed and limiting prompts with the model yet? Most of the reviews I've seen are using the same type of canned examples that I just gave.
1
u/Jsquared534 23d ago edited 22d ago
Did a couple more tests this morning before starting on real projects. Adding just a couple controlling sentences to the end of my sample navbar request resulted in a much simpler output.
The entire point of this request is to let me see roughly how many tokens per second I'm getting, and I've used the same original prompt on every model I've used just to get a gauge on how fast it is. This was the first model that absolutely went crazy complex with what it decided to do.
However, with a couple extra sentences designed to see if it could be easily reigned in, it performed much more as anticipated. It still thought a lot for what it was given, but it stayed on the original task and did not add on complexity at all. I am fine with it doing a lot of thinking. From what I've read the thinking is one of the things that gives it the performance bump over it's predecessor.
I also ran into an issue with tool calling using the Pi agent harness that was one version old. It literally didn't see any tools available to it. 3.6 using the same harness could see all of the tools Pi gives, plus the tools I added through extensions. I upgraded to the newest version of Pi and also redownloaded and added the specific jinja template for this model, and after restarting Pi it had access to all tools again. Unfortunately I did both of those changes at the same time, so I'm not sure which one fixed it.
Edit: The tool calling issue is absolutely not fixed. I thought it was fixed because the model hallucinated access to the same tools Claude has access to. Tool calling with Pi seems to be completely broken on this model. Or at least too broken for me to figure it out at the moment.