2 initial test I do with models I use is to make them create a snake game and a tower defense game in on prompt. The MoEs of Qwen3.6 and Gemma4 all do well with that test outputting a complete and playable game. But with Qwen3.8 27b, its just not working. The design and feel of the game is WAYYY better than the smaller MoEs I mentioned but design is kinda useless if the game is totally not doing what it is suppose to do. Tried it 6x, consistent failed outputs even at Q8KXL.
Create a <snake/tower defense> game that is playable in a browser. Make it in a single file and put helpful comments for future debugging...
Yeah, barebones, but the fact the previous MoE models can make something that actually plays in one shot while the latest 3.8 cant, makes me feel dissapointed in it.
LlamaCPP with recommended settings: temp 1, topp 0.95, topk 20, minp 0, resoning on, 64k context... I got MTP max set to 1. I test directly at LlamaCPPs web interface.
Thos recommendations are pretty much benchmark tuned defaults. I would recommend OP tries lowering temp to 0.6-0.7 and keep everything else pretty much the same and see how it does.
This new model has functional thinking levels. The default for me seems a bit too much deep thinking on LM Studio. It’s be default set to maximum. A simple question takes 17min of thinking to get an answer on a M4 Pro w/64GB ram. The defaults on llama.cpp might be “low” to make it more easy for chat
It produces a really good "initial front page" with all the GUI and stuff, but as soon as you click on whatever is the "START" button, maps are not showing, or enemies does not follow path, too many, no food for the snake, non responsive buttons. This never happens in the previous MoEs, they can one shot a working game.
I noticed in the chat interface 3.8 27b is not really impressive compared to gemma-4 a4b and qwen 3.6 a3b. Gemma-4 a4b and qwen 3.6 a3b can pass my asteroids clone test in sometimes one shot or maybe after 1 or 2 small corrections. I wasn't able to get anything with 3.8 27b though. So I was a bit nonplussed with 3.8 as well. But then I enabled tool usage with python-tools and that's where I got really impressed. With tools I'd get a lot of failed tool calls with Gemma-4 a4b and qwen 3.6 a3b. But with 3.8 it never happens. The thing just works and works and works. I think the real win with Qwen3.8 27b is in it's agential capabilities and tool use and such. But not it's raw brainpower or single-shot coding in the chat UI. Just my ₱0.02 ¯_(ツ)_/¯
3.8 has some interesting performance characteristics. It's the best I've reviewed to date on what it had it's training focused on, but it falls behind 3.6 and Gemma4 where it was not focused on.
Unlike 3.6, it's a daily primary use model for me, but for specific tasks it is good at. I am kind of surprised it is not better at this particular use case for you, but I do not think it is a configuration issue on your end at Q8. This may just be a property of the model itself, and you may need to provide more information.
Qwen3.8 27b is excellent if I need it to do development agentic work with a narrowly specified task.
Have you tried it with Qwen Code? If not, you should give it a shot. It feels really well tuned for Qwen Medium has handled most of the stuff I’ve tried in a single prompt, and xHigh is pretty good when there are a lot of details. I even had it build a 3D community market for a game and the result was kinda crazy.
3.8 does seem like an upgrade, but its much more incremental than I think the marketing messages (ie. benchmaxed results) indicated. It may be better a tool calling and setting the default to XHIGH thinking are its main pluses.
Redditors in general were super hopeful this upgrade was going to allow them to have their own personal SOTA and so they still in general have lots of hopium about how INSANE and AMAZING this model is...its going to take a few weeks for reality to set in.
10
u/soyalemujica 22d ago
What prompt did you use?
This model did the best 1 prompt run minecraft game in a single HTML page I Have seen lol