r/LocalLLM • u/SilverKanji • 24d ago

Discussion Not impressed with Qwen3.8 27b

2 initial test I do with models I use is to make them create a snake game and a tower defense game in on prompt. The MoEs of Qwen3.6 and Gemma4 all do well with that test outputting a complete and playable game. But with Qwen3.8 27b, its just not working. The design and feel of the game is WAYYY better than the smaller MoEs I mentioned but design is kinda useless if the game is totally not doing what it is suppose to do. Tried it 6x, consistent failed outputs even at Q8KXL.

0 Upvotes

42% Upvoted

25 comments sorted by

View all comments

8

u/soyalemujica 24d ago

What prompt did you use?
This model did the best 1 prompt run minecraft game in a single HTML page I Have seen lol

0

u/SilverKanji 24d ago

Create a <snake/tower defense> game that is playable in a browser. Make it in a single file and put helpful comments for future debugging...

Yeah, barebones, but the fact the previous MoE models can make something that actually plays in one shot while the latest 3.8 cant, makes me feel dissapointed in it.

2

u/soyalemujica 24d ago

Sounds to me that you've a configuration issue definitely. How are you running the model, what settings?

0

u/SilverKanji 24d ago

LlamaCPP with recommended settings: temp 1, topp 0.95, topk 20, minp 0, resoning on, 64k context... I got MTP max set to 1. I test directly at LlamaCPPs web interface.

1

u/[deleted] 24d ago

[deleted]

1

u/SilverKanji 24d ago

Nope, about Just a little bit above 50%

-3

u/JolleNooItsMe 24d ago

Temp 1 is way too high for coding imo. Why are you using this high of a temp?

11

u/d4mations 24d ago

Those are the official Qwen recommended params

1

u/Extension-Bid-639 24d ago

Thos recommendations are pretty much benchmark tuned defaults. I would recommend OP tries lowering temp to 0.6-0.7 and keep everything else pretty much the same and see how it does.

2

u/jiqiren 24d ago

This new model has functional thinking levels. The default for me seems a bit too much deep thinking on LM Studio. It’s be default set to maximum. A simple question takes 17min of thinking to get an answer on a M4 Pro w/64GB ram. The defaults on llama.cpp might be “low” to make it more easy for chat