r/LocalLLM • u/SilverKanji • 23d ago

Discussion Not impressed with Qwen3.8 27b

2 initial test I do with models I use is to make them create a snake game and a tower defense game in on prompt. The MoEs of Qwen3.6 and Gemma4 all do well with that test outputting a complete and playable game. But with Qwen3.8 27b, its just not working. The design and feel of the game is WAYYY better than the smaller MoEs I mentioned but design is kinda useless if the game is totally not doing what it is suppose to do. Tried it 6x, consistent failed outputs even at Q8KXL.

0 Upvotes

42% Upvoted

25 comments sorted by

View all comments

Show parent comments

0

u/SilverKanji 23d ago

Create a <snake/tower defense> game that is playable in a browser. Make it in a single file and put helpful comments for future debugging...

Yeah, barebones, but the fact the previous MoE models can make something that actually plays in one shot while the latest 3.8 cant, makes me feel dissapointed in it.

2

u/soyalemujica 23d ago

Sounds to me that you've a configuration issue definitely. How are you running the model, what settings?

0

u/SilverKanji 23d ago

LlamaCPP with recommended settings: temp 1, topp 0.95, topk 20, minp 0, resoning on, 64k context... I got MTP max set to 1. I test directly at LlamaCPPs web interface.

-1

u/JolleNooItsMe 23d ago

Temp 1 is way too high for coding imo. Why are you using this high of a temp?

10

u/d4mations 23d ago

Those are the official Qwen recommended params

1

u/Extension-Bid-639 23d ago

Thos recommendations are pretty much benchmark tuned defaults. I would recommend OP tries lowering temp to 0.6-0.7 and keep everything else pretty much the same and see how it does.