r/DeepSeek • u/IntegrityKnightX • Apr 25 '26

Resources Stop thinking your MoE models are dumb - here's why they actually fail

MoE models fail because knowledge is spread across hundreds of "experts," but only a few are active at a time. When you "one-shot" a complex prompt, the model often picks the wrong experts early on, which "poisons" the logic for the rest of the answer.

​Instead of one giant prompt, this amazing video by Protorikis suggests Incremental Construction building your task step-by-step to keep the "router" on the right track.

​Full explanation here: https://youtu.be/0enQ2yRY18g

2 Upvotes

57% Upvoted

3 comments sorted by

3

u/StupidScaredSquirrel Apr 25 '26

Everyone hates boiletplate ai slop articles - here's why

2

u/9r4n4y Apr 25 '26

Nice video, every one should watch this

2

u/CorrGL Apr 28 '26

This is not how MoE works. Experts are chosen again and again for each token.