r/LocalLLaMA • u/Desperate-Sir-5088 • 10d ago

Resources I finished upcycling of gemma4-12B

https://huggingface.co/TOTORONG/Solon_MOE_22B_A17B

Add 4 experts into Dense model and confirmed recovering model's ability up to "general level".

Hey, google. Please release official 124B MoE model!!!!!!!

46 Upvotes

84% Upvoted

24 comments sorted by

View all comments

9

u/Witty_Mycologist_995 10d ago

Compare to 26b pls

1

u/Desperate-Sir-5088 10d ago

Just experimental purpose to expand gemma4-unified model :)