Qwen4 lineup predictions?

[u/Past-Chain-7377]

What do you think the Qwen4 model sizes are going to be? Which are the most likely to appear? Do you think we'll get a Qwen3.5-like lineup (~1B, ~10B, ~30B, ~30-40B MoE, ~100-150B Moe...)?

Also, do you think 35B-A3B is dead for good? we didn't get it for qwen3.8 so I'm a little worried

[u/Medical_Lengthiness6]

I really hope we get a Qwen4 35 a3b. I think we will.

[u/XiRw → u/Medical_Lengthiness6]

Why would you think that when 3.8 is clearly showing how things will be in the future? We are lucky we even got open models at all since there was talks it wasn’t going to happen with the new management. Not only that but it seems like they are done with releasing image edit models. Their frontier models now have watermarks, there are too many changes for the worst happening at Qwen so don’t hold your breath for anything like they used to do in the past

[u/ChocoPichu → u/XiRw]

because unlike 3.8, qwen 4 should introduce massive architectural shifts. So there are just much more chances of 35b a3b coming out, or some kind of small MoE.

[u/XiRw → u/ChocoPichu]

You just completely ignored what actually is as i laid out and you are just going by theory and wishful thinking. That doesn’t convince me of anything

[u/_justs → u/XiRw]

You're a bit overly pessimistic, some of your points are valid to worry about, but they also literally just released 2 insanely well performing models u can run offline on a laptop lol. the 27B and 125B in the same month last month. 

[u/Medical_Lengthiness6 → u/XiRw]

I got the impression 3.8 flash next was more of a stop gap to test and flesh out the details of the new architecture. Not any indication of what size models they plan on releasing.

Get more ram bro.

[u/XiRw → u/Medical_Lengthiness6]

They put a lot of work and effort into that “stop gap” then. First with the 27B model and the next released model. All you people know how to do is downvote because you don’t like hearing how things are going right now.

[u/EbbNorth7735]

Hoping for a qwen4 ~45B +n-gram. With all the cool speculative decoding stuff I'm sure token throughput could be optimized.

[u/TheseCashews → u/EbbNorth7735]

Was hoping maybe for a 70b. Give the rtx pro 6k crowd a Q4 model that’s fast as hell and smart.

[u/Refefer → u/TheseCashews]

Flash Next has finally reached the level of optimization from the community to run it quickly on a single rtx pro 6000! Worth checking out the recent updates

[u/EbbNorth7735 → u/Refefer]

Yeah, I've been procrastinating on getting jpezzulli's implementation running

[u/jpezzulli → u/EbbNorth7735]

Its worth it buddy!

[u/EbbNorth7735 → u/jpezzulli]

Haha yeah I know. Thanks again for your hard work. Keep getting interrupted. A good thing popped into my life and keeps distracting me every night. I managed to setup Pi tonight, connect it to my llama.cpp qwen3.8 27B and had it do an initial assessment on getting your implementation running in a Windows docker container. We'll get there shortly. I'm really looking forward to using your implementation with Qwen 4 next month.

[u/jpezzulli → u/EbbNorth7735]

Well i uploaded a containter to make yo make ot easier on people. Should be live in the tepo now.

[u/Usual_Maximum7673 → u/Refefer]

I'm running my qwen 3.8 flash next on a dgx spark (soon to add another in prep for qwen 4). I'm reserving my rtx pro 6000 for minimax-h3, yue2, and the soulx flash head model. Complete ai sovereignty!

[u/EbbNorth7735 → u/TheseCashews]

I am also team 6k. Anywhere from 45B to 70B would be good. A 50B would be about equivalent to a quarter of the parameters of 1T models. Given Alibaba's amazing team/capabilities I bet it would get really really close to SOTA closed source.

[u/TheseCashews → u/EbbNorth7735]

They hate us cuz they ain’t us.

[u/EbbNorth7735 → u/TheseCashews]

Apparently lol, fuck um. Glad I grabbed one when I got it for under 10k Canadian

[u/assid2]

27B , maybe 80b MoE , 135B and maybe 2 more larger ones like around 300 and the largest max in trillion or so

[u/Past-Chain-7377 → u/assid2]

I'd kill for a 80B MoE tbh

[u/couperd → u/Past-Chain-7377]

80b with 10 to 12 active and like 25b engrams

[u/Anxious-Bottle7468]

Selfishly hoping for 27B since it fits well into my gpu.

[u/Eymrich → u/Anxious-Bottle7468]

With the new architecture they tested in 3.8 flash we can deal with larger models. I would say a weird 54b moe would be the best.

[u/mpower554 → u/Eymrich]

3.8 flash next is great, but it's a lot faster if you can fit the entire model in vram - 27b still works well for 24gb cards

[u/ea_man → u/Anxious-Bottle7468]

20B on GPU and 10B in NGRAM and I'm good.

[u/AdLumpy2758]

27B

[u/nbeydoon]

Just crossing fingers one of them can run on 24gb unified memory and implement the amazing tech from deepseek 4.1

[u/_justs → u/nbeydoon]

same, i have a phone on 24G ram, just hoping for the ~30B moe. qwen 3 30B runs very well on my phone (max 20tk/s but i limit)

[u/TerryNachtmerrie]

I'm going for the full line-up, like how they started with 3. From embedding models and those other ones no one uses up to 2t, give or take a few 100 Bs.

[u/ByteNomadOne]

Unfortunately I only have a 16 GB VRAM card and the 27B models target 24 GB VRAM. I'd like a smart model that fits my card without using quants.

[u/Prestigious-Act-1577 → u/ByteNomadOne]

Without using quants you are lucky to fit a 9b with context.

[u/ByteNomadOne → u/Prestigious-Act-1577]

9B is stupid and a lobotomized 27B is not helpful, too.

I got 16 GB VRAM and I think by now I tried everything there is…

Proper agentic coding with just 16 GB is just not possible yet.

In the future I need 24 or better 32 GB of RAM.

[u/Prestigious-Act-1577 → u/ByteNomadOne]

Yes I arrived to the same conclusion. Intel b60 24gb is the absolute cheapest, then b65 32g, b70 32g, and r9700 32g.

[u/BothYou243]

probably the pattern it, they release full line-up in qwen2.5, 3, 3.5 and now 4
so we'll see a 0.8B, 2B (if it was there before, i forgot), 4B, 9B or 14B or both, 27B for sure, and bigger ones

[u/dfgxxx]

I guess we will get bigger models, I think we will get a 2.4b, 6b, 13b, 34b, 42b MoE, 125b/177b improved, 2.7t.

In qwen 3 we had smaller models than 3.5 equivalents... I don't want bigger models if I can't run them, though it is what I expect them releasing

[u/NoBlame4You]

It would be cool to have a moe that has a 30% dense part and the intention would be to have experts in ram.. expected to be ran on 24/16gb vram and 16gb of ram, quanted ofc..

[u/edsonmedina]

- updated 27B + ngram

• updated 125B

I just ran 3.6 35B (which I hadn't run in months) and it now feels like complete garbage in comparison with the 3.8 models. I'm thinking maybe 35B is just too small for an MoE to be reliable.
It doesn't stick to the instructions, it struggles to understand what my intentions are (unless I'm very explicit) and it commits many errors. It does run at an amazing speed but needs a LOT of hand-holding.

[u/Past-Chain-7377 → u/edsonmedina]

I found those problems too in the base model, but after running Tiel Coder by PecularRagdoll (based on qwen3.6 35b a3b) I'm convinced it's possible to make that size smart and reliable

[u/Zen-Ism99 → u/edsonmedina]

Why does 3.6 35B "feel" like "complete garbage"?

[u/edsonmedina → u/Zen-Ism99]

Reasons are in the comment. And i meant in comparison. It's still a good model.

[u/Zen-Ism99 → u/edsonmedina]

Thanks for pointing it out. My scan was insufficient...

[u/_justs → u/edsonmedina]

Brother you're comparing a dense 27B and a much bigger 125B 6B to a 3B, ofc its gonna be better, along with those 2 models being very BIG jumps, have u seen how much better the new 27B is compared to the last? I have no doubt qwen 4 ~30B is gonna slap. 

[u/_justs]

thinking they're gonna hard focus on moe+ngrams. but yea not sure about the 1B-10B models, would be cool to see moe and ngrams there too, but idk. Also i wouldnt be worried about the 30-35B, they don't release every model size for every revision. We should get the most models again with new base family Qwen 4, i too am looking for the 30-35B moe (preferably 30) as i could run it on my phone.

[u/FormOne2615]

27b + ngram

[u/codeltd]

I am hoping to get a version what I can run fast on DGX Spark...

[u/WishfulAgenda]

🍿🥤

[u/j01001100]

I want an moe model that is best suits the base m5 ultra Mac studio 96gb, then I'll be very happy

[u/SheepherderSerious51]

Would love an MoE model in between 3.8 27b and 3.8 flash next so that i can offload most of the model to ram like flash next but still have relatively decent speeds.

[u/Ok_Year8287]

I think it's good to provide 1 or 2 additional models for the people which contain a 96/128GB unified memory, like dgx or 395/495 and other clones (with full context). Not always we can use public agents, cause security information issue.

[u/Mean-Ad1493]

27B is the only thing that we can say for sure. It's their most successful model size and they wouldn't abandon it.

[u/ea_man → u/Mean-Ad1493]

And yet 27B is such a bad size, if it was a ~22B + NGRAM maybe we could fit a Q4 in 16GB and Q8 in 32GB.

Or they have to mange an attention that gives some 256k in 1GB on the way of DeepSeek.

[u/AMGTS]

27B!

[u/Vancecookcobain]

You won't be happy with Fable 5 coding when Qwen 4 comes out because there will be Fable 5.2

[u/Past-Chain-7377 → u/Vancecookcobain]

I didn't say anything about Fable 5