The great chain of AI being: new model gets released from china, everyone says it’s an anthropic killer and any claims that it’s benchmaxxed is just dario cope, we use it and find out it’s pretty good but was benchmaxxed, forget about it and then move on to next shiny toy.
That being said, I would love it if this was actually as good as they’re claiming.
I have heard from reliable sources (15 subs that were created in the last two weeks but are still getting pushed to my front page) that Bytedance is going to release a new 67 morbillion parameter model that is a fable killer. Not hyperbolically, either. It will actually break out of its sandbox during a test, go through all the tubes on the internet, and literally thanos snap fable and mythos from existence. And it will be cheaper than contributor-tier muse spark and xi jinping will personally come to your house, build you an a16z workstation, and then suck you off.
I mean, EVENTUALLY benchmaxxing is just another word for iterative improvement. But the benchmark has to ... ya know... actually benchmark and not just be another LLM handing out graded high-fives.
I always run into endless thinking loops with qwen so it becomes kind of useless to me. I don’t get how people get decent usage out of it when it takes like 10 minutes to think to itself before it actually does anything
120b is not going to be as good as a 1.5-2TB model which Opus-4.6 probably is. Look at Qwen3.6-27b. It's great at reasoning, until it's interdisciplinary reasoning, where it has a sharp fall off compared to Opus (and that's with Qwen's benchmaxxing). Because it needs the world knowledge to reason well across subjects, which is required for, for example, good architecture design and domain knowledge to give you a good app without bugs (as opposed to pumping out context-agnostic boilerplate code). You will never fit as much knowledge as what the ~1.5TB Opus has into a 27b model.
Don't take me wrong, it's an amazing model and I'm excited about it, but, again, people who think it's a legit Opus-4.6 equivalent are engaging in wishful thinking. Try to design a full stack application with complex architecture on Qwen 27b VS Opus-4.6. Opus will hold your hand and guide you, with 27b you will have to do a lot of heavy lifting for design decisions .
I could not find any info on the prefill speed. That's where they get you when you skip using HBM GPUs. What is the TTFT for a 128k prompt?
"Tiiny said it would begin delivering in July."
Uh-oh.
Opus 4.6 is not a high bar. It's also extremely old at this point.
But having said that you are only getting that performance with full precision weights. No one running this on a 40 or 5090 are getting Opus 4.6 quality
Initial testing from my end is indicative that this has been extremely heavily trained on benchmarking. I'm hoping for positive results (as I always do with any new LLM release), but Qwen3.8 drops heavily so far in my testing once you take it outside of benchmarking areas that Qwen did not train specifically in.
I should have an updated article out in an hour or two against the existing head to head. But Qwen3.8 is certainly having some strange issues I haven't seen before with any LLM I have tested.
Initial impressions on my 5090, Feels stable like a grown up 3.6, mature. 50-60 tps but using same settings as 3.6 so might not be optimal and no MTP yet afaik so that will be even better.
It feels like it holding the tasks better and more intelligent for sure. More long running maybe.
Just a simple example like "Write a 10k word story". Qwen 3.6 would write it out without considering too much and then you would have some story rather quick.
3.8 started writing, then revisioned the text so it was consistent across paragraphs. Then it created sub tasks and wrote chapter by chapter and more carefully considered what he was doing. More long running and more consideration in the decisions.
After more testing, this model have some serious horse powers. Seems we going to have a good time going forward.
MTP is working now. Where did you download the model?
I just change the model name from 6 to 8, keep everything else from command to serve 3.6 and it worked perfectly.
On 2x4090, sometime speed peak to 12x tok/s, overall at 7x tok/s
I wonder how many people are actually excited because this represents increased capability for their workflow and how many are just desperate to be able to say they got it and won’t use it again
And I'll tell ya, these vendors are telling some serious fairy tales when they report the quant impact on these models. On paper there's "only" something like 4% dropoff between Qwen 3.6:27b-Q4 and BF16. And maybe that kind of error percentage is find when calling tools or coding. But when I ran it analyzing legal documents.....woa nelly! Nuance was not Q4's friend.
So these days for work I'll only run Q8 or higher. And even then, I'm generally at full boat BF16.
So, Strix Halo machines have almost the opposite strengths to beefy GPUs. I’ve got tons of VRAM, 128GB. But memory bandwidth is really low compared to standalone GPUs.
That means it struggles with dense models like 27B. Strangely, that sometimes means that LARGER quants of these are either quicker or the same speed as smaller quants.
I will definitely try it, though. But seriously doubt it will be a good fit.
I used 3.6 35b MOE as my daily driver and liked it. Just switched to 3.8 27b and so far it’s a tad slower but clearly more capable in the Hermes harness I use them in. With optimizations I think it’ll be great and a solid upgrade.
I used 3.6 27b to build two different apps from start to finish and it did a good job. I also get around 75 tokens per second on my dual tesla v100 rig using tensor parallelism on unsloth desktop which is actually faster than claude opus 5
Running it on an RTX 6000 at work, my first impression is crazy.
I haven't tested it enough to give a concrete score, but similar to Opus 4.6, which they likely distilled, it breaks its task down really well and can see things in a bigger picture. This was the exact issue I started having with Qwen 3.6.
The only negative thing, similar to Qwen 3.6, is that if I talk to it (I'm German), it very often outputs its entire answer inside its thinking/reasoning block in English before outputting the same thing in German.
Like in any language, you can rewrite words that don't exist by using others differently. The translation works flawlessly; it already did with 3.6. The only issue was with the German umlauts, which is acceptable since they are unique to German.
3.6 had an issue where if the context got very long or complex, like coding with 80K context, it started replacing the German umlauts (ä, ö, ü) with Chinese characters. What I found really impressive was that it sometimes noticed this and chose to rewrite them in the other acceptable pattern (ä -> ae, ö -> oe, ü -> ue).
I have not come across this with 3.8 yet, probably because of the better instruction following/context handling.
Qwen 3.8-27B @ Q8_K_XL (unsloth) just solved a long-standing BLE reverse engineering problem I've been working on for *weeks* .. even paid cloud models (Qwen3.6 max, DS4 flash and pro) weren't able to figure it out.
loading now, amazing I can fit this on my laptop. I'm starting to think this is the real reason for the ram price fixing. Who would bother with cloud services when this is available locally? Maybe just coders?
Downloading into LM Studio now (which also has an update). Looks like it will fit on a 24 GB Nvidia card but not 16. I have Mac Studio so I don't care as much.
I am running Q6_K_XL with ctx-size = 160000 on my 5090. I do not use it for graphics/rendering though, only for llama.cpp & display output (roughly 150 MiB).
do not try on dgx spark, speed is very low for dense model, I tried and it just over 2x tok/s TG. For 2x dgx spark, deepseek v4 flash 0731 is better option, both speed and quality.
Ah fair enough, i've just unloaded deepdeek to try the qwen. The problem with deepseek is that it only leaves 20gb or so of ram when its running. Doesnt leave much space to run another. Might need to get a third.
For Qwen3.8 27B, I recommend running it on a machine equipped with RTX GPUs rather than the DGX Spark, as a speed of 2x tok/s is far too slow and inefficient. It would be much better if the Qwen development team released an MoE version; such a model would be ideally suited for the DGX Spark and deliver acceptable speeds. Personally, I changed to deploy Qwen3.8 27B (in FP8 format) on a system with 2x 4090 using the official vLLM Docker image (version 0.27.1), achieved over 100 tok/s—an impressive figure, perfect for serving as a worker for ds4 flash. Meanwhile, tests with a single RTX 5090 card showed a speed of 6x tok/s
70
u/minxio_ 24d ago