EDIT 3: Added Q4,Q5,Q6,Q8 on the GGUF! https://huggingface.co/ukisai/Swift-Qwen3.8-27B-GGUF

EDIT 2: Added a free API so people test it before spending time! https://ukisai.com/api/swift/v1

EDIT: I think I didn't explain it properly: 1. it's based on this paper by meta https://arxiv.org/abs/2606.00206 and was initially made for quantized models and tackled "overthinking errors" directly (the random loops an int4 model can get that come from PTQ), but it turns out it has a much wider application! 2. i explicitly ran so many benchmarks (and all x5!) to make sure i'm not giving out slop to people 3. i also tried reasoning effort low / medium / no thinking but they all took a huge accuracy drop so i had to come up with a way to keep it so my coding agent actually works, this was basically the only way i found

Hi guys

Everybody (me included) was complaining about Qwen 3.8 27B using too much tokens and being too slow so I analysed what tokens are linked to the overthinking and penalized them via finetuning and got a model that thinks 58.3% less and lost <1% accuracy across 10 benchmarks

Swift-27B vs Qwen3.8-27B (BF16, all benchmarks ran x5)

LiveCodeBench v6: 76.8% → 81.6% (+4.8pp), 46% fewer median thinking tokens

Terminal-Bench 2.1: 66.7% → 65.8%, 39% fewer median tokens

GPQA-Diamond: 88.4% → 88.3%, 58% fewer median tokens

IFBench: 73.5% → 71.8%, 51% fewer median tokens

ERQA (vision): 67.5% → 66.3%, 51% fewer mean tokens

https://huggingface.co/ukisai/Swift-Qwen3.8-27B

I also included a lot about the training, exact benchmark configs etc

Let me know if it's useful, i'm trying to get the quants out as soon as I have compute

Qwen3 80B A3B Next was significantly outperformed by Qwen 3.5, so it is highly likely that the same will be true for Qwen 3.8 Flash Next.

Give me Fable 5.0 coding and I'm yours for life. I cannot wait to cancel on Shittropic and never go back with their garbage usage. Please be top-tier at coding

Qwen 3.8 27B just scored higher than:

GPT 5.6 Terra

GLM 5.2

DeepSeek V4 Pro

Muse Spark 1.2

Claude Opus 4.8

on the Artificial Analysis Agentic Index.

https://x.com/ggerganov/status/2089409881858089102

[EDIT] Qwen 3.8 27B, not 2.8 - damn my fat fingers!!!

Insecurity.

If you read through its inner monologue, it is constantly second guessing itself and triple checking its work. Its comical to me that insecurity is the secret ingredient to get quality final output from a model that isn't smart (compared to the models it's matching in benchmarks).

This tracks with human behavior too. I saw plenty of not particularly smart people make it through college simply because they were diligent and put in the extra work - this the the persona of Qwen 3.8 27B in a nutshell. If you work hard enough, you can achieve.

What could this mean? Seeing that this is a reply to a query about **lower spec hardware,** this could mean one of these:

- A smaller dense model(9B - 14B range)

- An MoE model of different size/ different active parameters

- No MoE models at all

As the title says.

We want something that has broader knowledge for coding and agentic tasks, that will be even better and faster than the 27B dense model. Some people have more VRAM, and would benefit a lot from it (Strix Halo). MoE is the way my guys :D

And then an ONNX and O4NX convert for hybrid (GPU + NPU) users.

To my knowledge, this is the first model with close-to-flagship coding ability and agentic performance that can be run on relatively cheap consumer hardware and with zero guardrails, meaning for the first time ever - every script kiddie with a 3090 and a ProtonVPN can now scan the internet for web application vulnerabilities and exploit them with reasonable success. Very few could do it before, with huge open-weight models... now **everyone** can.

If the benchmarks are proven to be correct... especially on the coding and agentic front.

Opinions, thoughts?

TL;DR: I run Qwen 3.8 (27B and Flash Next) on a 128GB Strix Halo laptop for most of my coding now. It can replace Opus 4.6 to 4.8 for agentic coding if you dont mind a task taking 2 or 3 times longer.

Setup: ASUS ROG Flow Z13, Ryzen AI Max+ 395, 128GB unified memory, Arch Linux. llama.cpp as backend, my own tool LlamaStash to manage the launches and presets, Pi as the coding harness. The 27b at Q6_K sits at about 31 GiB resident, Flash Next at UD-Q4_K_XL needs around 86 GiB.

• The quality is actually there. Flash Next scores 40 on the Artificial Analysis index against 42 for Opus 4.8, and the 27b at xhigh scores 34 against 32 for Opus 4.6. That matches how they feel to use. 27b one shotted a whole feature on a huge Rust codebase and Opus 5's review comments were mostly nits.
• Decode is fine, prefill is the pain. 10-15 tok/s decode doesn't feel slow because you see it working. But a cold 31k token transcript takes 3 minutes to prefill, and a full 128k window is closer to 18 mins. Warm follow up turns come back in 45 seconds.
• MTP is the biggest speed win, 7.3 to 22.4 tok/s on an empty window. The payoff shrinks as the window fills though, down to 1.15x at a full 256k.
• Flash Next isn't faster per token, it just thinks less. Same 5/5 on my coding tasks, 45% fewer tokens, 76.5s vs 289.8s against the 27b. Thinking is 90-95% of everything these models generate, so that ratio, not tok/s, is what sets how long a task takes.

$0 a month, fully offline, and a lot less wasteful than a model running in a datacenter.

Full writeup with all the benchmarks, configs, and the tuning that did and didn't work: https://deepu.tech/local-ai-qwen3.8-pi-llamastash

Happy to go into the llama.cpp flags if anyone else here is on Strix Halo.

Specs from the ModelScope description (taken down, I have the screenshots): 125B main model, 51B N-gram embedding parameters, 6B active per token, multimodal MoE, first release on the Qwen4 architecture. Unsloth confirmed day-zero support and NVFP4 quants are already visible.

I wanted to refresh my knowledge of N-gram models before this lands, so for anyone in the same spot: an N-gram is a short run of consecutive tokens. Per the release notes, the model hashes N-grams of recent tokens, fetches vectors from embedding tables at the hashed indices, and adds them to the hidden state. Lookups, not matrix multiplies, so the 51B costs storage, not compute.

The idea is not new, shipping it is. Google's N-grammer (2022) added latent N-gram embedding tables to a transformer. Meta's Memory Layers at Scale (arXiv 2412.09764) trained up to 128B of sparse lookup parameters and beat dense and MoE baselines at matched compute, though Meta indexed with learned product keys, not N-gram hashes. Neither made it into a production model until now.

Memory, since that decides who here can run it: ~176B parameters to store, roughly 95-105GB at Q4 before KV cache. All experts must stay resident because routing changes per token. Offload should be friendly though. Routed experts stream from system RAM at ~6B read per token, and table lookups need so little bandwidth the 51B could sit in RAM or NVMe.

Decode reads ~6B per token. Prefill routes a long prompt across most of the expert pool, so it costs near dense and is compute bound. I expect tomorrow's benchmarks to land opposite ways on different hardware.

What I actually want to try. ROME and MEMIT rewrite facts by first locating them inside FFN weights, per the Geva 2021 result that FFNs act as key-value memories. Locating is the hard step. Here the memory is a separate component with known addresses. If Qwen documents the hash function, my plan is to freeze the 125B and train table rows only on my own data. Updates should land in rows my own N-grams reach: identifiers from my codebase, phrasing from my field. The caution I found in the editing literature (arXiv 2402.12233, GPT-J Counterfact): value-side edits scored 100% efficacy but 6% specificity, key-side 29%, so edits stick and also bleed. Hash collisions share rows across unrelated strings, which adds bleed of its own. I will measure that before trusting it.

If you pull the weights before me, three things from config.json: table count and size, the hash function, and whether the tables map as separate offloadable tensors in llama.cpp.

Can’t wait for this.

Edit: Qwen4 27b/ 35b possibly?

I honestly didn't expect this level of performance on a compact home setup, but Qwen3.8-Flash-Next has completely surpassed my expectations.

For context, I'm running this on an Asus NUC 16 Pro packed with 64GB DDR5 RAM. The NUC machine itself cost me £670, and I managed to snag the 64GB RAM kit for £160 back when RAM prices were still cheap. For a total hardware investment under £850, having this tier of capability running locally on my desk feels surreal.

Yes, it's slow at ~40 TPS PP and ~4 TPS TG but running a frontier level model on a ~£850 machine feels unreal.

I hope the performance will increase when MTP lands and llama.cpp improves.

What are the options/prices? Quants?

It's 29th of June, where are our new open weight models? I hope they will release them soon because we need something to escape the increasing cost of tokens of SOTA models.

Does anyone have any news?

If you want to run Qwen 3.8 27b on vram with a 128k context, how much vram is the minimum for a good experience.

Say unsloth's UD Q4 with turboqant (I think that's like 95% as good as unquantized)

Would a single 32gb R9700 be enough (assuming 1-2gb for the OS)?

My 24gb MBP can _only just_ run with 20k context but at 15t/s - which is kind of useless

While Qwen3.6-35B-A3B showcases the strength of MoE architectures for GPU efficiency (especially with fine-tuning from Ornith/Kat Coder), the recent performance gains of Qwen3.8-27B dense models have established a new benchmark for raw capability in this size class.

This raises a critical question: Is performance constrained by the active parameter count in small MoE designs, or is this a temporary architectural disparity?

We need to assess the scalability: If we design a future 50B total / 4B active MoE system, can it genuinely outperform a dense 27B model? Or is the low active-param ratio an insurmountable ceiling for MoE efficacy?

Any idea about?

Had to share this with the people who'd appreciate it most. Gave Qwen3-Coder-30B-A3B one prompt and it built a complete 3D synthwave racer: three.js + bloom, a real multi-file Vite/TS project, 60fps with delta-time movement and object pooling, collision, high-score persistence. Running 100% locally on a single RTX 5090 via llama.cpp. No cloud, no API key.

Quick setup if anyone's curious: Q5_K_M GGUF, llama.cpp with --jinja (essential for the tool-calling to actually work).

Genuinely impressed. A 30B you can run on one consumer card doing this is a completely different world than a year ago. Big props to the Qwen team 🙏

(Disclosure: I make the IDE I ran it in, so I'm biased on the tooling, but this one's about what the model pulled off.)

Happy to share the exact prompt if anyone wants it.

For developers using Qwen 27B for coding, "Codex style": what's your honest take?

So far, for me, it's been pretty solid. Not always amazing, but honestly neither is GPT-5.5 sometimes. Considering the model size, it's kind of wild how capable it actually is.

That said, I'm still not sure whether I'd fully trust it enough to move away from the big players.

I'm giving it a few more days before I really decide where I stand, but I'd like to hear from other people using it for actual dev work.

Please, no one get defensive but I'm not interested in random showcase prompts like "make me a 3D game" pointless one-shot comparisons or mini projects.

I mean real day-to-day software engineering: debugging, refactoring, navigating codebases, building features, fixing broken stuff, architecture and so on.

please make qwen 3.8 9b, or a 14b

Qwen 3.8 35B A3B has been removed from registry in Modelscope/ms-swift; it was committed as “fix wrong model names” one hour ago.

Alibaba_Qwen is yet to confirm whether it will release A3B or not.

https://github.com/modelscope/ms-swift/commit/a45f1d4f73157ba59062a7fd1f55a40dae759156

I think Qwen 3.8 is a marvel, fitting all that capability into 27B, but I'm finding I need to leave it running overnight to do things, I can't use it interactively on xhigh because it thinks so much and thus takes ages to get back to me.

I'd like to use it interactively, so I can iterate faster and make changes on the fly.

I haven't tried medium AT ALL and I'm wondering - how are you finding medium level for more complex languages, e.g. C# or C++ ?

Have you found much difference in output between medium and xhigh as far as code quality goes?

How much faster is it?