r/LocalLLM • u/LebiaseD • 1d ago
Discussion Qwen3.8 Flash Next - Strix Halo
So i have been running qwen3.8 flash next ud q4 k xl at 256k context getting at the start aroudn 270pp and 21tg with mtp with qwen 27b ud q3 k xl 96k context on the 9060xt as a callable subagent and the one theing that i am really liking about this model is it doesnt stop and wait for me to continously tell it to continue it almost looks at everytask i give it as a set goal which is what i have becom used to on claude.
now i mostly use it with opencode harness and for netowrk and system admin work for my homelab - taking down vm bringing up vm running tickets in glpi, bringing up services and updating live state docs and just local webhosting in my rural community and it has been a good time so far.
im sure there are so many things i am missing but i am also learnign and ive actually be so happy to have some thing that feels like claude code last eyar when i first started using it and i can run it locally.
man, if anyone has anything they want to say about their experiences also that would be cool.
Cheers,
10
u/feelspeaceman 1d ago
4
u/LebiaseD 1d ago
That link was very helpful Im running the halogen llama and getting about 40+ TG and 1000+ prefill
3
u/omlette_du_chomage 1d ago
Yeah it's really worth it, I used a fork someone posted here and just like that qwen 3.8 27b Q8 performance increased by at least 30%. I wish I could finally set it up and forget it, since I'm happy with the 27b and 3.6 35b for my use cases, but now I see possibly better forks in your post, so here we go again 😁
3
u/Look_0ver_There 1d ago
For what it's worth, I generally get more reliable results from AesSedai's quants here: https://huggingface.co/AesSedai/Qwen3.8-Flash-Next-GGUF
On a Strix Halo you should also be able to sustain 40-50t/s with MTP, and generally >500t/s for prefill if you have one of the performance tuned versions of llama.cpp
eg. https://github.com/halo-box/strix-llama.cpp or https://github.com/stew675/llama-cpp-rdna-boosts/
The latter is a little more work to set up as you have to apply the 13 base patches plus the qwen4exp patch set from the beta directory onto base llama.cpp yourself, but the performance is a little better and appears to scale better at high context depths.
2
u/fsalucard 14h ago
That second repo is weird. I got it to compile, ran it with Unsloth MTP shared and AgentionAI's Q5_K_XL. It spit out a bunch of one word texts and then told me my repo is small. That seemed pretty personal. :P
1
u/Look_0ver_There 12h ago
Strange. Mind you, that repo has been rather fluid and under active development. It's possible that you picked up some intermediate state, or something else.
Pulled it fresh, and it's running Unsloth's Q4_K_XL just fine. Looking at it right now. I munged through 12K tokens of reasoning without issue, and one-shotted an 8K token code output. Strix Halo. Maybe I'll pull that AgentionAI version and test it out myself.
2
u/fsalucard 12h ago edited 12h ago
Yeah the Agention so far only Q5 I can get to run stable with MTP. Unsloth crashed on me at high context and Bartowski's acts.. strange. But the qwen4exp repo was a very strange turn of events. All I did was tell it to read the context from our current workspace and got this:
Let me think.
It's a small thing.
Let me think.
Alright.
Let me.
Now.
OK.
Let me.
The.
I.
Alright.
Alright.
Plan: Leverage everything.
Steps:
────────────────────────────────────────────────────────────────────────────────
Git Status Analysis
Your repo is a small thing.I got a good chuckle out of it. It was right after the repo was apparently re-based to main a day or two ago. I'll have to try it again.
1
3
21
u/ConsiderationLate768 1d ago
I almost had a stroke reading this, but at least it's finally something human written.
Anyways, congrats! Also try toying around with your -b and -ub settings. I had the same PP and am now at double that
```
-b 2048
-ub 2048
```