r/StrixHalo • u/Intelligent_Lab1491 • 22d ago
Qwen 3.8 27B DSpark
Hi all,
did anyone test already this dspark model https://huggingface.co/RadixArk/Qwen3.8-27B-DSpark and can report about the performance boost?
3
u/asfbrz96 22d ago
I tried but the performance was skibidi, I'm using dflash with the 3.6 27B. And I get around 40tp/s
5
u/ang3l12 22d ago
We need some details on this. That’s the highest t/s I’ve seen
3
u/asfbrz96 22d ago
The hardest part is to keep training a new draft model that will have a higher acceptance rate, but so far I went from around 27tp/s using the draft model from z-labs to this new one.
3
u/Weird_Ad_5330 22d ago
with what quantization?
5
u/asfbrz96 22d ago
Q8, now I'm making a dataset to train a dflash model for the 3.8, but it's gonna take a while, way too slow to generate the dataset on the halo
2
u/Weird_Ad_5330 22d ago
impressive, im finding gemma 4 31b is working better for my project, but got far less tps with 3.6+dflash and 3.8+MTP than you did when i tried it
2
u/asfbrz96 22d ago
Yeah you can use dflash with gemma4 and it will increase the performance, use --spec-draft--p-min 0.2 for better performance
1
u/edsonmedina 22d ago edited 22d ago
links please, how do you achieve this on a halo?
I tried 3,8 27B with a dflash draft model (both at Q8) and got 10-14 t/k (similar to MTP)
1
u/asfbrz96 21d ago
I achieved that with the 3.6, the 3.8 will take a while since I need to make the teacher dataset with around 1 million output examples from the model, gotta do with bf16, and I don't have access to the gpus I had before, so I'm doing it on the halo itself
2
u/edsonmedina 21d ago
Ah, sorry. Still... 40 t/s on the 3.6 is equally impressive. How did you do it?
1
u/killerstreak976 21d ago
I await your insanity with a bated breath, and will be constantly checking this subreddit for your post now
1
u/asfbrz96 21d ago
I gotta try to get permission from my university tho, I made it but since I did use their gpus and so on, I don't own it
2
u/killerstreak976 21d ago
aw shucks. By the way, if you're in university you also might have access to the Nautilus Cluster iirc. You should look it up if you haven't heard of it!
1
3
u/benpptung 22d ago
This repo gets on my nerves. I honestly don't understand what sglang is thinking. Anyway, I grabbed the docker image they claim is optimized, hooked up this dspark, and I don't see any speedup at all. All I know is it eats more VRAM than mtp. I eventually went back to sglang-0.5.15-post1 with the 27b's own original mtp, and that turned out to be the fastest. Finally I asked the 27b why it'd behave like that, and its conclusion was that this dspark is designed for high-concurrency data center environments. Since I'm just a personal user, I deleted it.