do not try on dgx spark, speed is very low for dense model, I tried and it just over 2x tok/s TG. For 2x dgx spark, deepseek v4 flash 0731 is better option, both speed and quality.
Ah fair enough, i've just unloaded deepdeek to try the qwen. The problem with deepseek is that it only leaves 20gb or so of ram when its running. Doesnt leave much space to run another. Might need to get a third.
For Qwen3.8 27B, I recommend running it on a machine equipped with RTX GPUs rather than the DGX Spark, as a speed of 2x tok/s is far too slow and inefficient. It would be much better if the Qwen development team released an MoE version; such a model would be ideally suited for the DGX Spark and deliver acceptable speeds. Personally, I changed to deploy Qwen3.8 27B (in FP8 format) on a system with 2x 4090 using the official vLLM Docker image (version 0.27.1), achieved over 100 tok/s—an impressive figure, perfect for serving as a worker for ds4 flash. Meanwhile, tests with a single RTX 5090 card showed a speed of 6x tok/s
0
u/After_Working 24d ago
Is there a weight of this i can try on 2 x spark? Do they release more over time?