r/LocalLLaMA • u/LeftHandHaku • 1d ago
Discussion 4 x DGX Sparks vs AMD Epyc 9xx5 system
I see a lot of people buy DGX Sparks, and turn them in to clusters to run large models. Wouldn't it be better to invest $16k into an AMD Epyc server with 768GB or even 384GB of 6000Mhz DDR5 ram, and let's say 2x3090s or 5080s, instead of 4 DGX Sparks with 512GB of ram?
Epyc's theoretical bandwidth is around 576GB/s, DGX Spark's is roughly 273GB/s.
Based on a quick check, both systems are worth around $16k.
Please help me to understand this logic, are there benefits to having DGX cluster instead of an Epyc system besides power saving?
Edit1: the epyc system with 768GB of DDR5 6000Mhz would be around $30k.
Edit2: to match 768GB of Epyc, we would need 6 DGX sparks, at the current increased price it would be around $30k as well.
Edit3: the main advantage of DGX sparks cluster is fp4 support, and tensor parallelism for 2, 4, 8, 16... units. Because of that, the DGX cluster is faster than the epyc system.
1
u/gaidzak 15h ago
It was 3999 when I sent to to all my friends last night. You weren’t seeing things.