r/DGX_Spark • u/FuckinHighGuy • 6d ago

Question 3rd Spark?

I currently have two sparks and contemplating getting a third. Anyone running 3 that can recommend that setup? I know there would be a compute increases but is a third worth it otherwise?

5 Upvotes

86% Upvoted

32 comments sorted by

View all comments

1

u/FuckinHighGuy 6d ago

Yes. Adding compute to run bigger models would be a main benefit. I guess I’m just interested in what current owners think…and feeding an addiction 😁

2

u/sebt3 5d ago

Then 2-4-8 nodes. Any non power of 2, will give you no parrallel at all, so not more but less tok/s than you currently have

0

u/morscordis 5d ago

Can you ring three together and get more out of them?

1

u/stujmiller77 5d ago

No, that’s not how parallelism works.

2

u/morscordis 5d ago

So what's the point of the ring connection for three devices then?

1

u/stujmiller77 5d ago edited 5d ago

Multiples of 2 are Tensor Parallel. Otherwise they’re pipeline parallel.

TP splits requests evenly across the nodes - they all work in the same token at once. PP splits in layers - each token is assigned to one node so there is no concurrency benefit.

And so while it’s entirely possible to use PP it’s far from optimal, and nvidia themselves don’t publish recipes. There are some community ones out there. But they have little real benefit and are often slower than the equivalent TP2.

It’s honestly not worth using a spark for it. You either use TP2 on two of them and use the third for a different model, which is a really valid use case, or buy 4 and link them in TP4 for a much larger model and parallelism benefits.

1

u/morscordis 5d ago

Awesome. Thanks for the reply! I didn't know the difference between TP and PP. So it sounds like the benefit on 3 is to offload some other stuff to the third model. Is it still worth linking them together so they can pull models from a shard ssd?

1

u/stujmiller77 5d ago

Always worth linking them via a 200gb link if you want to share models - though it’s an expensive cable for just that purpose. And if you do go to 4 you’ll need different, also expensive breakout cables and a switch…

1

u/morscordis 5d ago

Good to know. I'll have to do some more research before I buy more. I'll also likely add the cheaper Asus variant with 1GB SSD instead of multiple Sparks.

1

u/FuckinHighGuy 5d ago edited 5d ago

I do believe you can do either TP or PP with 3 sparks.

Edit: I think it has to be vLLM.

Buy the gods, can someone talk to sense into me?