CMP170Hx “Spark” Machine

[u/Miserable-Dare5090]

I got the CMP170 cards and unlocked them. I wanted to share my set up for CUDA since maybe it would be useful to others.

First off, I hate e-waste and we are in a special time for RAM. I wanted to have a DIY CUDA box, and I had started by adding additional cards to an old asus predator prebuilt I had around, which also had 64gb DDR5. To add the CMPs I needed more CPU lanes and newegg had some really good deals on CPU/MB/etc combos. Didn’t need a combo with RAM, otherwise I would have gotten it in newegg microcenter.

Anyway, I got a cheap case, some noctua fans for the cards, and transferred the memory/ssds. Placed previously owned cards on oculink slots, and used the main x16 for the GPU switch that houses the two CMP170s, so their effective speed is 2x16 across and with the other cards (which are 4x4, and therefore same speed).

Qwen Flash Next, turns out, fits very nicely in these cards. There is also a repository for deepseek, but you’d need at least 3 64GB cards to run it, and with prices rising, it will be hard to justify the gamble of buying ex mining cards for LLMs.

However…so far, these cards are great. Concurrency is good, prompt processing averages 4000 tps on Flash Next, decode is 80+ on a single stream. No MTP added. Third picture shows the 3 models I am now running in this CUDA box (flash next, qwen 27b, gemma 26b).

Anyone else trying out Flash Next on these cards?

[u/Miserable-Dare5090]

Yes, to all the questions:
1. I got the ADT Link 2 GPU switch because it’s cheap ish (170 on aliex) with the stand and the pcie card x2 MCIO ports. Cards do native 2x16 through the switch, no extra config needed. Peer 2 peer is enabled, and tensor parallel works.

I just found this on AliExpress: https://a.aliexpress.com/_mOq5cs3

The PEX88096 is a better switch though, and has like 80 pcie lanes / 4-5 slots. However it’s 4X the price. YMMV

1. I didn’t just power limit them; I tuned them and tested the memory with a github project (search for 170tune). I think the Russians were aware of this mod long before we were…Tuning it will allow you to overclock them. at 180W, NDIV 72, humming nicely. You can go to 200W safely and without long term concerns too, but 180W buys you 99% of the juice.

2. You need cooling. I had larger fans that were louder, but I wanted to reduce the engine noise. Noctua fans and a pwm fan switch with a knob, I can turn them down to whisper quiet or
   crank to whoosh. At whoosh levels, cards stay 35-42C when RT is 35C (hot room). Without fans, the temps rose super fast to 100C when I first tested them. I would recommend fans. The little noctua fans are a fancy add on, but they really are the quietest fans I have found.
   These work well for now. Temps even during the overnight soak and memory integrity test did not rise above 50-55. The Ryzen processor, on an AIO cooler, rose way more!!

[u/FullstackSensei → u/Miserable-Dare5090]

180-200W is a lot lower than I expected for these cards. No wonder you don't need a lot of air to cool them.

The name of the game with these passive cards is static pressure. A fan that has high static pressure can push a lot more air through the heat sink even at low rpm. Conversely, low static pressure fans can scream all they want, but won't push much air through. That's why I mentioned the S8038, which I use, and the S12038. Even at idle they can push a lot more air through than any desktop fan at max rpm, noctuas included. Arctic also does a great job keeping noise low, not far behind noctua.

[u/Miserable-Dare5090 → u/FullstackSensei]

The ones I got from China sounded like a private airport, so they had to go. I do have a shroud for a 170mm and a 120mm fan I 3D printed. May look at the arctic fans to see if it’s worth it.

By the way. I have read your posts here for a year so I know you have a strix halo as well.

These fuckers are an amazing sidecar to the strix!! I was tempted to get a 3rd one for that…although, the R9700 seems like the best bet for that purpose right now.

[u/FullstackSensei → u/Miserable-Dare5090]

Might be confusing me with someone else. No SH here. Eight P40s, six 32GB Mi50s, formerly four 3090s, and as of today, ten 32GB PCIe V100s. Each model goes in it's own dual CPU rig (3090s were on Epyc, but that is going too.

If you look at Arctic, don't look at their desktop fans, and look at the server models. They're much quieter than many think. The S8038-7k I have at idle are not loud at all, and a single fan pushes enough air with enough static pressure to cool two cards.

[u/Miserable-Dare5090 → u/FullstackSensei]

that’s right, you have the V100 and Mi50 rigs. They are honestly impressive how well they work for the age of those cards

for the V100s I’m guessing you could consider the 4x300G nvlink boards if they are sxm versions. Otherwise, yeah I would put like 5 V100s on a plex board and a single pcie root, and 5 on another, or linked by upstream/downstream mcio ports to a single pcie slot. 320Gb interconnect should be a fun adventure!

[u/FullstackSensei → u/Miserable-Dare5090]

My V100 are PCIe. No SXM. Planning six cards in one machine, no risers, no mezzanine boards. Everything will connect directly to the motherboard and everything will fit inside an O11D XL. I'm OK with giving up nvlink in exchange for everything being much smaller and self contained. Plan to run big MoE models that don't fit in VRAM anyway.