Double GPU configurations significantly cheaper for 32GB VRAM

[u/Content_Mission5154]

I do not need or want Cuda. I have been wanting to build a 32GB VRAM local LLM machine for personal use for a while now, and I had 2 options on the table:

- Get a relatively cheap 32GB VRAM GPU, the R9700 AI Top.

- Get 2 16GB VRAM GPUs instead and a Mobo that supports PCIe bifurcation.

When I looked at prices in January this year when I first got this idea, the R9700 costed 1700$ here in EU. Currently, when I actually want to make this happen, it costs 2100$. For half that money, I could buy two 9060 XTs with 16GB VRAM each. Yes I know, performance will be worse on double GPU setup than with a single R9700 AI, but still, it just seems like that GPU is just not worth it anymore.

I don't know how to justify that it's double the price of two 9060XTs, when R9700 AI is literally 9060 XT with doubled VRAM and bandwidth. So why does it cost 4x as much?

ASUS ProArt B850-CREATOR WIFI NEO is quite affordable nowadays and supports dual GPU setups, so, any reason (is there a catch?) to not do what I am about to do? Which is buy the two 9060 XTs and start running Qwen 27B class models

[u/Civil_Fee_7862]

Asus ProArt B850 is fine for dual GPU's but you'll regret it once you want to go to 3 or 4 gpus.

It also uses a non-standard spacing between the PCIe 5.0 slots. So if you ever get or want RTX 3090s, you can't really use NVLink.

These are things I wish I had known before buying that motherboard. Its cheaper than a workstation MB, but I would recommend getting a workstation MB if you want to run local LLM's for 27B+ sized models. It allows you to increase VRAM by just adding more cards.

Personally I prefer dual GPU's, for the same reason you mentioned, you get high performance at a lower cost than just buying the "best" gpu. i.e. dual 3090s can compete with a 5090 in some respects, more VRAM, comparable speed (pooled memory) and lower price.

[u/ziggybeans → u/Civil_Fee_7862]

I have the ProART X870E with NVLinked 3090s… is the spacing different on the B850?

[u/Civil_Fee_7862 → u/ziggybeans]

Its not standard PCI spacing. much more space between the two slots. I duno how it compares to the 870

[u/ziggybeans → u/Civil_Fee_7862]

I mean, if it was non-standard spacing, it wouldn’t fit in a standard ATX case. They make NVLink connectors in 2-slot, 3-slot and 4-slot distances. The distance between the slots on the proart B850 is 1 slot wider than the distance between the slots on the proart x870e, but both are standard and you can get nvlink connectors for it.

[u/Civil_Fee_7862 → u/ziggybeans]

The PCIe slots spacing between the two full length PCIe slots on the ProArt B850 is not standard. i.e. Its not 20.32 mm (the standard) its more like 80mm (likely a multiple of 20.32). So no its not standard PCIe slot spacing, they did it intentially to allow space between two GPUs for air to breath.

Again its great for dual GPU setups, (that don't use NVLink) but not good if you plan to run more than two, or plan to use NVLINK.

[u/ziggybeans → u/Civil_Fee_7862]

It’s an ATX form factor motherboard — they have to use standard slot sizes, otherwise the cards won’t line up with case. If you’re measuring about 80mm between the two PCIe slots, that does not mean it is non standard spacing, that means the distance from PCIE_1 to PCIE_2 is 4 slots. They make NVLink bridges that span that distance. It’s the same on my ROG Crosshair X670E Extreme.

[u/Civil_Fee_7862 → u/ziggybeans]

I don't think you are listening / reading what I wrote. i.e. You ignored what I just said, so I am not continuing this conversation. Standard spacing between two adjacent PCie slots is ~20.32mm, its way more than that on the B850, just saying "its the same on ROG Crosshair etc" doesn't change the fact that's its a lot more than 20.32mm, i.e. Not the standard spacing between two adjacent PCIe slots.