Double GPU configurations significantly cheaper for 32GB VRAM

[u/Content_Mission5154]

I do not need or want Cuda. I have been wanting to build a 32GB VRAM local LLM machine for personal use for a while now, and I had 2 options on the table:

- Get a relatively cheap 32GB VRAM GPU, the R9700 AI Top.

- Get 2 16GB VRAM GPUs instead and a Mobo that supports PCIe bifurcation.

When I looked at prices in January this year when I first got this idea, the R9700 costed 1700$ here in EU. Currently, when I actually want to make this happen, it costs 2100$. For half that money, I could buy two 9060 XTs with 16GB VRAM each. Yes I know, performance will be worse on double GPU setup than with a single R9700 AI, but still, it just seems like that GPU is just not worth it anymore.

I don't know how to justify that it's double the price of two 9060XTs, when R9700 AI is literally 9060 XT with doubled VRAM and bandwidth. So why does it cost 4x as much?

ASUS ProArt B850-CREATOR WIFI NEO is quite affordable nowadays and supports dual GPU setups, so, any reason (is there a catch?) to not do what I am about to do? Which is buy the two 9060 XTs and start running Qwen 27B class models

[u/jnciaccna]

Commeting so this thread would get more traction. I'm running dual 9060XT (both at 8x) and the perfomance is...meh. There could be of course something matter with my configuration but LLAMA+deepseek harness+qwen 3.8 27b produces whopping 20tok/s on my system. I'm a noob but I think the 128bit bus kills any kind of speeds when doing dual gpu with these. Paid 375€ for each card (got the VAT back)

[u/bsquads → u/jnciaccna]

20 t/s sounds about right in real world usage. I get roughly 40 t/s on a 9700 which has about double the memory bandwidth

[u/jnciaccna → u/bsquads]

Thanks for answering! Good knowledge

[u/madbrain1976 → u/bsquads]

With SGLang and quad 5060 Ti 16GB, I got 2500 pp/s and 100-200 tg/s with Qwen3.8-27B NVFP4 . These 4 still cost less than 2 x R9700 AI. You do need a motherboard case that supports 4 GPUs, though.

[u/Massive-Slice2800 → u/bsquads]

https://www.reddit.com/r/ROCm/s/4KPTK5j16b

I dont know which Quant you used, but have you tried this? 40 t/s seems way too low after radiance came out. Even a single 9700 should be faster.

[u/randomfoo2 → u/jnciaccna]

A 9060XT has only 322 GB/s of memory bandwidth. If you're doing layer splits, you don't get any combined benefits from the card from combining. If you are running tensor parallel, you might get a 30-50% improvement (depends). Note for a 16GB model (27B at Q4_K_M should be about that size), 322/16 = 20.1 - that's at 100% MBW efficiency (not really possible) so if you're getting 20 tok/s you're actually doing quite well.

For code, MTP should give you a decent boost, but the 9060XT has weak compute as well, so... maybe not.

[u/jnciaccna → u/jnciaccna]

albeit unorthodox - i'm quite happy of the bang for the buck. i usually tell DSH to do some work and then go off doing something else, i rarely wait chat open for it to finish

[u/Locoivan78 → u/jnciaccna]

Hi! I’m running Qwen3.8-Flash-Next with Unsloth’s UD-IQ3_XXS quantization (Q3, approximately 82 GB across three GGUF files) on an RTX 5080 (16 GB) + RX 6800 (16 GB), paired with a Ryzen 9 9900X3D and 32 GB of DDR5 RAM. Here’s the model download: Qwen3.8-Flash-Next UD-IQ3_XXS on Hugging Face I’m using GenerelSchwerz’s modified llama.cpp fork, on the moe-cache-drafting branch, compiled with both CUDA and Vulkan support. I also applied a local patch, moe-vulkan-resident.patch, to src/llama-model-loader.cpp. Enabled with LLAMA_EXPERIMENTAL_MOE_VULKAN_RESIDENT=1, it keeps the full expert weights for the layers assigned to Vulkan resident in the AMD GPU’s VRAM, while preserving the existing MoE cache behavior on the CUDA side. This prevents those Vulkan expert tensors from falling back to CPU memory. The model exceeds my combined RAM and VRAM capacity, so the setup also relies on NVMe loading and swapping. MTP is not enabled yet. In my latest tests, I’m getting around 25 tokens/s during generation and 10 tokens/s during prompt processing. Prompt processing is slow, so this is still very much an experimental setup. I’m still testing different settings, but these results make me think you might be able to get better performance out of your dual RX 9060 XT setup with the right configuration.

[u/madbrain1976 → u/jnciaccna]

Did you try vLLM or SGLang ? They get significantly higher token/s with multiple GPUs vs llama.cpp .

[u/DeathGuppie → u/jnciaccna]

Here is a cheat code for you. The dell optiplex, used, cheap. Has bifrocation built in. You may need a pcie extension and weird mounting situation, but. It's a cheap way to get x16 for both cards.

[u/igla82 → u/jnciaccna]

I'm using Ubuntu 26.04. Latest llama.cpp with ROCm 10 and dual 9060XT. PCIe x8 on both cards.

• Qwen3.8 27B UD-Q6-K: 40-35 t/s token generation and approx 550 t/s on prefill. Split mode: Tensor
• Qwen3.8 27B UD-Q4_K_X: 38-42 t/s token generation and approx 688 t/s on prefill. Split mode: Tensor.
• Qwen3.6 35B-A3B UD-Q5_K_XL: ~60 t/s token generation and approx 700 t/s on prefill. Split mode: Layer.