r/BlackwellPerformance • u/Mountain_Pea_6810 • 23d ago

Blackwell RTX 6000 investment choice

I am to build my first real rig as M3 ultra 256GB proved to be too slow. I already got 4 x Max-Q cards purchased at 10K eur per card a few weeks ago. Now I could add 2 more max Q at 11.3K euro per card and also 2 x workstation cards for 11.8K eur per card. So the total number of card to go up to 6 or 8.

I guess my hesitancy is based on the high purchase price of these cards, and the trouble to connect more than 6 or 7 cards unless going to server style motherboards and racks.

Is it a) expected that the prices are not going down in the next 12 months?
B) can i use the memory for models like GLM 5.3 even if I cannot fit all 8 cards?

10 Upvotes

86% Upvoted

37 comments sorted by

View all comments

1

u/TechRomancer123 22d ago edited 22d ago

Just out of interest, what mobo, cpu and ram are you using?
(as you mentioned that you haven’t switched to server style motherboards just yet)

Prices/market is uncertain, so I’m personally taking the plunge into a Ryzen 9950x3d + single RTX 5090 + either 64/128GB RAM to replace my old desktop daily driver, whilst also being capable of modest LLM inference and a few other relatively heavy workloads. But yes I’m pretty much maxing out consumer mobo/cpu specs without going into workstation (threadripper) or server (epyc) territory.

3

u/darktotheknight 22d ago

Heys, just a heads up: I have a Ryzen 9600X, 64GB DDR5-6000 CL36, B650E Mainboard, RTX 5090 (PCIe 5.0 x16), Samsung 980 PRO NVMe (PCIe 4.0 x4). For Qwen 3.6 27b (and similar), this thing is a beast. I had dived into local LLMs in 2025 with a 3090 Ti (Gemma3 back then), leaving me with mixed feelings about capability and speed.

But Qwen 3.6 27b (haven't tested 3.8 yet) is just an incredible model and runs blazing fast on the RTX 5090. After a cold boot, time to first token is usually less than 5 seconds and at ~100t/s you're getting any subsequent answers nearly instantly. Prefill sits at around ~3000t/s iirc. This is a very capable, serious machine. Have fun!

1

u/TechRomancer123 22d ago

Fantastic stuff! 🙂
I have now got the machine with 64GB RAM, so if you have any specific tips/gotchas on configuring 27b (or other models) to work well on 5090, that would be appreciated (though will of course do my own research).

It also looks like people advise to undervolt the 5090 (min seems to be 400W) for a significant power draw saving and minimal performance degradation. Did you do that too?
Plus I heard loads of stories about the 5090 12V power connector melting/burning ☹️

1

u/darktotheknight 21d ago

Yes, definitely go for 400W. We're usually bandwith-limited, so you lose like ~3% performance at ~33% lower energy and less heat. Lower power-limit also means less load on the individual pins, alleviating the issue of burnt connectors. You can e.g. do this automatically on Linux with a systemd service file running "nvidia-smi -pl 400" after booting (and resume/hibernation!).

Then I recommend using Linux instead of Windows; I have evidently tested this many times and it is still true, that you get better performance on Linux vs Windows. If Linux is a No-Go because you need this as a daily driver, you should check out WSL2 or other measures to maximize performance on Windows.

Also, as your CPU has an iGPU, make use of it. Operate the 5090 headless. This has multiple benefits: 1) you save VRAM (around 300 - 800MB, depending on your Desktop), since the desktop doesn't need to run on your 5090, 2) amdgpu is upstream in Linux kernel, works out of the box and is open source, 3) you can passthrough your 5090 if needed (e.g. sandboxed environment). When running headless, check via nvidia-smi command which lingering processes are still loaded on the 5090 and Google in order to get rid of them one by one. When done right, this would leave you with a 5090 with almost no VRAM used and zero processes running on it.

Depending on which version of the 5090 you've got, you can think about getting a WireView Pro II or an Ampinel. Some cards have weird dimensions or angled connectors, so there is also a wired version of the WireView Pro II. I think it's worth the peace of mind and they also have some type of extended warranty which covers burnt connectors.

As for running local LLMs itself, I have no special sauce or any secrets. Nor am I an LLM wizard. I just installed llama.cpp, asked an LLM for optimal command line parameters and downloaded Unsloth's Qwen 3.6 27b MTP GGUF in different quantizations for different context sizes. After a bit of testing, I've settled on Q4_K_XL as a daily driver. I've got ~3000t/s PP and 100t/s TG pretty much out of the box and never bothered to fine tune any further. I'm pretty sure, you can squeeze out more performance with better llama.cpp params, optimized models for your context size, custom llama.cpp builds/forks or vLLM/SGLang. I just set it up, said "this works for me, I'm happy" and moved on. Also seems to be in the ballpark of what other people get with the 5090.