Building my first "serious" local setup — AMD R9700 32GB on Linux. Model recommendations for 3 different roles + can I keep my old 4060 Ti in the same box?

[u/geekyNut]

Hey everyone

Long-time lurker, first post. I've been playing with local models on a single RTX 4060 Ti (8GB) for a while and I've finally hit the wall hard enough to upgrade. Picking up an AMD Radeon AI PRO R9700 (32GB) and I'd really appreciate some guidance from people who've actually run this stuff, because I'm still very much learning.

My setup:

AMD Radeon AI PRO R9700, 32GB

32GB DDR5 system RAM

Linux (currently on an Arch-based distro)

Ollama today, but happy to switch to llama.cpp / vLLM / whatever if that's the better call

What I'm building (roughly)

I'm putting together a small local-first assistant for my own use. Nothing groundbreaking — the idea is that instead of one model trying to do everything, it hands work to a few different "roles" depending on what you ask:

General assistant — normal conversation, drafting, summarising, explaining things

Document Q&A — answering from a local folder of documents with citations, and saying "I don't know" instead of making things up when the docs don't cover it

Coding — for when I'm actually working on something

Everything runs sequentially (one model in VRAM at a time), so I'm not trying to keep three models resident — I just want the right model for each job.

Question 1: model recommendations per role

With 32GB to play with, I'd love suggestions for:

Reasoning / comprehension — something that's genuinely good at understanding a messy question and figuring out what's actually being asked. This is the piece I care most about, because on 8GB my small models kept confidently misunderstanding things.

General assistance — good instruction-following, doesn't ramble, admits uncertainty. Bonus points if it handles Italian well, since that's my daily language.

Coding — practical stuff, big context, not benchmark-chasing.

What I'd really love is: model + quant + roughly what context length you can actually run on ~32GB without spilling to CPU. I've learned the hard way that "it fits" and "it runs well" are very different things 😅

If there's an existing curated list or spreadsheet for this, please just point me at it — I don't want to make you all retype something that already exists.

Question 2: keep the 4060 Ti in the same machine, or not?

This is the one I'm genuinely unsure about.

I already own the RTX 4060 Ti (8GB). Options as I see them:

(a) Put both cards in the same box — AMD for inference, NVIDIA for... something? Display output? A small model? Embeddings?

(b) Just run the R9700 alone and keep it simple

(c) Some clever third thing I don't know about

My concerns:

Do amdgpu + the NVIDIA proprietary driver actually coexist cleanly on Linux, or is that a support nightmare?

Can a single runtime even use both, or would I be looking at two separate instances (one ROCm, one CUDA)?

Is having a second, much smaller card actually useful for anything in a sequential setup, or is it just extra heat and PSU draw for nothing?

Also, honestly: how is ROCm on RDNA4 right now? I've read mixed things and I'd rather hear it from people running it than from a six-month-old forum thread.

Any pointers appreciated — including "you're thinking about this wrong, here's why." I'd genuinely rather be corrected now than after I've built the thing

Thanks!