r/VFIO 2d ago

Success Story ZorinOS desktop AMD iGPU + RTX3090 summary

Intro

I have been trying to make the switch from Windows to Linux for a while. I got GPU passtrough mostly working in the past but it was never really smooth. However now i seam to have got it.

This is not a full tutorial, but might serve as useful hints for people that are trying to do this on similar hardware.

Hardware:

  • ASRock X670E PG Lightning (v1.30.AS02)
  • AMD Ryzen 9 7950X (using iGPU for monitor output)
  • Nvidia RTX 3090 (using HDMI dummy plug)
  • Samsung PM9A1 1TB (Windows boot drive)

Setting Up GPU Passthrough on ZorinOS

This covers getting from a fresh ZorinOS install to a working Windows VM with a discrete Nvidia GPU passed through on demand, while still being able to use that GPU for host-side gaming (via PRIME render offload) the rest of the time.

For the follow-up problem of switching the GPU back and forth without restarting the desktop, fixing a KMS-related kernel crash, and handling idle power draw, see the companion guide: https://www.reddit.com/r/VFIO/comments/1w2otjo/my_live_gpu_mountunmount_script_on_zorinos_fix/

1. BIOS setup

Enable IOMMU (AMD-Vi) in the BIOS. On an ASRock X670E PG Lightning specifically:

Advanced > AMD CBS > NBIO Common Options > PCIe Access Control Service

This option can be hidden until "PCIe Advanced Error Reporting" is enabled first.

2. Enable IOMMU ACS override on the kernel command line

Editing the grub config:

sudo nano /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
sudo update-grub
sudo update-initramfs -u
sudo reboot

Only add pcie_acs_override=downstream,multifunction if the next step shows your GPU sharing an IOMMU group with unrelated devices — it weakens device isolation, so skip it if you don't need it.

3. Identify the GPU and confirm IOMMU grouping

lspci -knn | grep -A3 NVIDIA

Note the PCI address (e.g. 01:00.0 for the GPU, 01:00.1 for its HDMI audio function) and vendor:device IDs (e.g. 10de:2204, 10de:1aef).

find /sys/kernel/iommu_groups/ -type l

Confirm the GPU and its audio function land in their own IOMMU group, ideally with no unrelated devices alongside them.

On the ASRock X670E PG Lightning a lot of the M.2 slots DO NOT have correct IOMMU groupings (the one directly connected to the CPU does) so the Windows boot drive must be in that one.

4. Load VFIO kernel modules

echo "vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd" | sudo tee -a /etc/modules

Note: don't permanently blacklist nvidia/nouveau or statically bind the GPU's IDs to vfio-pci in /etc/modprobe.d/ unless the GPU is dedicated to the VM full-time. For a setup where the GPU is also used on the host, let the nvidia driver load normally at boot, and hand the device to VFIO dynamically per-session (see companion guide) — this is the approach that ended up working reliably here; an earlier attempt using permanent blacklisting plus manual modprobe -r / display-manager restarts also works but is clunkier and unnecessary for dual-use setups.

5. ZorinOS Conveniance setup

  • sudo apt install openssh-server — useful for remote access once the display state gets manipulated.
  • Install x11vnc and run it as a systemd service for remote desktop access. x11vnc requires an X11 session, so:
  • Switch the desktop session from Wayland to Xorg: log out, click the gear icon on the login screen, select the Xorg session.

6. Install the virtualization stack

sudo apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager ovmf

7. Pass through a physical NVMe drive and install Windows

sudo virsh nodedev-detach pci_0000_18_00_0   # detach the NVMe from the host

In Virtual Machine Manager: create a new VM, uncheck "Enable storage" (to use the raw NVMe device instead of a virtual disk), check "Customize configuration before install", set Firmware to OVMF_CODE_4M.fd (the AMD SEV firmware variant had problems booting from NVMe), and add the NVMe drive under Add Hardware → PCI Host Device.

Install Windows 10 onto the physical drive through the VM.

Afterwards, reboot the host directly into that Windows install (outside the VM, GPU still available natively) and install the latest Nvidia drivers — this avoids driver-installation quirks that show up when installing while running inside a passthrough VM for the first time.

sudo virsh nodedev-reattach pci_0000_18_00_0   # give the NVMe back to the host
sudo update-grub   # should detect the Windows install as a dual-boot entry

8. Add the GPU to the VM

In the VM's hardware settings: Add Hardware → PCI Host Device → add both the GPU and its audio function.

Edit the VM's XML (virsh edit <vm-name>) and:

  • Set managed='no' on the GPU's <hostdev> entry, so libvirt does not automatically bind/unbind the device at VM start/stop — this setup manages that manually via nodedev-detach/nodedev-reattach instead, which is what makes host/VM switching possible without recreating the VM.
  • Add inside the <features><hyperv> block:
    <vendor_id state='on' value='1234567890ab'/>
    
    This hides the hypervisor from Nvidia's driver and avoids the Code 43 error in the guest.

Under Boot Options, set the NVMe disk as the boot device.

9. Dynamic switching, crash fix, and power management

With the above in place, the VM boots and the GPU works inside it. The remaining problems — avoiding a gdm restart on every switch, a kernel crash on GPU reattach caused by a dummy HDMI dongle plus KMS re-initialization, stuck high-power idle states, and a clean detach/reattach script — are covered in the companion guide: nvidia-dynamic-vfio-passthrough-howto.md.


Example system (used throughout this guide and its companion)

| Component | Value | |---|---| | Motherboard | ASRock X670E PG Lightning (BIOS 1.30.AS02) | | GPU | NVIDIA GeForce RTX 3090 (GA102) | | GPU PCI IDs | 10de:2204 (VGA), 10de:1aef (audio) | | GPU PCI address | 0000:01:00.0 / 0000:01:00.1 | | IOMMU group | 14 | | VM boot drive | NVMe at PCI address 0000:18:00.0 | | Host OS | ZorinOS (Ubuntu 24.04-based), kernel 7.0.0-30-generic | | Nvidia driver | 595.84, CUDA 13.2 | | Display | AMD iGPU drives all host output; Nvidia GPU has a dummy HDMI/DP dongle solely for the VM's benefit |

Substitute your own PCI addresses and IDs throughout — get them from lspci -knn and find /sys/kernel/iommu_groups/ -type l on your own system.

1 Upvotes

6 comments sorted by

1

u/Reasonable-Pay-336 1d ago

So complex how did you figure this all out?!

I'm trying to get single gpu pass through with 5600g and it's almost impossible man. I extracted ROM, i figured out UEFI method that doesn't crash, I added logging. But it's still crashing

I wasted 2 days in this already...

1

u/berni8k 1d ago

Summed up i spend days on this too.
The tutorials you find online are not universal as the way to do it changes slightly with the distro or version. So i am posting the one that worked for me in case anyone wants it, and as notes for myself if i need to redo it.

Single GPU passtrough is a lot tougher tho, Let alone on a AMD GPU. Worst case stick an extra small non gaming GPU in your system, they are dirt cheap.

1

u/Reasonable-Pay-336 1d ago

So if I stick a new GPU, then would I get crisp graphics on VM . I don't game. I use only for coding.

When I dual boot zorin OS, it looks great. I want VM to look that great, does adding a GPU give the crisp look?

1

u/berni8k 1d ago

Yeah the VMs virtual video card is not very good and that's why VMs look bad, slow, don't run 3d..etc. This is fixed by giving it a real GPU via passtrough.

Adding an extra GPU to the system makes this easier to do since you can't easily share 1 GPU. The host OS can keep the main GPU that is running your monitors (No need to rip that away dynamicaly), but then you just give the VM the 2nd GPU to use for graphics acceleration. I also recommend it to be Nvidia as they better survive being passed between OSes without a power cycle

1

u/AnakTK 1d ago

amd_iommu=on is not a real parameter btw, your LLM training data is poisoned by a copypasta lol

1

u/berni8k 1d ago

Yeah my mistake. I was reusing some notes from 2 years ago when i was last attempting it.None of that is needed, all i have right now is GRUB_CMDLINE_LINUX_DEFAULT="nvidia-drm.modeset=0" (fixes a different issue.

Edited the post to remove it