Proxmox VE 9.2 GPU Passthrough Upgrade Check Guide

Run a Proxmox VE 9.2 GPU passthrough upgrade check to confirm VFIO binding, guest driver state, and real workload output before production traffic resumes.

Proxmox PulseProxmox Pulse
11 min read
Magnifying glass over graphics card pins on server motherboard with blue sparks and dark studio lighting

A Proxmox VE 9.2 upgrade can leave GPU passthrough looking healthy while the guest has lost display or compute. The VM boots, the host reports the PCI device, and the web console may still show a login screen, but the NVIDIA or AMD compute stack can fail without a loud error. Run a short pre- and post-upgrade GPU check so you know the card is still bound to VFIO, still visible in the guest, and still usable for the workload that needs it.

Key Takeaways

  • Upgrade blind spots: A GPU VM can boot after a VE 9.2 update while the host has reclaimed the card or the guest driver is no longer happy.
  • Host first: Confirm IOMMU, vfio-pci binding, and the VM PCI assignment before asking the guest why it is sad.
  • Guest proof: A running VM is not proof. Check the guest PCI tree, driver state, and compute status.
  • Real workload: One small CUDA, OpenCL, or transcoding job catches failures that a status command misses.
  • Fast feedback: A focused test takes about eight minutes per GPU VM, which is cheap compared with a production outage.

Why GPU Passthrough Can Pass the Upgrade and Still Fail

GPU passthrough sits across several moving parts: the host kernel, IOMMU settings, vfio-pci binding, QEMU device assignment, the guest BIOS, and the vendor driver inside the VM. A minor Proxmox VE upgrade can touch more of that stack than a simple package update suggests. The kernel may boot with a different initramfs, QEMU may be newer, and firmware packages may change. Any one of those changes can turn a working GPU into a silent failure.

Before you test a GPU VM, make sure the node itself finished the Proxmox VE 9.1 to 9.2 upgrade checklist. A GPU test on a half-upgraded node wastes time. Also confirm the guest is healthy enough to report problems. If you rely on guest IP reporting or clean shutdowns, the QEMU Guest Agent check after VE 9.2 upgrade is a quick companion test.

The failure I worry about is not a red error screen. It is a VM that runs, answers SSH, and still has no usable GPU. The Proxmox web console can be misleading here. A separate display device may show the firmware or a text login even when the passed-through GPU has no output. For a node that also carries SDN or VLAN traffic, pair this GPU check with the reboot drill to verify SDN and VLANs before calling the node healthy.

How to Confirm the Host Still Hands the GPU to VFIO

Start on the Proxmox host, not inside the VM. If the host has the GPU, the guest cannot use it. If the host does not have it, the VM may still start and then fail in a way that looks like a driver problem.

Check IOMMU and the vfio-pci binding

pveversion
dmesg | grep -i -e DMAR -e IOMMU
cat /proc/cmdline

The version command tells you the node really is on the release you think it is. The kernel log should show IOMMU enabled, and the boot parameter line should include the IOMMU setting your platform needs. On Intel systems, that is usually the Intel IOMMU parameter; on AMD systems, it is the AMD IOMMU parameter. If the parameter disappeared during a boot loader change, the GPU may be visible but not assignable.

Next, look at the physical device and its kernel driver. Replace the PCI address with the one your card reports.

lspci -nnk -s 0000:01:00

For a passed-through GPU, you want the kernel driver in use to be vfio-pci, not the host vendor driver. Many GPUs expose a second function for audio, and that function also needs to be bound to vfio-pci. If the host still shows the vendor driver, the VM cannot own the card cleanly.

If you bind by device ID, the module option usually looks like this. The IDs come from the PCI list, not from the VM configuration.

options vfio-pci ids=10de:1b80,10de:10f0

After changing the binding, rebuild the boot artifacts and reboot.

update-grub
update-initramfs -u -k all
reboot

Confirm the VM still owns the device

Once the host is clean, check the VM configuration. The PCI address can change if the card moved, if a BIOS update changed PCIe enumeration, or if the VM was restored to a different node.

qm config 100 | grep -E 'hostpci|args|display'

Look for the hostpci line and any custom args line. If the VM uses args for a device override, the hostpci line may not tell the whole story.

Then check the running QEMU command. This is a good upgrade check because it shows what the current Proxmox packages actually generated.

qm status 100
qm showcmd 100 | grep -E 'vfio|hostpci|display'

You should see a vfio-pci device entry and the expected display setting. If the VM is stopped, start it before running the command-line check. If the VM is running but the command line lacks the vfio entry, the problem is between the VM configuration and the QEMU process, not inside the guest.

How to Run a Fast Guest-Side Smoke Test

The guest is where display and compute failures become visible. Do not stop at the first sign of life. A login prompt can mean the VM is healthy, but it does not mean the GPU is healthy.

Linux guests

Start with the PCI tree and kernel messages. These commands tell you whether the guest sees the card and whether the vendor driver attached.

lspci -nn | grep -i -e vga -e 3d
dmesg | grep -i -e nvidia -e amdgpu -e vfio

For NVIDIA guests, the status tool gives a compact summary.

nvidia-smi --query-gpu=name,driver_version,vbios_version,compute_cap --format=csv,noheader
nvidia-smi --query-compute-apps=pid,name --format=csv,noheader

The first command should name the card and report a driver version. The second command may be empty when no compute job is running, and that is normal. For AMD guests, the equivalent check is often the ROCm status tool.

rocm-smi --showproductname

For desktop VMs, also confirm the display stack can see the GPU as a provider.

xrandr --listproviders

If this command fails in a headless VM, do not panic. A compute-only VM may not have an X session. But if you expect a desktop, a missing provider is a real display failure.

Windows guests

Windows gives better clues through the device status than through the desktop alone.

Get-CimInstance Win32_VideoController | Select-Object Name, DriverVersion, Status
Get-PnpDevice -PresentOnly -Class Display | Select-Object FriendlyName, Status, Problem

The first command should show the GPU name and a current driver version. The second command is more useful when the display adapter appears but is not working. A problem code such as 43 often points to a driver, virtualization, or device reset issue.

What to Check When the VM Boots but the GPU Is Silent

Display and framebuffer

A silent display can mean the passed-through GPU never became primary, the guest firmware did not initialize it, or the guest display driver failed to attach. Check the guest kernel log for the vendor driver and for display-related warnings.

journalctl -b -p warning | grep -i -e nvidia -e amdgpu -e display

If the VM uses a Proxmox VNC display for management, remember that the console may be unrelated to the passed-through GPU. Do not treat a working VNC window as proof that the GPU has output. For a GPU VM, the real question is whether the guest sees the GPU as a display provider or whether the intended application uses it.

Compute and reset errors

Compute failures are sneakier. The driver may load, the status tool may show the card, and a real workload may still fail. Look for reset errors and vendor-specific error codes.

dmesg | grep -i -e 'not fully supported' -e Xid -e reset

NVIDIA Xid messages are especially useful. A reset timeout, a page fault, or an unsupported GPU message can explain why a media transcoder or inference service stopped working after the upgrade. In my lab, a 24 GB workstation card passed the basic status check, then failed a small inference job because the guest driver had been updated to a branch that did not match the container runtime. The host assignment was fine; the guest stack was not.

A Small Post-Upgrade GPU Test Matrix

A full validation suite can take an afternoon. A targeted post-upgrade check can be much shorter. The table below shows what each check is worth and what it still misses.

Check What it proves What it can miss Typical time
Host PCI binding The host gave the card to vfio-pci Guest driver state 1 minute
QEMU command line The running VM received the vfio device Guest visibility 1 minute
Guest PCI and driver check The guest sees the card and loaded a driver Display or compute failure 1 minute
Display provider check The desktop or framebuffer sees the GPU Compute workload 1 minute
Real workload The application can use the GPU end to end Long-term stability 3 to 10 minutes

The tradeoff is time. A one-minute status check is easy to run after every upgrade, but it can still miss a CUDA context failure. A real inference job catches more, but adds minutes and a software dependency. For production GPU VMs, I keep both: a fast smoke test after the upgrade, and a scheduled workload health check that runs later.

A Practical Ten-Minute Test Script

Capture a baseline before the upgrade, then repeat the same commands after the upgrade. The exact output matters less than the comparison.

qm config 100 > gpu-vm100-before.txt
qm showcmd 100 >> gpu-vm100-before.txt
lspci -nnk -s 0000:01:00 >> gpu-vm100-before.txt

After the upgrade, run the same checks and compare the files. A quick host-side script can keep the comparison honest.

#!/usr/bin/env bash
set -euo pipefail

VMID=100
PCI=0000:01:00

qm config "$VMID" | grep -E 'hostpci|args' || true
qm showcmd "$VMID" | grep -E 'vfio|hostpci' || true
qm status "$VMID"
lspci -nnk -s "$PCI"

Run the script while the VM is running. If the VM is stopped, start it first, because the QEMU command line is the point of the test. Then move inside the guest and run the driver and workload checks from the sections above.

Common Gotchas After a VE 9.2 Upgrade

The following problems come up often enough to keep on a short checklist.

  • The vfio module option file survived, but the initramfs did not. The new kernel booted without the early vfio binding, so the host grabbed the GPU.
  • The VM configuration still pointed to the old PCI address after a BIOS update or PCIe slot change.
  • The guest driver updated separately from Proxmox and now reports a driver compatibility problem. The host assignment may be fine.
  • The display device was changed to VNC for remote console, and the passed-through GPU was never initialized as the primary display.
  • A vendor reset override worked before, but a kernel or QEMU change exposed a reset timeout that only appears during the first application launch.
  • A GPU that works in a Linux desktop VM may still fail in a container or a different guest OS because the guest driver stack is not equivalent.

In one VE 9.2 test node, the host looked perfect: the IOMMU parameter was present, the card was bound to vfio-pci, and the VM configuration was unchanged. The failure was inside the guest. The driver had been updated before the upgrade, and the new kernel in the guest exposed a mismatch. The lesson is to keep the host and guest checks separate, then compare them before and after the upgrade.

Conclusion

Run the host, VM, and guest checks after a VE 9.2 upgrade, not just after the first boot. You will know whether the GPU is still assigned, visible, and usable for display and compute. Next, pick one GPU VM and capture a baseline before your next maintenance window, then repeat the same commands after the upgrade.

Share
Proxmox Pulse

Written by

Proxmox Pulse

Sysadmin-driven guides for getting the most out of Proxmox VE in production and homelab environments.

Related Articles

View all →