Proxmox OCI LXC GPU: Attach NVIDIA to LXC Containers

You can attach a physical NVIDIA GPU to a Proxmox VE 9.2 OCI LXC container by installing the NVIDIA driver on the Proxmox host and mapping the NVIDIA device nodes into the container. The result is a small AI inference host that starts in seconds, uses only the memory the application needs, and avoids the extra guest kernel and VM overhead of a full QEMU virtual machine.

Key Takeaways

Host driver: Install the NVIDIA driver on the Proxmox host, not only inside the OCI LXC.
Device nodes: Map the NVIDIA device files into the container and allow their device majors.
Fast start: A stopped OCI LXC with GPU access can be running an inference API in under a minute.
Shared kernel: The container and host share the kernel and GPU driver, so driver changes affect the container.
Best fit: Single-node AI inference, lab services, and containers that need a GPU without a full VM.

Why Put a GPU in an OCI LXC Instead of a VM?

An OCI LXC is an LXC container created from an OCI image, so it starts as a lightweight Linux userspace on the Proxmox host kernel. For many AI inference workloads, that is enough. You do not always need a second guest kernel, display stack, and full virtual GPU controller just to run a model server.

The tradeoff is isolation. A VM with GPU passthrough gives each guest its own driver stack and kernel boundary. An OCI LXC with GPU access gives you speed and low overhead, but the container depends on the host driver and kernel. That is usually fine for a trusted lab or a small production inference service, but it is not the same as putting an untrusted tenant on a dedicated GPU.

Aspect	OCI LXC with GPU device mapping	VM with GPU passthrough
Start time	Usually under a second after the host is up	Often several seconds to tens of seconds
Memory overhead	Container memory plus application	Guest kernel, QEMU overhead, display stack
Isolation	Shares kernel and host driver	Separate kernel and driver stack
GPU driver updates	Host driver update changes all containers	Each VM can run its own driver
Best use case	Single GPU inference host for one trusted workload	Multi-tenant, mixed OS, strict isolation

If you already run Docker or application services inside an OCI LXC, the GPU work is mostly device mapping and driver placement. For the base container workflow, see Docker in OCI LXC on Proxmox VE 9.2.

How to Check the Host GPU Before You Touch the Container

Start on the Proxmox host. The container cannot reach the GPU unless the host has the driver, kernel module, and device files ready.

lspci -nn | grep -i nvidia
ls -l /dev/nvidia*

If the device listing is empty, install the NVIDIA driver on the host. On a Debian-based Proxmox host with the non-free component enabled, this is usually enough:

apt update
apt install -y "pve-headers-$(uname -r)" nvidia-driver
update-initramfs -u
reboot

After the host comes back, confirm the driver and load the CUDA helper module if needed:

nvidia-smi
modprobe nvidia-uvm
ls -l /dev/nvidia*

The nvidia-uvm device matters more than it first appears. Many CUDA applications fail with confusing errors when the container can see the main GPU device but cannot reach the unified virtual memory device.

How to Pass the NVIDIA GPU Into the OCI LXC

The examples below use container ID 100. Replace that with your container ID. If you plan to run Docker inside the OCI LXC, enable nesting first:

pct set 100 -features keyctl=1,nesting=1

Then stop the container and add the NVIDIA device rules:

pct stop 100 || true
cat <<'EOF' >> /etc/pve/lxc/100.conf
lxc.cgroup2.devices.allow: c 195:* rwm
lxc.cgroup2.devices.allow: c 234:* rwm
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
EOF
pct start 100

What the Config Lines Actually Do

The two device allow lines let an unprivileged OCI LXC access NVIDIA character devices. Major number 195 is the main NVIDIA device range, and major number 234 is commonly used for the NVIDIA unified memory devices. Proxmox VE 9.2 uses cgroup2, so the config uses the cgroup2 form.

The mount entries bind the host device files into the container filesystem. The optional flag is useful because it prevents the container from failing to start if one of the device files is temporarily missing.

If you use a privileged container, the device allow lines are less critical, but the bind mounts still make the container view predictable. If you use an unprivileged container, the device allow lines are usually the difference between a visible device file and a permission error.

How to Verify CUDA Inside the Container

First, check that the device files are visible inside the container:

pct exec 100 -- ls -l /dev/nvidia*

If the files appear, install a small GPU-aware application. Ollama is a practical choice because it is easy to test and gives a clear signal when a model is using the GPU. The following steps assume a Debian or Ubuntu OCI base image.

pct exec 100 -- apt update
pct exec 100 -- apt install -y curl
pct exec 100 -- bash -c 'curl -fsSL https://ollama.com/install.sh | sh'

If your OCI image does not run a service manager, start Ollama manually:

pct exec 100 -- bash -c 'ollama serve > /var/log/ollama.log 2>&1 &'

Pull a small model and run a prompt:

pct exec 100 -- ollama pull llama3.2:3b
pct exec 100 -- ollama run llama3.2:3b "Answer in one sentence: what is Proxmox VE?"

Then check whether Ollama reports GPU usage:

pct exec 100 -- ollama ps

A successful GPU-backed run looks similar to this:

NAME              SIZE     PROCESSOR    UNTIL
llama3.2:3b       4.4 GB   100% GPU     4 minutes

On a used 12 GB RTX 3060, the first prompt loaded the 4.4 GB model in about eight seconds and answered in a couple more seconds. That timing is not a benchmark, but it is the kind of result that makes the OCI LXC approach feel practical: you get a usable inference endpoint without building a full VM.

What Runs Well on a GPU-Backed OCI LXC

This setup fits workloads that are predictable and trusted. Good candidates include:

Local LLM servers for internal tools.
Ollama or llama.cpp serving a GGUF model.
Whisper transcription services.
Stable Diffusion or similar image generation when the container has the right libraries.
Small AI agents that need low latency and fast restarts.

The pattern works best when the OCI LXC is the application host, not a general-purpose sandbox for many unrelated users. If you already run a Docker stack inside an OCI LXC, the same device mapping applies before Docker or Podman starts.

A Simple Inference API Test

Once Ollama is running, test the API from inside the container:

pct exec 100 -- curl -s http://127.0.0.1:11434/api/generate -d '{"model":"llama3.2:3b","prompt":"Hello","stream":false}'

If you expose the API to a LAN, keep the container firewall rules tight. The Proxmox OCI LXC traffic lockdown checklist is a good fit before you open port 11434 to more than localhost.

Tradeoffs You Accept When Skipping the VM

The biggest honest tradeoff is driver coupling. Because the OCI LXC uses the host GPU driver, a host driver update can affect every GPU container on that node. A kernel update can also matter if the NVIDIA module needs to be rebuilt. In a homelab, that is often acceptable. In a shared production environment, you want a maintenance window and a tested driver version.

Security is the second tradeoff. Unprivileged LXC containers are much safer than privileged containers, but they still share the host kernel. That is fine for a trusted inference service running one application. It is a weaker boundary than a VM, especially if you plan to expose the container to users who can influence what runs inside it.

GPU memory is the third tradeoff. A model server can consume all available VRAM, and multiple containers sharing one GPU can compete for that memory. Proxmox does not magically partition the GPU just because you passed the device into a container. If you need strict per-workload GPU memory limits, you need a different design, such as one container per GPU or a VM-based isolation layer.

Common Gotchas That Break GPU Access

The first time I added NVIDIA device nodes to an LXC container, I mapped only the main GPU device and control device. CUDA still failed until I realized the container could not reach the unified virtual memory device. The missing piece was the NVIDIA unified memory device and its major number.

Check the container config if something is wrong:

pct config 100 | grep nvidia

Restart the container after config changes:

pct restart 100

Confirm the critical device is present:

pct exec 100 -- ls -l /dev/nvidia-uvm

Other issues are less obvious:

The host driver may be installed, but the NVIDIA kernel module may not be loaded after a reboot.
A non-root application may still fail if the device file permissions do not match the application user.
A container image may have CUDA libraries that do not match the host driver well enough for your application.
A Docker-in-LXC setup may need the NVIDIA container toolkit inside the container; raw device mapping alone is not the same as a full container GPU runtime.
A model server may fall back to CPU without a loud error, so check the application log and GPU usage output.

Before treating the container as a long-running service, run the Proxmox OCI LXC pre-production audit and repeat the GPU checks after any host package update.

Conclusion

Passing a GPU into an OCI LXC gives you a fast, low-overhead inference host on Proxmox VE 9.2 without the extra guest kernel and memory cost of a VM. The main work is installing the NVIDIA driver on the host, mapping the NVIDIA device nodes, and confirming CUDA inside the container. Your next step is to pick one model, run the Ollama test, then decide whether the shared-driver tradeoff fits your environment.