← The X99 Build

A Tesla V100 Next to the Unlocked CMP 40HX: Two Drivers, One Box

Adding a V100 SXM2 16GB to the 200 Bucks Inference Machine without giving up the CMP 40HX unlock: VFIO passthrough to a VM on the closed 580 driver, then llama.cpp RPC to pool both cards. October 2026.

X99 build series: The build · mistral.rs vs llama.cpp · Four LLMs compared · Wake on LAN · ReBar BIOS mod · CMP 40HX unlock · V100 + VFIO

The CMP 40HX unlock only exists for NVIDIA's open kernel module, and the open module refuses the V100 because Volta has no GSP. Linux loads one nvidia.ko, so the two cards can't share a driver without giving up the unlock. Instead the V100 goes to a KVM guest through VFIO and runs the closed 580 driver there, while the host keeps the patched 610 driver for the CMP. In the guest the V100 runs Qwen3 8B at 106 tok/s, against 62 on the unlocked CMP. llama.cpp's RPC backend then pools both cards for models neither can hold alone: Qwen3 32B runs at 24 tok/s across the two, against 5.7 with the V100 and the CPU.

The problem

With the V100 in the second x16 slot, the patched driver loaded for the CMP and rejected the new card at probe time:

NVRM: The NVIDIA GPU 0000:04:00.0 (PCI ID: 10de:1db1)
NVRM: installed in this system is not supported by open
NVRM: nvidia.ko because it does not include the required GPU
NVRM: System Processor (GSP).

NVIDIA's open kernel modules only drive Turing and newer, because they hand most of the driver's work to the GSP on the card. Volta doesn't have one, so the V100 needs the closed module, and 580 is the last branch that supports it. That rules out the obvious idea of porting cmpunlocker to 580: a port would still be an open module. The patches edit the resource manager source (kernel_gsp_booter_tu102.c and friends), which in the closed driver ships as a prebuilt binary, so there's nothing to apply them to.

Passing the V100 to a VM

IOMMU. The BIOS already had VT-d on (the ACPI DMAR table was present), but the kernel wasn't using it. I added intel_iommu=on iommu=pt to GRUB_CMDLINE_LINUX_DEFAULT. After a power cycle there were 33 IOMMU groups, and the V100 sat alone in group 31, with the CMP alone in group 30. That's the best case: nothing else has to come along with the card.

Claiming the card for VFIO. The usual recipe is options vfio-pci ids=10de:1db1 in /etc/modprobe.d. It did nothing here, because Ubuntu builds vfio-pci into the kernel (CONFIG_VFIO_PCI=y), so module options files never apply. The ID has to go on the kernel command line instead:

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on iommu=pt vfio-pci.ids=10de:1db1"

After update-grub and a reboot, lspci -k -s 04:00.0 showed Kernel driver in use: vfio-pci. The ID matches only the V100, so the CMP (10de:1f0b) stays on the patched driver, and its boot-time Gen2 retrain kept working.

The VM. With qemu-kvm libvirt-daemon-system virtinst ovmf installed:

sudo virt-install --connect qemu:///system --name v100 \
  --memory 6144 --vcpus 8 --cpu host-passthrough \
  --machine q35 --boot uefi --os-variant ubuntu22.04 \
  --disk path=/var/lib/libvirt/images/v100.qcow2,size=80,format=qcow2 \
  --hostdev 04:00.0 --network network=default \
  --location /var/lib/libvirt/images/ubuntu-22.04.5-live-server-amd64.iso,kernel=casper/vmlinuz,initrd=casper/initrd \
  --extra-args 'console=ttyS0,115200n8' --graphics none

The installer runs on the serial console, which suits a headless box. The ISO has to live somewhere libvirt's QEMU user can read, not in a home directory. Inside the guest the V100 showed up with its full 16 GB BAR1 mapped, so the large-BAR workarounds (a bigger 64-bit MMIO window in OVMF) weren't needed.

Secure Boot got in the way. On Ubuntu, --boot uefi picked the OVMF build with Microsoft keys enrolled, so the guest booted with Secure Boot on. The 580 driver built through DKMS without complaint, then failed to load:

modprobe: ERROR: could not insert 'nvidia': Operation not permitted
Lockdown: modprobe: unsigned module loading is restricted

Rather than enrolling a MOK key, I pointed the domain at the plain firmware, OVMF_CODE_4M.fd, with a fresh NVRAM from OVMF_VARS_4M.fd, and redefined it. Ubuntu's fallback bootloader picked up the existing install, Secure Boot came up disabled, and nvidia-smi in the guest showed the V100: 16384 MiB, Gen3 x16, ECC enabled, 300 W limit.

Inside the guest: nvidia-driver-580-server (the closed module) with its packages held, then Ollama set to listen on all addresses through a systemd override. Ollama picked its cuda_v12 backend, since CUDA 13 dropped Volta. On the host, a DHCP reservation keeps the guest at 192.168.122.178, and virsh autostart v100 brings it up at boot.

Exposure. The guest sits on libvirt's NAT network. It can reach out, but nothing can open a connection to it except the host: not the LAN, and not my Tailscale devices, because the host doesn't advertise 192.168.122.0/24. That matters because neither Ollama's API nor llama.cpp's RPC server has authentication.

Results: V100 in the guest vs. the unlocked CMP

The same llm.sh script as the CMP post, run from the host against the guest's Ollama: Qwen3 8B Q4_K_M, a 1.7k-token prompt with a unique prefix per run, 128 generated tokens. The first V100 run included warm-up (1,911 tok/s prompt), so the table uses the two after it.

Generation tracks memory bandwidth, and the V100's HBM2 runs at about 900 GB/s against the CMP's 448. Going through the VM doesn't seem to cost much: the guest pulled 52 W at 46 °C during the run, and the numbers were stable after the first one.

Pooling both cards with llama.cpp RPC

Ollama can't split one model across two machines, and the guest is effectively a second machine. llama.cpp can, through its RPC backend: ggml-rpc-server runs next to a GPU and exposes it over TCP, and the client treats it as one more device and assigns it a share of the layers. Only activations cross the link per token, a few kilobytes, so a virtual network is plenty.

Guest side. CUDA 12.9 from NVIDIA's repo (CUDA 13 can't target sm_70), and llama.cpp built at exactly the same commit as the host (f498f86), because the RPC protocol has to match on both ends. I copied the source over with git archive f498f86 | ssh v100 tar -x rather than cloning, to be sure. Only the server target is needed:

cmake -B build -DGGML_CUDA=ON -DGGML_RPC=ON -DCMAKE_CUDA_ARCHITECTURES=70 -DCMAKE_BUILD_TYPE=Release
cmake --build build --target ggml-rpc-server -j 5

That took 12 minutes on 8 vCPUs. I kept it to 5 jobs so nvcc wouldn't run the 6 GB guest out of memory. A systemd unit runs it at boot:

ExecStart=/home/james/src/llama.cpp/build/bin/ggml-rpc-server -H 192.168.122.178 -p 50052 -d CUDA0 -c

-H binds to the guest's private address, which only the host can reach (the server prints a large warning about exposure, and it's right to). -d CUDA0 offers just the V100, not the guest's CPU. -c caches received tensors on the guest's disk, so a reload doesn't push the weights over the link again. After the 32B run that cache was 16 GB.

Host side. The host's existing build (sm_75, -DGGML_RPC=ON) only needs --rpc. One gotcha: --rpc has to come before --list-devices, or the RPC device isn't registered in time and doesn't show up.

$ llama-bench --rpc 192.168.122.178:50052 --list-devices
  CUDA0: NVIDIA CMP 40HX (7796 MiB, 7703 MiB free)
  RPC0: 192.168.122.178:50052 (16144 MiB, 15833 MiB free)

That's 23.5 GB free across the two cards. -dev CUDA0/RPC0 splits the layers across both, by default in proportion to free memory.

Llama 2 7B: each card, then both

llama-bench, Q4_0, flash attention on, all layers on GPU, 3 repetitions. It fits on either card, so this isolates what the RPC hop and the split cost.

The V100 through RPC beats the CMP running locally by 47% on prompt and 58% on generation, protocol overhead included. The split lands between the two, as it should: each token runs through the CMP's layers and then the V100's in sequence, so it's no faster than the slower stage allows. For a model that fits on the V100, the V100 alone is the right choice.

Qwen3 32B: the model that fits on neither

Qwen3 32B Q4_K_M is 18.4 GiB (19.8 GB on disk). It won't fit on the 16 GB V100 or the 8 GB CMP, but it fits across both. As the baseline without pooling, I put as much as fits on the V100 (50 of 65 layers) and ran the rest on the host CPU, all 12 cores.

Pooling is 4.2× faster on generation and 11× on prompt processing. 24 tok/s is comfortably faster than reading speed for a dense 32B model. The CPU baseline's prompt number is especially poor because for large batches llama.cpp copies the CPU-resident weights to the GPU for each pass, and here that GPU sits behind RPC. The whole pooled llama-bench run, model load included, took 114 seconds, even though the host only had about 7 GB of RAM free for a 19.8 GB file.

Running it day to day

For the pooled case, llama-server gives the same OpenAI-compatible API Ollama does. Note the device list is comma-separated here, where llama-bench uses a slash:

llama-server --rpc 192.168.122.178:50052 -m Qwen3-32B-Q4_K_M.gguf \
  -ngl 99 -fa on -dev CUDA0,RPC0 -c 4096 --host 127.0.0.1 --port 8089

With the guest's tensor cache already warm it was serving in about 80 seconds, and a short chat request generated at 22 tok/s.

Everything comes back on its own after a cold boot: the host's retrain unit takes the CMP to Gen2, vfio-pci claims the V100, libvirt autostarts the guest, and the guest starts Ollama and the RPC server. One manual step is left: the CMP's 300 MHz idle clock lock still applies at boot and has to be released with sudo cmp40hx-uncap before any benchmark. My first pooled run hit it and reported 8 tok/s for the CMP.

If you're doing this yourself

Check the IOMMU groups before buying anything. If the card shares a group with other devices, passthrough gets much harder. On Ubuntu, put vfio-pci.ids= on the kernel command line, since the module options file is ignored. Pick the OVMF build without Secure Boot, or be ready to sign the DKMS module. Build llama.cpp at the same commit on both ends. And keep the RPC server on a network nothing else can reach, because it has no authentication at all.