200 Bucks Inference Machine
X99 build series: The build · mistral.rs vs llama.cpp · Four LLMs compared · Wake on LAN · ReBar BIOS mod · CMP 40HX unlock · V100 + VFIO
I turned an X99 board and secondhand CMP 40HX mining cards into a private LLM server for about $200. Soldering the missing PCIe capacitors unlocked x16 and cut model load time 46%, and finding a stuck 300 MHz clock cap made prompt processing about 10× faster.
Overview
This is a Huananzhi X99 BD4 motherboard and two CMP 40HX mining GPUs turned into an inference box. I built it for privacy. I'd bought proprietary GPU board schematics for a separate repair project, and I didn't want those going to an external LLM API, so I needed something local that could run a real model at a usable context length.
Hardware
The board is a Huananzhi X99 BD4, originally with two CMP 40HX cards (TU106, compute capability
sm_75, 8GB VRAM each). One card has since died, so the machine runs on a single 8GB card with only
15 GiB of system RAM next to it, and everything below is sized around that. CMP cards were made for mining, so
they have no display output and limited driver support, and getting them recognized properly took some work.
The cards weren't negotiating full x16 at first, which capped throughput. Soldering AC coupling capacitors onto the PCIe lanes fixed it and got both cards to x16. On this machine, model load time dropped 46%.
A systemd service, cap-cmp40hx.service, locks the GPU clocks to 300 MHz when the cards are idle,
which brings idle draw down to around 12 to 15 watts. Before running inference I have to reset the clocks
with nvidia-smi -rgc, or the locked clock caps performance.
Forgetting that is expensive. In one session the GPU showed 100% utilization but sat at 300 MHz and 35W,
processing prompts at about 18 tokens per second. Once the clock came up to 1935 MHz, the same work ran at about
190 tokens per second, roughly 10x faster. A 15.6k-token agent prompt went from an estimated 17 minutes to first
token down to about 80 seconds. The quick check is nvidia-smi dmon -s uc: if pclk
reads 300 under load, the cap is still on.
Software
With one card left, the everyday model is Gemma 4 E4B through Ollama (llama.cpp). It has about 4B active parameters, keeps a per-layer embedding table in system RAM, and uses mostly sliding-window attention, so its KV cache stays small enough for an 8GB card. I use it for schema-constrained extraction in my job-application pipeline.
The rest of the series:
- mistral.rs vs llama.cpp covers the serving stack, KV cache reuse (54 s → 0.27 s to first token on mistral.rs, with a token-level check that cached answers match), and the same models rerun on both engines.
- Four small LLMs on one 8GB card covers why I picked Gemma 4 E4B over three Qwen models for extraction.
- ReBar BIOS mod covers adding Resizable BAR to this board's firmware for an Arc B580.
What others have found
People have been experimenting with CMP mining cards as cheap hardware for a while, mostly outside English-language sources. Two of those shaped how I approached this build.
A Bilibili video from the channel 吃饭团的佳乐同学, part of a long series on modded mining cards, benchmarks a modded 40HX against an RTX 2060 Super and an RTX 2060. In their numbers, theoretical performance is at least 10 percent ahead of the 2060 Super. Gaming is about 20 percent behind the 2060 Super but 7 percent ahead of the plain 2060, and productivity and video editing land close to the 2060 Super. They didn't test deep learning, which is part of why I wrote this up.
Closer to what I'm doing is a Habr article by Oleg, "Dark horses of AI," on LLM inference with the CMP 40HX, 50HX and 90HX. It matches what I saw: in synthetic FP32 benchmarks these cards look ten times slower than older gaming GPUs, yet real LLM inference lands close to an RTX 2060 or 3060, around 30 tokens per second on 8B models. The article explains that Ollama and similar frameworks run inference mostly in FP16, and FP16 on these cards is nowhere near as crippled as FP32. In some cases it beats an RTX 3060.
The same article also describes the PCIe capacitor fix I used. Mining cards usually ship with capacitors for only 4 lanes even though the board supports 16, and soldering 0402 100 or 220 nanofarad capacitors onto the rest unlocks the missing lanes. In their testing, lane width barely changes token throughput (a percent or two from 4 lanes to 16), since 8 lanes is already enough for inference. Model and context load time is where it helps, improving by close to half from 4 lanes to 16. That's why I went for full x16: the server comes up and reloads context faster.
Why build it
Cloud inference is quick to set up, but everything you send it, sensitive or proprietary data included, leaves your machine. I wanted to see whether a usable local setup could be built on a tight budget from hardware most people would write off for this. Mining cards, a patched build and a few soldered joints ended up running a real model at real context length, all on my own hardware. Losing a card cost capacity, and the one 8GB card left still runs a model that beat an 8B competitor on the work I actually give it.