Building a Local AI Coding Server – Part 1: The Hardware Reality Check

Why I built this

I wanted an AI that writes code for me around the clock, on my own hardware, without a monthly bill or my code leaving the house. My specific goal is Screeps, a programming game where you write JavaScript to run a colony, so I wanted an agent that could keep improving my colony's code while I sleep. A cloud subscription could do that too, but where's the fun in that.

This series is the honest build log: what I set up, what broke, how it got fixed, and what I learned along the way. I'm comfortable on Linux, but I wrote these posts so they're easy to follow even if you aren't, so I explain terms as I go.

The series:

1. The hardware reality check (this post): the machine, drivers, and a surprise about my motherboard.
2. Two Radeon GPUs, one stubborn bug: debugging why my two GPUs refused to work together, ending in a bug report to AMD.
3. llama.cpp to the rescue: running an 80-billion-parameter coding model across both GPUs as an always-on service.
4. Giving an AI agent its own user account: a fully autonomous coding agent, and the guardrails that keep it in line.

The hardware

The heart of the build is two AMD Radeon AI PRO R9700 cards, with 32 GB of video memory each. For running AI models, VRAM (the GPU's own memory) matters more than anything else: the whole model has to fit in it, plus working space for the conversation. Two cards give me 64 GB.

The CPU and system RAM barely matter once the model lives on the GPUs. The R9700 is a newer card, which turned out to matter: support for brand-new GPUs in AI software always lags a bit behind.

Surprise #1: my two GPUs aren't equal

Before installing anything, I collected a full hardware report. One check turned out to matter a lot: the PCIe link of each GPU. PCIe is the connection between a card and the motherboard, and it comes in widths (x16, x8, x4) and generations (3.0, 4.0, 5.0).

My second card gets about one sixteenth of the bandwidth of the first. It isn't plugged in wrong: on this consumer board, only the top slot is wired directly to the CPU. The other slots go through the chipset, the motherboard's hub chip that also handles the SSDs, USB, and network. So one card gets a highway and the other gets a side street it shares with everything else.

The command that showed it follows every hop from the CPU to each GPU and prints the link width at each step:

for d in 0000:03:00.0 0000:09:00.0; do
  echo "== GPU $d"
  for p in $(readlink -f /sys/bus/pci/devices/$d | tr '/' '\n' | grep -E '^[0-9a-f]{4}:'); do
    echo "-- $p $(lspci -s $p | cut -d' ' -f2-)"
    sudo lspci -vv -s $p | grep -E 'LnkCap:|LnkSta:'
  done
done

One trap: these cards have a small switch chip on board, so the GPU's own line says x16 even when the slot is x4. The narrowest width along the whole path is the real limit.

Why it matters: splitting one model across two GPUs means the cards talk to each other constantly. At the time it seemed like a minor detail. It was not a minor detail. This slot comes back in Part 2.

Choosing the drivers: Ubuntu's ROCm or AMD's?

I went with AMD's own ROCm 10.1 packages instead of the ones built into Ubuntu, because Ubuntu's version was too old for current AI software.

ROCm is AMD's GPU computing platform, the equivalent of NVIDIA's CUDA: the libraries that let programs do math on the GPU. Ubuntu 26.04 was the first release to ship ROCm in its own package archive, so sudo apt install rocm just works. But it installs ROCm 7.1, and AMD had already released 10.1, which lists my exact card as supported. The AI software I planned to use is built against the newer versions.

The second decision was about the kernel driver, the low-level part of Linux that talks to the cards. AMD offers its own, but it's built on your machine (via a system called DKMS), and with Secure Boot enabled that means enrolling a signing key at a blue screen during reboot, and rebuilding the driver on every kernel update. My cards already worked with the driver built into Ubuntu, so I kept that one and only added AMD's ROCm libraries on top.

Installing ROCm and checking both GPUs

The install took seven steps and one reboot, following AMD's Ubuntu 26.04 instructions:

1. Update the system: sudo apt update && sudo apt full-upgrade -y
2. Add AMD's signing key, so apt can verify packages really come from AMD.
3. Add AMD's ROCm repository for Ubuntu 26.04.
4. Install ROCm built for my chip: sudo apt install amdrocm10.1-gfx1201. Before typing Y, I checked the list for amdgpu-dkms, AMD's driver, to make sure it wasn't sneaking in.
5. Give my user GPU access: sudo usermod -a -G render,video $LOGNAME
6. Reboot.
7. Verify.

The check that matters:

rocminfo | grep -E 'Marketing Name|Name:\s+gfx'

It listed gfx1201 and AMD Radeon AI PRO R9700 twice. Both cards visible, first phase done.

A few gotchas worth knowing:

• rocm-smi is gone. Many guides use it to check GPU status, but ROCm 10.1 replaced it with amd-smi.
• Card numbers change between boots. Linux names GPUs card0, card1 and so on in whatever order it finds them. After one reboot, the numbers in my notes no longer matched. PCI addresses (like 03:00.0) never change, so I use those now.
• My monitor was plugged into GPU 0, so the desktop was eating some of its memory. The motherboard has its own video port driven by the Intel graphics, which keeps both Radeons free for AI.
• Remote access: I installed the SSH server so I can run the machine from my main computer. Remote desktop only worked without a local login after a reboot, so if yours doesn't, check that "Remote Login" (not just "Desktop Sharing") is on in Ubuntu's settings.

What I learned

• Check your PCIe slots before buying a second GPU. On consumer boards, usually only one slot is wired to the CPU.
• Newer isn't automatically better, but older can be too old. The distro's built-in packages are convenient, but AI tools move fast.
• Keep the built-in kernel driver if it works. It saved me Secure Boot headaches and future rebuilds.
• Use PCI addresses, not card numbers, whenever you refer to a specific GPU.
• Keep notes. I logged every command and result in a status file, and it saved me more than once.

At this point both GPUs showed up and everything looked healthy. Then I tried to make them work together on one model, which went about as well as the next title suggests. Next: Part 2, "Two Radeon GPUs, one stubborn bug."