Home Lab

Cloud is fine until the AI box is on all day and the bill is the project. I run Rivet here, try local models, keep my own graphs, run the house. I want the hardware, the data, and a bill I can predict.

Enterprise surplus. eBay and patience.

The GPU box, a Supermicro SYS-4029GP-TVRT. Two Xeon Gold 6130s, 384GB RAM, eight Tesla V100 SXM2 32GB on an NVLink mesh. Ubuntu, Incus, ZFS. It runs the big local models, DeepSeek and GLM. The memory datahub lives here too, on its own NVMe with space reserved so a model download can't starve it.

GERTY, a DL380 G9. Dual E5-2699 v4, 44 cores, two Tesla V100 32GB over PCIe. It was a Proxmox host and got reinstalled as Ubuntu with Incus this fall. It runs the Rivet agent containers, the Qwen 27B, and the embedder that RivetOS memory and this site's chat use.

The two DL380 G7s, pve1 and pve2, retired in September after their workloads moved to the G9.

Mini PC: Home Assistant and Frigate, Coral TPU. Cameras and the house.

OpenWrt on a Linksys WRT3200ACM. WireGuard back to the production box and phones. Flat 10.4.20.0/24. Servers static, clients DHCP. Names like rivet.home, grafana.home, cam-driveway.home.

philtompkins.com comes back over WireGuard so monitoring can scrape it. DDNS keeps the VPN endpoint alive when the home IP moves.

One container: Grafana, ClickHouse, OpenTelemetry Collector. Node exporters on every host. Out-of-band exporters for fans, temps, watts.

Logs to Loki. Alerts to Discord. The stack sits on about 4GB of RAM.

Home Assistant: Tesla charging through Fleet API and a self-hosted auth proxy, ecobee, Frigate cameras, an Eufy lock, a Bambu. Seven automations. Charge schedule, welcome home, battery.

Frigate on the mini PC with a Coral USB. Driveway and front door, 24/7, record on motion, Discord if it sees a person or a car.

Ten V100s across the two boxes, all running 1Cat-vLLM. Volta has no FP8 or FP4 tensor cores, so W4A16, EXL3, and speculative decoding are what actually work. The recipes, downloads, and benchmarks are on the models pages.

The embedder for RivetOS memory and this site's chat runs locally. Eval artifacts live on shared storage so Rivet can compare runs without pulling weights again.