Remember my last post, where my homelab didn’t have a powerful enough card for anything serious, thanks current hardware market prices…? Turns out the fix was to go backwards in time. The Tesla V100 is a datacenter card from 2017, and CUDA 13 officially dropped Volta, so most new ML docker images just refuse to run on one. But each card carries 32 GB of VRAM at secondhand prices, which is a combination that basically doesn’t exist anywhere else right now. When a pair of V100-PCIE-32GB cards landed in the homelab I couldn’t say no.
The goal was to run Qwen3.8-Flash-Next (125B total, 6B active, hybrid MoE) locally with Strata, in Docker, managed by Dockge like the rest of my stacks.
The Volta Problem
Strata’s stock image builds on nvidia/cuda:13.0.0, and CUDA 13 removed sm_70 support. On top of that, Strata’s CMakeLists refuses CUDA_ARCHITECTURES below 75 unless you set -DSTRATA_EXPERIMENTAL_SM60=ON, which swaps in the Volta-safe kernels: BF16 projections running on FP16 tensor cores, mma.m8n8k4 prompt attention, and dp4a expert kernels.
The fix is three small changes to the Dockerfile. CUDA 12.9 base image, the experimental flag on the engine build, and the flag again on the vision encoder build. I keep the patch in the repo, or you can do the same thing with three sed one-liners if you prefer.
One gotcha worth knowing: on first start Strata detects Volta and compiles its own engine inside the container (about 9 minutes on 64 cores), and that layer is ephemeral. docker commit the container back to the image once it is healthy, otherwise every recreate pays the 9 minutes again. After baking it in, a full compose down && up is back to healthy in 2-3 minutes.
The Rig
Nothing exotic. A 2016-era dual socket board: 2x Xeon E5-2682 v4, 48 GB of RAM, Debian 13, NVIDIA driver 580. The whole thing runs in a QEMU VM with both V100s passed through, so no bare metal required. No NVLink either; the layer split between the cards only needs a host RAM hand-off.
The Numbers
Unsloth UD-IQ4_XS quants, int8 KV cache, MTP spec-4, 128K context:
| Metric | 2x V100 (Strata) | 2x V100 (llama.cpp, same weights) |
|---|---|---|
| Decode (256 tok, temp 0.7) | 53.1 - 57.5 tok/s | 37.3 tok/s |
| Prompt processing | ~650 tok/s round-trip | - |
About 45% faster than the llama.cpp baseline on the same cards, with vision working and both cards sitting near 32 GB each. A small TTS container even coexisted on GPU 1 while I ran these.
The 6B-active MoE design is what makes this workable at all. With 64 GB of VRAM across the two cards, all ~59.5 GB of routed experts stay resident with no SSD spill, and Strata pins about 23 GiB of host RAM for the expert cache.
Running It Day to Day
The compose stack is in the repo. A few details that matter more than they look:
KV=int8withGPUS=0,1matches Strata’s documented V100 reference config.stop_grace_period: 120s, because Strata releases page-locked RAM slowly./modelsmounts read-only on purpose. You get one harmless “finish mark cannot be written” log line during setup because of it.
The API is OpenAI-compatible on :8184/v1 (plus an Anthropic-style /v1/messages), tool calling works, and there is a web chat UI at /. Add an API key before you expose it past your LAN. Anything OpenAI-compatible can point at it; I included an example client config in the repo.
Everything lives in strata-v100 on my GitHub: the Dockerfile patch, the compose stack, full benchmarks, and the hardware breakdown.
These Days
The same Qwen3.8-Flash-Next is also what I run day to day through PolyStrata, my own local model runner. It drives Strata’s engine for the big model and adds a second engine I wrote for everything else (GLM 5.3 Flash, DeepSeek V4 Flash, Qwen3.6, Gemma 4, and friends), so a normal gaming PC can run these too. One click installs it on Windows or Linux, and it puts an OpenAI-compatible API on localhost along with an Anthropic-style one. If running models like this at home sounds fun but two datacenter cards sounds like a lot, start there.
Credits where due: Strata for the engine and the Volta kernels, Unsloth for the UD-IQ4_XS quant, and Qwen for the model. Strata’s docs/NVIDIA_V100.md and docs/OLDER_GPUS.md are worth a read if you are touching anything pre-Turing.
Next up is pointing more of the homelab at it and seeing what breaks. The little Gemma assistant running my lights should probably watch its back.
P.S. One thing I glossed over above: cooling. Passive Tesla cards are built for server chassis airflow, so out in the open they will happily cook themselves. I started with themayeah’s Dual K80/P40/V100 80mm Fan Adapter, which does the job, but I wanted the same airflow at lower RPM, so I remixed it into my own 120mm Fan Adapter for Passive Tesla K80 / P40 / V100 Coolers. Same idea, but the manifold flares out to a 120mm flange with a shorter stack, so the fan sits closer to the card and spins slower while moving the same air. Dual, triple, and 4-slot variants are included depending on your card spacing, along with the parametric OpenSCAD source if you want to tweak it yourself. It is a derivative of the original under the same CC BY-SA license, so credit to themayeah for the starting point. And there is a fan curve script keeping the fans quiet too, more on that in a future post.