Home GPU inference server setup (BRI-249)
Superseded by apps/worker/ + home-worker-local-run.md.
Home GPU inference server setup (BRI-249)
Phase 1 of BRI-248 — turning the
home PC’s RTX 4070 Ti Super into a dedicated, headless Ubuntu Server running
apps/worker, the poll-based home GPU runner (see that package’s own code
and apps/worker/AGENTS.md for the worker itself). Phase 2 (benchmarking,
BRI-250) and Phase 3 (BRI-251, folding the earlier push-based HTTP server
into this poll worker) are complete — this box now runs apps/worker under
pm2, with no inbound HTTP surface.
This doc covers the parts that need physical/BIOS access to the box and
can’t be done from a coding session: disk wipe, OS install, boot order, base
hardening, and the NVIDIA/CUDA stack. For the worker’s own code, env vars,
and pm2 supervision, see apps/worker/AGENTS.md.
Decision recap
The home PC dual-drives Windows: Drive C has Windows, Drive D is a second
physical disk with only a disposable Steam library. Nothing on either drive
needs preserving. A same-drive dual-boot was ruled out (risks Windows during
partition resize, can hang on a boot menu with no monitor attached — breaks
the “auto-starts after power loss” requirement). Instead: wipe Drive D,
install Ubuntu Server there as a fully separate disk, and set the BIOS to
boot Drive D by default. Drive C / Windows is never touched, and stays
reachable later via the BIOS’s one-time boot-device menu.
0. Pre-flight check
Before wiping anything, confirm from BIOS or Windows Disk Management:
- Drive D is a real second physical disk, not a partition on the same disk
as C. - It has 100-200GB free (plenty for OS + venv + model weights).
- Note whether it’s SSD or HDD — doesn’t block anything, just useful context
for Phase 2 benchmarking.
1. Wipe Drive D and install Ubuntu Server 24.04 LTS
- Delete the Steam library on Drive D.
- Boot the Ubuntu Server 24.04 LTS installer from USB.
- Use manual/custom partitioning and explicitly select Drive D as the
install target. Do not let the installer touch Drive C. - During install: enable OpenSSH server, create a non-root user with sudo.
2. BIOS boot order
- Set Drive D (Ubuntu) as the default/primary boot device.
- Enable “restore power state on AC loss” / “power on after power failure”
if the BIOS supports it — this is what satisfies the “auto-starts after
power loss” requirement (BRI-249 DoD #9-10). - Leave Windows on Drive C selectable via the BIOS’s one-time boot menu
(commonly F8/F11 at boot) — no shared bootloader, nothing to configure on
the Windows side.
3. Base OS setup
sudo apt update && sudo apt upgrade -y
# Unattended security patches
sudo apt install -y unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades
# Firewall — allow SSH only. The worker is a poll-based outbound client
# (apps/worker), not a server — it opens no inbound port, so nothing else
# needs a firewall exception.
sudo apt install -y ufw
sudo ufw allow OpenSSH
sudo ufw enable
# Time sync (systemd-timesyncd ships enabled by default on 24.04; verify)
timedatectl status
# Disk monitoring
sudo apt install -y smartmontools
sudo systemctl enable --now smartd
Static/reserved LAN IP: prefer a DHCP reservation on the router (simplest,
survives netplan changes across reinstalls). If you’d rather set it on the
box, edit /etc/netplan/50-cloud-init.yaml (or whatever the installer
generated) with a static addresses:/routes:/nameservers: block and run
sudo netplan apply.
Hostname: sudo hostnamectl set-hostname <name> if the installer’s default
isn’t what you want.
journald is enabled by default. pm2 (section 6 below) manages its own log
files under ~/.pm2/logs/; brioso logs tails those directly, no journald
integration needed.
4. NVIDIA driver + CUDA
packages/stable-audio-3/pyproject.toml pins torch==2.7.1 /
torchaudio==2.7.1 off the pytorch-cu128 index for Linux — the driver and
CUDA toolkit need to support CUDA 12.8.
# Driver — ubuntu-drivers picks the recommended version for the installed GPU
sudo apt install -y ubuntu-drivers-common
sudo ubuntu-drivers install
sudo reboot
# After reboot, confirm the driver loaded
nvidia-smi
# Expect: RTX 4070 Ti Super, 16384MiB total, temps/power draw visible
# CUDA toolkit (nvcc etc.) — only needed if something requires compiling
# against CUDA headers; the PyTorch wheels bundle their own CUDA runtime.
# Install if the `apps/worker` setup below hits a missing-toolkit
# error (Triton JIT-compiles a small driver shim at runtime — see
# apps/runpod/Dockerfile.sa3's build-essential comment for why that needs a
# real GPU present, which this box has and RunPod's build stage doesn't).
sudo apt install -y build-essential
Validate CUDA is visible to PyTorch once the Python env below is set up:
uv run --project apps/worker python -c \
"import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
# Expect: True NVIDIA GeForce RTX 4070 Ti Super
5. Python environment / repo checkout
Mirrors .agents/docs/home-worker-local-run.md’s prerequisites and
apps/runpod/Dockerfile.sa3’s dependency sequence (same pytorch-cu128
pin, same flash-attn wheel-matching, same build-essential-at-runtime
gotcha for Triton).
Git auth: the repo is private and org policy disables repo-scoped deploy
keys (“Disabled by briosoco” under Settings → Deploy keys) — use a
fine-grained Personal Access Token instead (https://github.com/settings/personal-access-tokens/new,
resource owner briosoco, repository access limited to brioso, Contents:
Read-only). Clone over HTTPS and enter the token as the password when
prompted; git config --global credential.helper store caches it for future
pulls (only takes effect on prompts after it’s set — if you configured it
after the first clone, the next git fetch/pull will prompt once more,
then it’s cached in ~/.git-credentials).
curl -LsSf https://astral.sh/uv/install.sh | sh
source ~/.local/bin/env # puts uv on PATH for this shell; new shells pick it up automatically
sudo apt install -y git ffmpeg build-essential
sudo mkdir -p /opt/brioso-inference
sudo chown "$USER:$USER" /opt/brioso-inference
git clone https://github.com/briosoco/brioso.git /opt/brioso-inference/repo
cd /opt/brioso-inference/repo
# apps/worker is a member of the repo's root uv workspace — see
# apps/worker/AGENTS.md for the full install (uv pip install -e apps/worker
# plus the stable-audio-3/musicgen-adapters/inference-core packages it needs
# at generation time). This installs sa3-worker, inference-core,
# musicgen-adapters, and the pytorch-cu128 torch/torchaudio build for linux
# into the shared workspace .venv.
mise trust
bun install
uv sync --all-packages
Repo root’s .python-version pins 3.12 — needed because Ubuntu 26.04’s
default python3 is 3.14, and torch==2.7.1+cu128 only ships wheels for
cp312/cp313/cp313t. Without the pin, uv sync picks the system 3.14
interpreter and fails with “doesn’t have a source distribution or wheel for
the current platform”. uv downloads its own managed CPython 3.12
automatically the first time, no manual Python install needed.
flash-attn is not part of the uv-resolvable dependency set (no prebuilt
wheel on PyPI) — apps/runpod/Dockerfile.sa3 installs it as a separate step
against a matching cu12/torch2.7/cxx11-abi wheel from the flash-attention
GitHub releases. Only needed if sa3_worker.model ends up requiring it at
import time; skip until proven necessary — stable-audio-tools==0.0.20’s
default attention path does not hard-require it.
6. Run the worker
Supervision is pm2, the same on every platform (see
apps/worker/AGENTS.md’s “Supervision” section) — no systemd unit, no
per-box service file to copy or chown.
The worker reads plain env vars (not Infisical) from its own .env in
apps/worker/. Copy apps/worker/.env.example and fill in at minimum:
cd /opt/brioso-inference/repo/apps/worker
cp .env.example .env
SA3_MODEL_ID=stable-audio-3-medium is a gated HuggingFace repo — you’ll
need an account that has accepted the license at
huggingface.co/stabilityai/stable-audio-3-medium, and a read-scope token
from huggingface.co/settings/tokens:
HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxx
Also set WORKER_SHARED_SECRET (must match the backend’s value — pull it
from Infisical dev/prod, this box is not on Infisical) and
JOBSTORE_API_URL=https://usebrioso.com. No tailnet, no admin request — the
worker reaches the backend over the public API the same way any HTTPS
client does; see apps/worker/AGENTS.md’s “Environment” section for the
full variable list.
Then start it under pm2:
pm2 start deploy/pm2/ecosystem.config.cjs
pm2 save
pm2 startup # prints a native init command; run the command it prints
Watermarking (packages/inference-core/src/inference_core/watermark.py,
embed_watermark) is not wired into the generation path — it isn’t called
from either RunPod’s handler or the home worker’s live path today either.
Open question for the team before this box’s output reaches production
catalog tracks, not a setup blocker.
7. Verify (BRI-249 DoD)
brioso status # human summary: pm2 process state
brioso status --json # same, machine-readable — useful from a coding session
brioso logs --lines 50 # confirm the claim loop is polling, and the model
# load log line appears only once across jobs
brioso engine # prints SA3/diffusers on this box (CUDA)
Presence telemetry (the /health replacement — state, model_loaded,
accelerator, idle seconds, load_error) lands in Phoenix rather than a
local endpoint; see apps/worker/AGENTS.md for where to look. To confirm
end-to-end generation, insert a runner="home" job against the real
backend (JOBS_DEFAULT_RUNNER=home on apps/web routes new jobs here
without manual row patching) and confirm it reaches paid with tracks in
R2 — a second identical job should reuse the resident model (no reload log
line the second time).
8. Unattended reboot test
sudo reboot
With zero manual intervention afterward, confirm:
- BIOS boots Drive D (Ubuntu), not the boot menu waiting on input.
- The box comes up on the network at its reserved IP and SSH is reachable.
brioso statusshows the worker running — pm2’spm2 startupunit
restarts it automatically, no per-service systemd config needed.- The model auto-loads without any manual step —
brioso logsreaches a
ready/ engine-loaded line within a few minutes of the box coming up.
This is what satisfies DoD items 9-10 (“service survives/restarts after
reboot”, “after reboot, machine automatically reaches READY with model
loaded”).