WARP-based worker rollout attempt (paused, pivoted to Tailscale)

Superseded by deploy v2 (BRI-338) + tailscale-acls.jsonl.

WARP-based worker rollout attempt (paused, pivoted to Tailscale)

Status: Paused 2026-07-08. The team uses Tailscale for laptop↔VPS connectivity for now. This doc captures what was tried and how to revisit.

Context: .agents/plans/2026-07-07-rollout-inference-workers-to-team.md set the rollout goal: every macOS teammate’s laptop runs the briosoco/inference worker, claiming jobs from the VPS over a CF Tunnel + WARP. The blockers plan (.agents/plans/2026-07-07-worker-home-node-blockers.md) and the code-gaps plan (.agents/plans/2026-07-08-fix-worker-code-gaps.md) shipped the required worker changes.

Pivot: The team already runs Tailscale for client work. Adding WARP on top added two DNS layers (Tailscale resolver + WARP resolver) plus Cloudflare Gateway DNS for *.brioso.internal. The DNS resolution layer turned out to be the hard part — see “Why it stalled” below. Tailscale gives us direct L3 access to VPS services with one config file (100.x.x.x → VPS), so we deferred the WARP path.

What was set up (working state on 2026-07-08)

Cloudflare side

WARP client (this laptop, MB-JXKW29RPHY)

Why it stalled: DNS resolution for *.brioso.internal

Worker runbook assumes the laptop can resolve *.brioso.internal over the WARP+tunnel stack. It cannot, out of the box. Diagnosis chain:

1. Naive assumption: dig @127.0.2.2 jobstore.brioso.internal returns NXDOMAIN. WARP’s local resolver forwards to Gateway.
2. Gateway behavior: Gateway does public recursion for brioso.internal. Root nameservers return NXDOMAIN (because it’s not a real TLD). Authority section in dig output confirms this.
3. What the docs claim: Cloudflare’s “Connect a private hostname” doc says Gateway should match Zero Trust hostname routes and return a CGNAT IP (100.80.0.0/16) for WARP clients. This didn’t happen in our setup, even with:
   • Hostname routes registered ✅
   • Tunnel healthy ✅
   • Gateway proxy enabled ✅
   • *.brioso.internal in include list ✅
   • brioso.internal removed from fallback_domains ✅
4. Root cause (per Cloudflare docs and community): Zero Trust hostname routes are an L4 mechanism primarily for SSH/RDP/SMB/gRPC, not HTTP. They route traffic after DNS has already resolved. For HTTP private hostnames via WARP, Gateway needs an additional configuration to handle DNS resolution — either a Gateway resolver policy routing queries through the tunnel to a local resolver on the VPS, or a Local Domain Fallback pointing to a specific DNS server. Neither was in the original bootstrap plan.
5. Why the bootstrap plan is incomplete: .agents/plans/2026-07-07-bootstrap-brioso-vps.md step 11 (PUT /zerotrust/settings/local-domain-fallback with brioso.internal) is the wrong endpoint and wrong semantic — that endpoint no longer exists in the current API, and adding brioso.internal to Local Domain Fallback would have shipped queries to local DNS, which has no records.

What would actually fix it (if we revisit)

Pick one of these:

Option 1 — Gateway resolver policy + local resolver on VPS (canonical Cloudflare approach)

1. Add /etc/hosts on VPS: 127.0.0.1 jobstore.brioso.internal refine.brioso.internal otel.brioso.internal
2. POST /accounts/{id}/gateway/resolver_policies with selector Host in "brioso.internal", resolvers [{"ip": "127.0.0.1", "port": 53}], network: private. Routes DNS queries through the tunnel to cloudflared’s host.
3. Verify dig @127.0.2.2 jobstore.brioso.internal +short returns 100.80.x.x.

Option 2 — Switch to a public TLD (recommended for HTTP)
Use *.brioso.com (or similar public domain you own) instead of *.brioso.internal. Cloudflare handles DNS, SSL, Access automatically. Worker .env vars (JOBSTORE_API_URL, BRIOSO_REFINE_URL) change only. Requires owning the domain and pointing it at the tunnel.

Option 3 — Local dnsmasq on each laptop (Ponytail option, simplest)
Run dnsmasq on the laptop returning 100.80.0.0/16 IPs for *.brioso.internal. Configure WARP to route 100.64.0.0/10 through the tunnel. No API or DNS changes needed server-side.

Rollback checklist (if you want to clean up Cloudflare state)

These were the only changes we made to Cloudflare config. None of these are destructive — the rollback just returns the org to a “no Brioso Worker Node profile changes” state.

• Cloudflare: Brioso Worker Node device profile → fallback_domains should NOT contain brioso.internal. (We added it then removed it; verify via dashboard.)
• Laptop: Delete /etc/resolver/brioso.internal and /etc/resolver/veena.internal:

  sudo rm /etc/resolver/brioso.internal /etc/resolver/veena.internal
  

• Laptop (optional): warp-cli disconnect. Leave the WARP client installed for now — uninstalling requires cleaning LaunchAgents and kexts.
• Verify: scutil --dns | grep -i brioso should return nothing; dig @127.0.2.2 jobstore.brioso.internal +short should fail to resolve.

Files / endpoints involved

For reference if revisiting:

• Cloudflare account id: ea4dcf9f4d3179bfead02c492d8f59e3
• Tunnel id: 3204467f-9dde-46bd-a0ac-11a50c7b5970 (vps-webapp)
• Device profile id: 4c18b9cc-b5e6-40d9-a1e0-c9e8504b56be
• API endpoints touched:
  • GET /accounts/{id}/cfd_tunnel/{id} (read tunnel config)
  • GET /accounts/{id}/cfd_tunnel/{id}/configurations (read ingress)
  • GET /accounts/{id}/zerotrust/routes/hostname (read hostname routes)
  • GET/PUT /accounts/{id}/devices/policy/{id}/fallback_domains (added then removed brioso.internal)
• Local files touched (this laptop):
  • /etc/resolver/veena.internal (created)
  • /etc/resolver/brioso.internal (created)
  • /Applications/Cloudflare WARP.app (installed via standard installer)

Re-enabling later

When the team grows beyond Tailscale’s comfort zone (~10 laptops, or compliance changes), revisit Option 1 or Option 2 above. The tunnel, hostname routes, and worker code are all in place — only the DNS resolution layer is missing. Estimated effort: 2-4 hours including team-lead review of the resolver policy.