Worker secrets sync (BRI-427)

Worker secrets sync (BRI-427)

Infisical is the only source of truth for worker-facing runtime secrets.
deploy-secrets-sync.yml renders the configured key set from Infisical and
pushes it into each configured runtime’s remote secret store on every beta
tag, then redeploys the runtime’s app so the new values actually take effect.

What syncs where

The key set and each key’s source (infisical vs literal ops config) live
in config/secrets-sync.yaml — change policy there, never in the workflow.
literal is only for non-secret config (runtime ids, model names, URLs); the
script refuses a literal under a secret-shaped name.

When it runs

• Every v*-beta.* tag push (same trigger family as the deploy pipeline).
• Manually: Actions → Worker secrets sync → target=beta, dry_run to
  preview (render + names-only drift report; no writes, no redeploy).

What a run does (per configured runtime)

1. Render staging from Infisical into $RUNNER_TEMP (0600) — same
   universal-auth export as deploy-pyinfra.yml.
2. secrets_sync.py render resolves the runtime’s keys → JSON in
   $RUNNER_TEMP (names only on stdout).
3. modal environment create beta (idempotent).
4. Pre-sync assert-live — warn-only: drift is the input to a sync, not
   a failure.
5. modal secret create -e beta --force brioso-worker --from-json … —
   replaces the whole secret (extra live keys are dropped).
6. Post-sync assert-live — hard gate: every rendered name must now be
   live or the job fails.
7. modal deploy -e beta apps/modal-worker/modal_app.py — required: the
   deployed app pins a secret snapshot, so a --forced secret is invisible
   until redeploy (this was incident 9 of the 2026-10-06 beta-tag chain).
8. Job summary: names + sha256(value) digests (D3 EnvKeyFingerprint
   convention — digests cross the wire, values never do).

Drift detection limits (deliberate)

Modal exposes no API that returns a live secret’s key names or values
(modal secret list prints secret names only). assert-live therefore uses
the server-side required_keys assert on SecretGetOrCreate — it catches a
live secret that is missing rendered keys, and nothing else. The other
direction (live has extra keys) self-heals: --force replaces the whole
secret. Value drift is unobservable through the API by design — the sync
itself is the convergence (an authoritative push each beta tag).

Ops prerequisites (one-time)

• GitHub secrets (already used by deploy-modal-worker.yml):
  MODAL_ESPETRO_TOKEN_ID / MODAL_ESPETRO_TOKEN_SECRET — deploy-scope token
  for the espetro Modal workspace.
• Infisical staging (project brioso) must carry:
  • WORKER_SHARED_SECRET — required (the existing beta value; sync
    fails loudly if absent).
  • CF_SERVICE_TOKEN_ID / CF_SERVICE_TOKEN_SECRET — optional
    (required: false): synced when present, warned-and-omitted when not.
    Add them if beta sits behind Cloudflare Access.
• JOBSTORE_API_URL, BRIOSO_WORKER_RUNTIME, BRIOSO_WORKER_MODEL are
  checked-in literals in config/secrets-sync.yaml — flip any to
  source: infisical to drive it from Infisical instead.

RunPod

Not wired. The repo has no endpoint env-update mechanism
(runpod-v2-cutover.md updates endpoint env via dashboard/saveEndpoint),
the only RunPod runtime in gpu-gateway.beta.yaml is disabled/unmerged
(kind: runpod-pod, BRI-357), and RunPod serverless endpoints are
push-runners that don’t consume this pull-worker key set. Extension point:
add a kind: runpod-endpoint runtime once the endpoint-id / RUNPOD_API_KEY
provisioning decision is made; scripts/secrets_sync.py fails loudly on
unknown kinds until then.