Chapter 6 · operations

Ops & Testing

One dev server runs the whole system locally; one GitHub push deploys it to the world. In between: a live ops console, a REST API foundation, an AI layer, and a test pyramid from pure engine units to full headless-browser walks.

Hub console (/hub)

live

The ops + corporate console: live match telemetry, bot health, and an activity feed — everything the demo runner needs on one screen.

Hub console with stats, matches table and activity feed — ops-hub.png · /hub live — stats, matches table (expandable player rows), activity feed

How it works. Polls /api/state every 3 seconds. That endpoint reads the bot match store, recomputes each player's raw/adjusted/progress/verified % server-side (same multipliers as the engine), merges bot heartbeats (fresh if seen < 45s) and the Qalarc Hub health, and returns one JSON. The feed is derived client-side by diffing successive polls — no server push needed. Degrades gracefully when anything is missing; the corporate tab (org leagues, employer-funded pots, k≥5 aggregate-only wellbeing dashboards) renders from the same state.

The match visible in the screenshot was played through the real CommandBus (the /debug simulator) minutes before capture.

lives: apps/hub/{index.html,hub.js,hub.css} · serve.ts apiState() · verified end-to-end by apps/figma-app/bots-sim.mjs (plays a match → asserts /api/state → asserts the hub renders it)

Debug console (/debug)

live

Debug bot simulator console mid-match — ops-debug.png · the bot simulator mid-match — see Bots for the transcript

What. Chat-shaped access to the real CommandBus plus the element gallery. The fastest way to demo the game end-to-end without a phone or a WhatsApp account — documented in depth in Bots.

lives: apps/debug/ · /api/sim in serve.ts (isolated scratch store)

Deploy pipeline

live

git push (main)
   └─▶ GitHub Actions: .github/workflows/deploy.yml
         ├─ bun setup
         ├─ bash scripts/build-deploy.sh        ← assembles deploy/public from the monorepo
         │     site → / · design → /design · web app → /app · hub → /hub · debug → /debug
         │     slack → /slack · connect → /connect · demo → /demo · system → /system
         │     figma-app → /figma-app · wiki → /wiki · functions/ → /api/{ai,state}
         ├─ wrangler pages deploy → Cloudflare Pages (project "rwf")
         └─ bun test packages/game-core packages/bot-core apps/api   ← 122 tests gate every push
~20 seconds later: live at rwf.qalarc.com

How it works. The deploy bundle is assembled by scripts/build-deploy.sh — pure copies, no per-app build except the web app bundle (Bun build, minified). Pages Functions (deploy/functions/api/) provide the AI proxy and state endpoint in production, mirroring the local serve.ts routes. The local dev story is one command — bun serve.ts on :4173 serves everything including this wiki at /wiki — so local and production surface the same paths.

Always-on hosting: the GMKtec minirig kit is prepared (scripts/hosting/ — install script, systemd units for the Slack bot) for bots that run without a dev machine; see blocker T3 in Status.

lives: scripts/build-deploy.sh · .github/workflows/deploy.yml · deploy/functions/api/ · scripts/hosting/ · docs/16_HOSTING.md

The REST API (apps/api)

built · unwired

What. A unified backend for crews, players, matches and seasons — the foundation that replaces the app's localStorage and the bots' file state with one source of truth.

How it works. Bun server on :4174 with CORS for the dev origin and production domain. Persistence is a single JSON file (.data/api-db.json) written atomically (tmp + rename) so a crash never corrupts the store. Crew records carry the 5-char join codes (no 0/O/1/I) that match the CREW-XXXX grammar the app and bots already display. The wiring step (app → API ← bots) is mechanical and tracked as blocker T5.

lives: apps/api/src/{main,routes,db}.ts
proven by: apps/api/test/api.test.ts — 24 tests, run in CI

AI layer

live · metered

What. GLM-5.3 behind a server-side proxy — powers the site's AI guide, the app narrator and the bots' taunts/digest one-liners.

How it works. Clients post to /api/ai; the key lives server-side only (local .env / Pages Function secret) and is never shipped to a client. The proxy injects a system prompt describing the whole product, keeps the last 12 messages, caps content length, rate-limits 60/min, preserves upstream error classes (429 ≠ 502), and every consumer has a canned fallback — if the AI is asleep the product still talks. Locally the same endpoint is served by serve.ts; in production by the Pages Function.

lives: serve.ts aiChat() · deploy/functions/api/ai.js · packages/bot-core/src/ai.ts (2s timeout + fallback)

The test suites

~700 checks green

Unit → integration → headless-browser. Every push runs the packaged suites; the E2E walks run on demand and produce the screenshots this wiki is built from.

Suite	Tests	What it proves	Run
game-core — ENGINE V4	110	the daily model: createDay/logSet/closeDay, first-to-target wins, bank/fail outcomes, streaks, seasons 1:1, stakes lifecycle, SOT power-up canon (pure-gain steal, shield saves, bomb fuse)	bun test packages/game-core
bot-core — bus + gfamily + SOT additions	89	command grammar, aliases, cards, store behaviour, error cards, rematch/nemesis/digest, the daily-model additions	bun test packages/bot-core
api — api.test.ts	24	REST routes, crew codes, atomic persistence	bun test apps/api
sot (v4) — e2e.mjs	120 checks	the full SOT product walk on the real app: onboard → wizard → season → steal → logs → Daily Wins → bombs → charity resolution → join flow; language + console gates; zero console errors	bun apps/sot/e2e.mjs
sot (v4) — e2e-states.mjs	59 checks	the critical states: offline banner + queued logs + badge, reconnect replay, sync conflict + duplicate warning sheets (confirm/undo), rest-day home, winner-known + battle-complete transitions; zero console errors	bun apps/sot/e2e-states.mjs
figma-app (v1) — engine.test.js	23	the 300-era engine port matches the spec (incl. power-ups)	bun test apps/figma-app/engine.test.js
figma-app (v1) — e2e / e2e-daily / e2e-squads	122 + 74 + 90	the v1 product walks: full journey, the temporal DZ loop with time travel, squads + wagers	bun apps/figma-app/e2e.mjs …
board (v2) — e2e / geom	56 + 27	the board presentation + lane geometry (centred, 1.2–1.4ms frames)	bun apps/board/{e2e,geom}.mjs
v3 — e2e / geom	69 + 35	the 3D course + SFX wiring	bun apps/v3/{e2e,geom}.mjs
figma-app (v1) — bots-sim.mjs	—	plays a real match through the CommandBus then asserts /api/state + the hub render it	bun apps/figma-app/bots-sim.mjs
wiki — walk.mjs	25 checks	this wiki: 10 pages load with zero console errors, every referenced asset resolves, every PNG decodes	bun apps/wiki/test/walk.mjs

CI runs the packaged suites on every deploy push — a red suite blocks nothing at Cloudflare but fails the workflow loudly. The E2E walks are the design contract: if a screen breaks, its screenshot can't be taken. v4's two walks were the wiki's own source material — 120 + 59 checks re-run for this page's screenshots.

lives: the test dirs above · CI gate in .github/workflows/deploy.yml · v4 walks measured 4 Sep 2026: 120 + 59 pass · zero console errors