diff options
Diffstat (limited to 'building_guide/concurrency.md')
| -rw-r--r-- | building_guide/concurrency.md | 232 |
1 files changed, 0 insertions, 232 deletions
diff --git a/building_guide/concurrency.md b/building_guide/concurrency.md deleted file mode 100644 index 187b942..0000000 --- a/building_guide/concurrency.md +++ /dev/null @@ -1,232 +0,0 @@ -# Concurrency & durability - -This guide documents the runtime architecture decisions that keep live game -state safe: the single-owner loop, single-file durable money, the account -lifecycle transactions, and the graceful-shutdown sequence. - -## Single-owner game state - -One goroutine — the **owner loop** (`internal/game/owner.go`) — executes all -mutation of live game state: session input handlers, connect/disconnect -transitions, every tick callback (the master per-tick system sequence and all -dynamically registered callbacks), and admin mutations. Network and admin -goroutines submit tasks; they never touch live players, sessions, Hub -membership, or game runtime maps directly. - -Rules: - -- `Game.RunOnOwner(fn)` (and `Game.HandleSessionOnOwner` for session input) - block the caller until the closure has executed on the owner. Connection - goroutines submit one task per input line, giving per-client backpressure - without ever blocking another client. -- A FULL task queue applies backpressure: submitters block until the owner - drains a task, and a full queue never bypasses the owner — running a task - on the caller's goroutine while the owner keeps executing would break the - single-owner invariant. A STOPPED owner (graceful shutdown, after - admissions have closed) refuses the task and drops it rather than running - it on the caller's goroutine: `RunOnOwner` returns without executing the - closure, because running it there would mutate live state concurrently - with the shutdown sequence and after the final-save barrier. (Synthetic - tests that construct a Game without an owner run tasks inline on the - caller.) -- Code already running on the owner must NOT call `Game.RunOnOwner` — it - would deadlock waiting for itself. Owner-executed code calls functions - directly. -- Task execution is FIFO, so tick callbacks serialize against session input - and admin mutations in submission order. Tick callbacks still run in - engine registration-ID order (see tick_order.md); the owner wrapper only - moves their execution onto the owner goroutine. -- Slow clients cannot stall the world: `Session` writes are queued - (`net.Session.StartWriter` puts a transport session into queued mode) and a - dedicated per-session writer goroutine performs the network I/O with - per-message write deadlines. A full queue disconnects only that client. -- Saves run on the owner and serialize a value snapshot; concurrent - read-modify-write cycles of the same files cannot happen. -- Password work is the one deliberate exception: Argon2id verification and - hashing (~64 MiB, tens of milliseconds) run on the CALLER's connection - goroutine in `HandleSessionOnOwner` (`StatePassword` and - `StateNewAccount` are intercepted before any owner task is submitted). - A burst of logins therefore costs memory and CPU on connection - goroutines, never a stall of the owner's command/tick queue. The owner - task re-checks the stored hash against the one that was verified so a - password change mid-login cannot authorize a stale attempt, and - `player.hashLimiter` caps concurrent Argon2id computations at 4. - All resulting state transitions (echo, session state, account saves) - still happen on the owner. - -Unit tests construct Games without an owner; such Games run submitted tasks -inline on the caller's goroutine (legacy behavior), which is fine for -synthetic tests but production Games always have the owner (started by -`game.New`). - -## Admin editor isolation - -All `/api/` routes run through one boundary in `internal/admin/owner.go`: - -1. Authenticate, read the request body (maximum 8 MiB) on the HTTP goroutine. -2. Submit one synchronous `Game.RunAdminOnOwner` task. The **whole handler** - runs there: validation against current files, plan construction, file - commits, runtime migration/eviction, store reloads, and undo history. -3. Write the buffered response on the HTTP goroutine after the task finishes. - -Reads use the same boundary so editor API reads cannot observe half an edit. -Neither slow uploads nor slow downloads hold the owner. File I/O does: large -bulk edits can briefly pause gameplay. This deliberately avoids a second -transaction scheduler, fine-grained locks, or optimistic plan retries. Use a -maintenance window for large world reorganizations. - -`RunAdminOnOwner` rejects requests during shutdown with HTTP 503, including -requests queued before shutdown but not yet started. It never executes on the -HTTP goroutine after the owner stops. A disconnected request is skipped if it -has not started; once started, its edit completes even if the browser leaves. -A failed/lost HTTP response is not proof that the edit did not commit. - -### Endpoint inventory and callback contract - -| API family (all behind the same boundary) | Mutations / live dependencies | -| --- | --- | -| `/rooms`, `/rooms/{id}`, `/rooms/new-empty` | Room create/update/delete, room index and runtime room state | -| `/rooms/link`, `/rooms/insert`, `/rooms/remove`, `/rooms/move`, `/rooms/rename`, `/rooms/swap` | Multi-file room plans, course changes, room-ID migration or eviction where applicable | -| `/rooms/bulk-edit`, `/rooms/bulk-delete`, `/rooms/resolve-duplicate` | Bulk room changes, indexes and runtime cleanup | -| `/room-dirs/create`, `/room-dirs/rename` | Room file organization and indexes | -| `/items`, `/mobs`, `/drops`, `/hazards` and their ID routes | Definitions plus existing cache/index invalidation; items also rebuild crafting | -| `/objects`, `/techs`, `/modules`, `/triggers` and their ID routes | Definition files; existing manual-reload semantics remain unchanged | -| Entity `/rename`, `/move`, `/dirs` routes | Definition file organization, undo history, existing cache invalidation | -| `/courses`, `/courses/{id}` | Course files, exit reconciliation, course store reload | -| `/files`, `/files/rename` | Raw file data and history; NOT a semantic room-ID migration or automatic cache reload | -| `/colors`, `/colors/reset` | Persisted config and live color maps | -| `/undo/undo`, `/undo/redo` | File images, room-ID migration, index/cache rebuilds, history together | -| Remaining API routes | Read-only queries; serialized to get coherent snapshots | - -Register new endpoints on `apiMux`, never outside this boundary. Do not wrap -individual callbacks in `RunOnOwner`: they are **already on the owner**. -`cmd/thoi/main.go` wires direct course reload, craft rebuild, casino eviction, -and `MigrateRoomIDsOnOwner` callbacks. `UndoStack`'s mutex protects history, -not gameplay: its apply path and the following rebuild must remain in the -same enclosing API task. Login password rehash saves and history clearing -also run on the owner, with expensive password hashing outside it. - -This guarantees **isolation from gameplay**, not new crash-atomicity or -rollback guarantees for every editor operation. Existing file transaction -and error behavior is unchanged. Raw Files edits and external filesystem -writes still bypass semantic validation/migrations; use structured room -operations for live renumbering, and stop the game for external bulk edits. -Some definition editors still need the in-game `reload` command; serialization -does not add missing cache invalidations or migrate existing spawned entities. - -## Durable money (single-file wallet) - -Money lives in the character file and **nowhere else**. Every casino money -mutation is one atomic `AccountStore.SaveCharacter` (`persistence.WriteFile`: -temp file → fsync → rename → directory fsync) that carries both the balance -and the `Player.PendingWagers` ledger. There is no second durable store, so -there is no cross-file crash window to reconcile. - -- **Wager** — debits chips then credits and appends a `PendingWager` (its - wallet handle plus the exact chips/credits debited) in the SAME save. The - entry stays until the round resolves. -- **Settlement** — `PayCredits(amount, stakeIDs...)` credits the payout and - removes the listed stakes in the same save. A table passes the hand's stake - IDs (its payout includes their return); a slot win passes its triggering - stake (its payout is winnings only). Either way the credit and the stake - consumption are atomic, so a crash can never pay the win and refund the - stake. -- **Loss/forfeit** — `ResolveStake`/`ResolveStakes` consume the pending - stakes in one save; the round resolved and the stake is gone. -- **Refund** — `RefundOutstanding` returns every pending stake (chips in - kind, credits as credits) and clears the ledger in one save. Graceful - shutdown (`Game.runShutdown` → `Casino.RefundAll`) and eviction use it. - -A save failure **before replacement** reverts the in-memory balance and -ledger together. On `persistence.ErrDurabilityUncertain`, every wallet -operation (wager, payout, stake resolution, refund) instead keeps the applied -balance and ledger, because the visible file already carries both. The wallet -logs the storage fault and reports the operation as applied so callers do not -retry a payout or refund a resolved stake. This does not promise survival of -a subsequent OS/power crash; it prevents a later save from overwriting the -visible result with reverted memory. - -At startup (`game.New` → `recoverPendingWagers`, `internal/game/money.go`) -any stake still listed in a character's `PendingWagers` is refunded: a listed -stake means the process died before its round resolved. That is the entire -recovery story — no ordering, checkpoints, evidence, or operator queue. - -### Two-phase journal recovery is out of scope - -This game deliberately does **not** implement a durable money journal or -two-phase crash recovery. That design was built and then removed: it required -a second file (`data/economy/journal.jsonl`) plus balance checkpoints, record -ordering (`Seq`), payout `Covers` linkage, `open_payouts` evidence, and a -startup recovery pass with per-record ambiguity handling. Its entire purpose -was to make an ungraceful stop mid-round produce zero credit discrepancies, -at a cost of ~1,400 lines of engine code and ~2,000 lines of tests. - -With the crash-consistency bar deliberately lowered to "a rare ungraceful -stop may lose or refund one in-flight stake, and must never double-pay", the -single atomic character save is sufficient: a listed pending stake is -refunded, and a settlement whose save fails before replacement leaves the -stake pending and the win unannounced (never paid on top of the refund). -Post-replacement uncertainty follows the applied-state policy above. Do not reintroduce -a second money store, per-record ordering, or an operator-inspection queue -without an explicit decision to restore bank-grade guarantees. - -Currency arithmetic is checked (`internal/casino/credits.go`): explicit -`MaxCredits` cap, bounded `DefaultMaxBet` default (the explicit unlimited -sentinel is still honored), floor conversion for payout multipliers, and -overflow rejected instead of wrapping. The cap is shared by the wallet: -bank withdrawals are rejected when they would push the on-hand balance -past it, and gameplay credit grants (shop sales, pickups, activity -payouts) clamp at the cap and log rather than wrap. - -## Account lifecycle transactions - -Account and character lifecycle operations (create, rename, delete, purge) -are single journal transactions (`internal/persistence/txn.go`): a journal -entry with before-images is written durably BEFORE any file changes, the -operations apply, and the journal is removed at the commit point. Deletes -sync their parent directory; moves sync both affected directories (once if -they are the same). Rollback also syncs deletions, including retries where -the file is already absent, before retiring the journal. An apply failure -attempts rollback; if rollback fails, the journal stays for recovery. A crash -mid-commit is rolled back at the next startup by `persistence.Recover`. - -Characters carry a stable UUID (`player.Player.ID`) and owner field -(`player.Player.Account`), assigned at creation and backfilled at first -startup. Cached membership authorizes by the character file's owner, not by -its renameable filename, so a stale cached name can never authorize -connecting, renaming, or deleting a character reassigned to another account. - -After any lifecycle mutation, `Game.publishAccountUpdate` republishes the -account state to every authenticated session of that account — including -sessions sitting in menus — resetting stale pending selections. The online -check and the file mutation run as one owner step, closing the -check-then-rename gap. - -## Graceful shutdown - -Signals (SIGINT/SIGTERM) and the in-game `shutdown` command converge on -`Game.RequestShutdown`, which runs the sequence exactly once: - -1. **Admissions stop:** session input and disconnect forfeit/save callbacks - become no-ops (server-initiated shutdown is not a player disconnect). -2. **Tick stop and join:** `Engine.StopAndWait` stops the ticker and waits - for the in-flight callback to finish. -3. **Drain and persist (on the owner):** remaining queued commands execute, - outstanding wagers are refunded (`Casino.RefundAll`), and every online - character is saved. Save failures are logged for the operator. -4. **Owner stop:** the owner loop drains and stops. -5. **Transports close:** HTTP(S) servers drain, listeners close, and - sessions flush their outbound queues before their sockets drop. - `ListenAndServe` returns and main exits normally — no `os.Exit`. - -## Room-ID migration - -Live room-ID changes (admin rename, swapid) execute as one mutation plan: -the renamed room's file (id field, filename, self-loop references) plus every -other room whose references moved, applied as a single journal transaction, -followed within the same owner task by the full runtime migration -(`Game.MigrateRoomIDsOnOwner` for the editor): player -state, Hub room buckets, casino room keys, world room-keyed state (ground -items, seeded markers, object states), and live mob instances. Undo/redo -store the whole plan (all before/after images plus the idMap) and apply the -inverse/forward runtime migration through the same wired callback. |