path: root/building_guide/concurrency.md

# Concurrency & durability

This guide documents the runtime architecture decisions that keep live game
state safe: the single-owner loop, the durable money journal, the account
lifecycle transactions, and the graceful-shutdown sequence.

## Single-owner game state

One goroutine — the **owner loop** (`internal/game/owner.go`) — executes all
mutation of live game state: session input handlers, connect/disconnect
transitions, every tick callback (the master per-tick system sequence and all
dynamically registered callbacks), and admin mutations. Network and admin
goroutines submit tasks; they never touch live players, sessions, Hub
membership, or game runtime maps directly.

Rules:

- `Game.RunOnOwner(fn)` (and `Game.HandleSessionOnOwner` for session input)
  block the caller until the closure has executed on the owner. Connection
  goroutines submit one task per input line, giving per-client backpressure
  without ever blocking another client.
- A FULL task queue applies backpressure: submitters block until the owner
  drains a task, and a full queue never bypasses the owner — running a task
  on the caller's goroutine while the owner keeps executing would break the
  single-owner invariant. Only a STOPPED owner (graceful shutdown, after
  admissions have closed) lets stragglers execute inline.
- Code already running on the owner must NOT call `Game.RunOnOwner` — it
  would deadlock waiting for itself. Owner-executed code calls functions
  directly.
- Task execution is FIFO, so tick callbacks serialize against session input
  and admin mutations in submission order. Tick callbacks still run in
  engine registration-ID order (see tick_order.md); the owner wrapper only
  moves their execution onto the owner goroutine.
- Slow clients cannot stall the world: `Session` writes are queued
  (`net.Session.StartWriter` puts a transport session into queued mode) and a
  dedicated per-session writer goroutine performs the network I/O with
  per-message write deadlines. A full queue disconnects only that client.
- Saves run on the owner and serialize a value snapshot; concurrent
  read-modify-write cycles of the same files cannot happen.
- Password work is the one deliberate exception: Argon2id verification and
  hashing (~64 MiB, tens of milliseconds) run on the CALLER's connection
  goroutine in `HandleSessionOnOwner` (`StatePassword` and
  `StateNewAccount` are intercepted before any owner task is submitted).
  A burst of logins therefore costs memory and CPU on connection
  goroutines, never a stall of the owner's command/tick queue. The owner
  task re-checks the stored hash against the one that was verified so a
  password change mid-login cannot authorize a stale attempt, and
  `player.hashLimiter` caps concurrent Argon2id computations at 4.
  All resulting state transitions (echo, session state, account saves)
  still happen on the owner.

Unit tests construct Games without an owner; such Games run submitted tasks
inline on the caller's goroutine (legacy behavior), which is fine for
synthetic tests but production Games always have the owner (started by
`game.New`).

## Admin editor isolation

All `/api/` routes run through one boundary in `internal/admin/owner.go`:

1. Authenticate, read the request body (maximum 8 MiB) on the HTTP goroutine.
2. Submit one synchronous `Game.RunAdminOnOwner` task. The **whole handler**
   runs there: validation against current files, plan construction, file
   commits, runtime migration/eviction, store reloads, and undo history.
3. Write the buffered response on the HTTP goroutine after the task finishes.

Reads use the same boundary so editor API reads cannot observe half an edit.
Neither slow uploads nor slow downloads hold the owner. File I/O does: large
bulk edits can briefly pause gameplay. This deliberately avoids a second
transaction scheduler, fine-grained locks, or optimistic plan retries. Use a
maintenance window for large world reorganizations.

`RunAdminOnOwner` rejects requests during shutdown with HTTP 503, including
requests queued before shutdown but not yet started. It never executes on the
HTTP goroutine after the owner stops. A disconnected request is skipped if it
has not started; once started, its edit completes even if the browser leaves.
A failed/lost HTTP response is not proof that the edit did not commit.

### Endpoint inventory and callback contract

| API family (all behind the same boundary) | Mutations / live dependencies |
| --- | --- |
| `/rooms`, `/rooms/{id}`, `/rooms/new-empty` | Room create/update/delete, room index and runtime room state |
| `/rooms/link`, `/rooms/insert`, `/rooms/remove`, `/rooms/move`, `/rooms/rename`, `/rooms/swap` | Multi-file room plans, course changes, room-ID migration or eviction where applicable |
| `/rooms/bulk-edit`, `/rooms/bulk-delete`, `/rooms/resolve-duplicate` | Bulk room changes, indexes and runtime cleanup |
| `/room-dirs/create`, `/room-dirs/rename` | Room file organization and indexes |
| `/items`, `/mobs`, `/drops`, `/hazards` and their ID routes | Definitions plus existing cache/index invalidation; items also rebuild crafting |
| `/objects`, `/techs`, `/modules`, `/triggers` and their ID routes | Definition files; existing manual-reload semantics remain unchanged |
| Entity `/rename`, `/move`, `/dirs` routes | Definition file organization, undo history, existing cache invalidation |
| `/courses`, `/courses/{id}` | Course files, exit reconciliation, course store reload |
| `/files`, `/files/rename` | Raw file data and history; NOT a semantic room-ID migration or automatic cache reload |
| `/colors`, `/colors/reset` | Persisted config and live color maps |
| `/undo/undo`, `/undo/redo` | File images, room-ID migration, index/cache rebuilds, history together |
| Remaining API routes | Read-only queries; serialized to get coherent snapshots |

Register new endpoints on `apiMux`, never outside this boundary. Do not wrap
individual callbacks in `RunOnOwner`: they are **already on the owner**.
`cmd/thoi/main.go` wires direct course reload, craft rebuild, casino eviction,
and `MigrateRoomIDsOnOwner` callbacks. `UndoStack`'s mutex protects history,
not gameplay: its apply path and the following rebuild must remain in the
same enclosing API task. Login password rehash saves and history clearing
also run on the owner, with expensive password hashing outside it.

This guarantees **isolation from gameplay**, not new crash-atomicity or
rollback guarantees for every editor operation. Existing file transaction
and error behavior is unchanged. Raw Files edits and external filesystem
writes still bypass semantic validation/migrations; use structured room
operations for live renumbering, and stop the game for external bulk edits.
Some definition editors still need the in-game `reload` command; serialization
does not add missing cache invalidations or migrate existing spawned entities.

## Durable money (casino wallet)

Every wager and payout is journaled before the balance mutation
(`internal/game/economy.go`, `data/economy/journal.jsonl`):

1. A durable journal record with balance checkpoints (before/after credits
   and chips) is appended and fsynced.
2. The in-memory balance mutates.
3. The character save makes the balance durable.

Two record kinds exist:

- **wager** — a committed stake. The record stays OPEN while the stake is
  outstanding (round in flight) and is resolved when the round settles:
  won, lost, pushed, forfeited (disconnect), or refunded (shutdown). Games
  resolve stakes through `casino.Wallet.ResolveStakes` after settlement
  (or per hand via `ResolveStake`); `casino.Wallet.RefundOutstanding`
  returns every outstanding stake (chips as chips, credits durably) at
  shutdown/eviction, exactly once. A stake already covered by an open
  settlement payout is resolved WITHOUT a refund: a table settlement
  includes its hand's stake return and a slots payout proves the round
  resolved, so refunding would double-pay the round — recovery re-applies
  the owed payout instead, and the covered stakes receive their money
  through it.
- **payout** — money movement (winnings, pushes, refunds). The record
  closes when the balance mutation is durable.

A failed save reverts the in-memory mutation and voids the record — the
game must not proceed on a stake that could not be made durable, and the
player is told the house could not process the transaction (distinct from
an insufficient-funds message). A failed payout keeps its record open (the
reported win is never announced as paid) and startup recovery re-pays the
owed settlement.

If the JOURNAL ITSELF cannot be appended (persistent write failure, e.g. a
deleted or unwritable `data/economy`), appends are retried and then the
wallet degrades safely:

- A **wager** is never charged without a journal record — the bet is
  rejected with the persistence message (unchanged).
- A **payout** falls back to paying through the character file alone (the
  save is the commit point), so the settlement is real, announced, and
  durable even with no journal record. `ResolveStakes` then records the
  unresolved stake IDs in the character file (`player.Player
  .ResolvedStakes`) as durable resolution evidence, and recovery settles
  those stakes without refunding them (no double pay, and a settlement's
  chip stake is not returned in kind on top of credits that already
  returned it). If the character save fails too, no evidence exists and
  the open wager record drives the ordinary refund-on-lost-state recovery.
- A journal that cannot be fully READ (oversized or unreadable line) or
  that contains a malformed record with later records present aborts
  recovery and compaction entirely: records are preserved verbatim for
  operator inspection instead of compensating over a partial fold or
  truncating the file. A torn FINAL line (crash mid-append) is still
  tolerated and skipped.

At startup (`game.New`), open records are compensated in two phases:

1. Open **payout** records (append order, evolving balance): file at the
   pre-payout balance → re-apply (the player is owed it); at the post-payout
   balance → already durable, settle; anything else stays open and is
   surfaced for operator inspection — a liability is never silently
   dropped.
2. Open **wager** records (newest first, so each refund restores the
   balance the previous record's checkpoints refer to): file at the
   pre-debit state (credits AND chips) → void (nothing was charged); at
   the post-debit state → the stake is **refunded** (the round state was
   lost in the crash) — unless a later payout record proves the round
   resolved (the win/push/refund already returned the stake inside that
   payout), in which case only the lost resolution marker is re-applied.
   Settlement payouts may carry an explicit `covers` list naming the stake
   IDs they return (a table hand's payout): recovery then resolves
   exactly those stakes. A payout without a list (legacy records, slots
   winnings) resolves every open stake of its player — a journaled
   settlement means the round resolved, and its re-application delivers
   the money. Ambiguous records stay open for operator inspection.

Currency arithmetic is checked (`internal/casino/credits.go`): explicit
`MaxCredits` cap, bounded `DefaultMaxBet` default (the explicit unlimited
sentinel is still honored), floor conversion for payout multipliers, and
overflow rejected instead of wrapping. The cap is shared by the wallet:
bank withdrawals are rejected when they would push the on-hand balance
past it, and gameplay credit grants (shop sales, pickups, activity
payouts) clamp at the cap and log rather than wrap.

## Account lifecycle transactions

Account and character lifecycle operations (create, rename, delete, purge)
are single journal transactions (`internal/persistence/txn.go`): a journal
entry with before-images is written durably BEFORE any file changes, the
operations apply, and the journal is removed at the commit point. Any
failure rolls back every applied operation; a crash mid-commit is rolled
back at the next startup by `persistence.Recover`.

Characters carry a stable UUID (`player.Player.ID`) and owner field
(`player.Player.Account`), assigned at creation and backfilled at first
startup. Cached membership authorizes by the character file's owner, not by
its renameable filename, so a stale cached name can never authorize
connecting, renaming, or deleting a character reassigned to another account.

After any lifecycle mutation, `Game.publishAccountUpdate` republishes the
account state to every authenticated session of that account — including
sessions sitting in menus — resetting stale pending selections. The online
check and the file mutation run as one owner step, closing the
check-then-rename gap.

## Graceful shutdown

Signals (SIGINT/SIGTERM) and the in-game `shutdown` command converge on
`Game.RequestShutdown`, which runs the sequence exactly once:

1. **Admissions stop:** session input and disconnect forfeit/save callbacks
   become no-ops (server-initiated shutdown is not a player disconnect).
2. **Tick stop and join:** `Engine.StopAndWait` stops the ticker and waits
   for the in-flight callback to finish.
3. **Drain and persist (on the owner):** remaining queued commands execute,
   outstanding wagers are refunded (`Casino.RefundAll`), and every online
   character is saved. Save failures are logged for the operator.
4. **Owner stop:** the owner loop drains and stops.
5. **Transports close:** HTTP(S) servers drain, listeners close, and
   sessions flush their outbound queues before their sockets drop.
   `ListenAndServe` returns and main exits normally — no `os.Exit`.

## Room-ID migration

Live room-ID changes (admin rename, swapid) execute as one mutation plan:
the renamed room's file (id field, filename, self-loop references) plus every
other room whose references moved, applied as a single journal transaction,
followed within the same owner task by the full runtime migration
(`Game.MigrateRoomIDsOnOwner` for the editor): player
state, Hub room buckets, casino room keys, world room-keyed state (ground
items, seeded markers, object states), and live mob instances. Undo/redo
store the whole plan (all before/after images plus the idMap) and apply the
inverse/forward runtime migration through the same wired callback.