build-in-public · receipts, not vibes

The build log.

Every claim labeled, every artifact digested, every interpretation falsifiable. A metric without its measurement method is a vibe.

The scoreboard

Browser acceptance (live production)
7/7 checks · 60.0 FPS · 32.1 Hz sim
CDP harness over the real page, GPU (ANGLE), 8 s window after 2 s warmup
Frame-perfect prediction recovery
329/329 snapshots handled by rollback replay · 0 snaps
reconcile counters from a live game session over an intercontinental link
Presence bandwidth at 200 concurrent players
89 KB/s per client (was 627) · 7× reduction
200 real WebSocket clients, measured frame sizes, all co-located (worst case)
Rules authority latency / throughput
0.20 ms p50 · 4,525 evals/s · ~4 MB RSS
200 sequential + 8-thread benchmark against the service
Failover state drift across primary loss
0.00 m
settle the avatar → scale primary to 0 → resume on the promoted standby → compare positions
Split-brain fencing
exactly one server through startup races and holder changes (epochs 1→2→3)
the world_leases table + service logs during live failure drills
Secret integrity before GitOps flip
13/13 values identical to live across 6 secrets
sops -d vs kubectl get secret, sha256 comparison (values never printed)
GDD projection fidelity
12/12 structural checks · 0 bounding-box overlaps at 500 px and 1280 px
element-count pinning against the source + rendered rectangle intersection probes
Test suite
85+ Rust tests · clippy -D warnings clean · golden-fixture wire parity
cargo test --workspace on every change

2026-10-25 · server-owned-encountersThe world started running combat, not just gravity

The training tier's authored targets — the Archive Sentinel and the Lattice Warden, stat-for-stat from the D4 encounter tier contract — are now world entities with server-owned combat state. Casts are world actions: the authoritative tick resolves them through the Sigil rules evaluation and commits the typed damage split into encounter and session state. Clients render and request; they never predict combat.

Shipped

  • {'EncounterState in the shared core': 'profiles pinned to the contract YAML and the realm placements'}
  • {'CombatResolver': "the Sigil tick evaluation inside the world loop, with the engine's shape-identical defense-in-depth fallback"}
  • damage numbers derive from BoltSpec over the baseline stat sheet — the contract's armor now actually mitigates
  • bandwidth restoration (the mana model), encounter exposure clocks, defeat rewards (the authored XP), and respawns
  • casts in the deterministic replication stream as resolved effect journals (CombatDelta)
  • encounters rendered from the wire (bodies + health bars); the demo button targets the nearest living encounter

Receipts

claimmeasuredmethod
combat resolves in the authoritysentinel 200 -> 152 health through the Sigil tick (24/hit)live WS sessions casting at enc-0 over production
24 is the contract's armor at workBoltSpec(30) vs armor 25 with baseline stats = 24 raw — the number on the wire matchesthe domain damage formula + the live snapshots
combat state is shared across playerssession two continued from session one's damage (152 -> 14)two live sessions, one encounter (encounter_verify)
the sentinel is defeatable and pays what the contract saysexactly 48 XP (the authored reward)depletion_pays_xp_and_respawns (hammered to defeat in the tick loop)
combat replays exactly on a standbyencounter state + XP bit-identical after effect-journal replaycombat_deltas_replay_exactly_on_a_replica

Bugs found on the way

bandwidth never regenerated — the mana model (restoration 8/s) was missing from the world loop, so combat locked after five castscaught by: the defeat test refusing to completeif it had shipped: a permanently exhausted (and therefore immortal) target
the replay path did not run the housekeeping clocks (restoration, exposure, respawn)caught by: design review while extending the determinism testif it had shipped: a standby would silently diverge from the primary mid-combat

Evidence

  • commits — palimpsest (this release) · cloud-ops 314c014 (image 2026-10-25-1840)

What comes next

  • encounter behaviors beyond the docile training targets (the corruption mechanics from the GDD)
  • more authored tiers — the Trial stratum is declared, not yet authored

2026-10-25 · gitops-sopsThe cluster became a projection of git

The whole Palimpsest surface moved to hearthandcode-cloud-ops as the authoritative layer (ArgoCD, sync-wave 6, prune + selfHeal), and ArgoCD learned to decrypt SOPS through a Config Management Plugin sidecar. The science-of-the-sacred flip closed the loop.

Shipped

  • {'apps/palimpsest in cloud-ops': 'site, rules service, fenced world shards, persistence, ingress'}
  • sops-kustomize CMP sidecar (argocd-cmp-sops image) on the repo-server
  • the repo-server itself brought under cloud-ops management
  • both secret surfaces fully ArgoCD-tracked (7/7 secrets Synced)

Receipts

claimmeasuredmethod
every live value survived the migration13/13 hash-identical across 6 secretsdecrypted git vs live secret, sha256, values never printed
the flip introduced no change0 rollouts, 0 pod restartspod ages before and after the sync
the fleet convergedroot, argocd-config, palimpsest, science-of-the-sacred all Syncedkubectl get applications

Bugs found on the way

the registry pull secret was double-encoded in git (base64 text in stringData)caught by: the hash audit BEFORE the flipif it had shipped: an un-audited flip would have broken sacred's image pulls at the next rollout
ignoreDifferences jsonPointers is silently dead weight in ArgoCD v3.xcaught by: live sync that refused to convergeif it had shipped: permanently OutOfSync surfaces
the CMP socket is name-version.sock but unversioned app refs dial name.sockcaught by: live comparison errorif it had shipped: plugin unusable
application plugin.version is not a field in the v3.4 CRDcaught by: root stuck OutOfSyncif it had shipped: root could never apply the Application
sops -e requires a path matching the creation rules; my first repair wrote an empty file over the encrypted originalcaught by: byte-count sanity check (16718 restored from git immediately)if it had shipped: lost secrets file (recoverable, recovered)

Evidence

  • cloud-ops commits — hearthandcode-cloud-ops f2bb492..ac0b7a9
  • the plugin and its README — cloud-ops infrastructure/argocd/

What comes next

  • a secret for ArgoCD's own install remains out-of-band by design (key material never enters git)

2026-10-25 · fencingThe world cannot fork

Silence no longer confers authority. Promotion and serving are gated on an external lease (a CAS row in Postgres with compare-and-set acquisition, TTL expiry, and an epoch fencing token); holders self-fence within one TTL of lost renewal — before any successor can be granted the lease. Serving implies lease held, at every instant.

Shipped

  • {'fencing.rs': 'Arbiter trait + MemoryArbiter + hold_lease, with the safety property tested'}
  • palimpsest-persist /lease/* routes (CAS against world_leases)
  • lease-gated promotion, demotion to follower, epoch in welcome frames
  • authority-following replication (whomever holds the lease publishes the stream)

Receipts

claimmeasuredmethod
startup races resolve to exactly one serverone "serving as … (lease epoch 1)", all contenders deniedlive rollout of both world deployments, service logs + lease table
primary loss → standby acquires → state survives0.00 m drift, epoch 3, session resumed with its idsettle → scale to 0 → resume → compare (failover_verify)
a revived primary demotes instead of forkinglease held by a live successor — demoting to standbyscale back to 1, read the pod's log
the fence holds against a live-but-partitioned holderlease denied (live holder) — remaining on standbysilence-triggered standby during a still-renewing holder

Bugs found on the way

postgres inferred $3::interval AS interval, not text — the ttl parameter failed to serialize and the error read as "denied"caught by: live lease test (the table stayed empty while everyone was denied)if it had shipped: the fence would have been inert while appearing active
two lock guards in one struct literal self-deadlocked the hello pathcaught by: the server never answering a plain helloif it had shipped: the world server freezes on first contact
an unbootstrapped standby never promoted (pre-arbiter safety rule) — all nodes waited on a stream nobody publishedcaught by: live livelock during a restartif it had shipped: total outage after any full restart
a never-bootstrapped standby seized an empty world on silencecaught by: live deploy logsif it had shipped: players entering a world with no restored state
connect() could block past the failover window and reset the silence clockcaught by: failover test timing outif it had shipped: promotion missed during a real outage

Evidence

  • commits — palimpsest e4c9eae..6fee0ae
  • the safety argument — docs/wave-3/authoritative-world-server-v0.md §Fencing

What comes next

  • graceful SIGTERM lease release is wired in the transport but not to a shutdown signal

2026-10-25 · failover-persistence-shardingThe standby reader, the ledger, and the shards

The deterministic replication stream crosses the process boundary (genesis + per-tick entries over TCP), a standby tails it and promotes on loss, Postgres remembers sessions and the world journal, and zone sharding gives each authored realm-01 zone its own authority process.

Shipped

  • {'replication.rs': 'the line-JSON replication link + standby tail/promote'}
  • palimpsest-persist (storage trait + memory double + Postgres) and the palimpsest-pg CNPG cluster
  • session persistence on detach + periodic sweeps + boot restore
  • {'PALIMPSEST_ZONE shards': 'training-floor and antechamber (real authored geometry)'}

Receipts

claimmeasuredmethod
a standby rebuilds the primary exactlybit-identical state (replica_rebuild_is_deterministic)scripted 60-tick input stream replayed on a fresh World
sessions survive primary loss and process restart23 session rows through the full world → HTTP → Postgres pathGET /sessions after live sessions, including the failover session's exact settled state
shards are isolated by constructioncross-shard players never see each other; same-shard peers dothree live connections across the two shards (shard_verify)

Bugs found on the way

the world server printed "listening" before bindingcaught by: a load test measuring the wrong processif it had shipped: misleading ops logs
rolling updates wedge on pods that are never "ready" until promotioncaught by: stuck rollout during live deployif it had shipped: deployments unable to complete
the primary selector collided with the standby's (k8s selector overlap)caught by: manifest design review before applyif it had shipped: deployments stealing each other's pods

Evidence

  • commits — palimpsest 9028e74..7ca9fc0

What comes next

  • persistence writes are best-effort (the game runs when the store is down); writes are never blocking

2026-10-25 · rollback-replayFrame-perfect prediction, proven twice

The client records each tick's exact pre-step input state (a 128-tick ring) and rebuilds its prediction from every authoritative snapshot by replaying those inputs through the shared stepping function. Under identical inputs the rebuild is bit-exact — proven as a property test and then measured live.

Shipped

  • PredictionBuffer + replay() in the shared world core
  • renderer records inputs per tick and replays first (soft convergence is now only the fallback)
  • input sequence numbers on the wire for stream correlation

Receipts

claimmeasuredmethod
the model is frame-perfect under identical inputsbit-for-bit equality with continuous stepping (rollback_replay_is_frame_perfect)200-tick scripted input stream, property test
production movement is absorbed without visible correction329/329 snapshots replayed · 0 snaps · 0 fallbackslive game session across an intercontinental link with movement held
drift cannot compoundevery replay re-anchors the prediction to the authoritydesign property, pinned by the convergence tests

Bugs found on the way

the replay tick seed used the prediction's tick, guaranteeing a mismatch by constructioncaught by: the frame-perfect property test failing on tick aloneif it had shipped: exactness would never have held
the test's capture point was off by one (a record replayed twice)caught by: the same property testif it had shipped: false confidence in the model

Evidence

  • commits — palimpsest 8a640ae, 60519f5

What comes next

  • remote-player interpolation exists; shared world objects do not

2026-10-25 · interest-managementPresence fan-out stopped being quadratic

Presence is proximity-filtered through a spatial grid (3×3 cell neighborhood, 48 m radius, 24-peer nearest-first cap) and sent as batched updates at 10.7 Hz instead of every tick. Worst-case measured at 200 co-located players.

Shipped

  • interest_set in the shared core (grid + radius + cap, four tests)
  • presence batches with Option semantics (None keeps clients interpolating; Some([]) means alone)
  • a per-tick deterministic replication stream (the future failover substrate)

Receipts

claimmeasuredmethod
the fan-out is bounded627 → 89 KB/s per client at 200 players (true average ≈ 53)200 real WS clients all co-located — the worst case
the simulation itself is not the constraint<5% of one core · 4 MB RSS at 200 playerscputime + RSS sampling around the load window
cross-player state is consistent by constructionalice sees bob and bob sees alice (positions authoritative)two live sessions (presence_verify)

Bugs found on the way

my first re-measure ran a stale binary and reported the old numbers unchangedcaught by: the numbers refusing to move after the fixif it had shipped: a wrong capacity claim (re-run on the fresh build; both runs are recorded)

Evidence

  • commits — palimpsest 8c13f23

What comes next

  • interest management enables 500+ players per shard; sharding by proximity remains the next scaling step

2026-10-25 · latency-compensationThe movement jitter, diagnosed and measured

Jitter was three specific defects: 10 Hz blind input sampling, no latency projection, and hard snaps at 0.75 m. Fixed with input-on-change delivery, a smoothed server↔client clock offset (NetClock), and soft convergence — measured before the replay model superseded it as the primary path.

Shipped

  • NetClock (NTP-style, α=1/8) + reconcile() with exponential easing
  • prompt input delivery + keepalive; auto-reconnect with resume

Receipts

claimmeasuredmethod
the corrections are invisible268 aligned · 67 smoothed · 0 snapped (each smooth would have been a teleport before)live session over the intercontinental link
connections survive hiccupssession resumed after disconnect, state preservedWS close + resume round-trip (ws_verify)

Bugs found on the way

a direct page load (no queue token) entered a reconnect stormcaught by: live probe reading 7 reconnects and 0 snapshotsif it had shipped: endless WS churn for visitors who skip the queue

Evidence

  • commits — palimpsest 8a640ae

2026-10-24 · gdd-projectionThe design document, in the site's voice

The Ember Circuit share draft was projected into the site design language (DejaVu → Fraunces, print pages → flowing sections, site chrome) with composition and imagery kept byte-faithful, the faithful print edition preserved at /gdd/print/, and a projection manifest recording source digest, transformations, and declared losses.

Shipped

  • {'tools/project_gdd.py': 'deterministic source → projection converter'}
  • /gdd/ (projection), /gdd/print/ (verbatim), /gdd/review/ (contracts), /gdd/projection-manifest.yaml

Receipts

claimmeasuredmethod
the translation preserved the document12/12 structural checks (h2/h3/card/quote counts pinned to source)element counts before and after, palette byte-identical
the cover is legible on every surface0 bounding-box intersections at 500 px and 1280 pxrendered rectangle intersection probes over every cover text block
the imagery was broken before thiscover.png and social-card.png were 404 (renamed files)live curl; the converter resolves the rename

Bugs found on the way

the cover's lower furniture is anchored to the 11 in print page (bottom:2.55in) and overlapped the title on the web surfacecaught by: user report, confirmed by bounding-box probeif it had shipped: the title block was unreadable
print-width flex rows overflowed narrow viewports (644 px content in 485 px)caught by: whole-document overflow probeif it had shipped: horizontal scroll on mobile
string patches against escaped-YAML scalars silently no-op (twice)caught by: a test asserting the change existedif it had shipped: config that looked edited but wasn't

Evidence

  • commits — palimpsest 12631d7..d65d06b

What is not proven

  • {'The MMO topology': 'world authority exists, but accounts, persistence of characters across seasons, and shared world objects are unbuilt.'}
  • The single-node cluster is a single point of failure for the platform (the world layer now fails over; the node does not).
  • Split-brain fencing holds against process and network failure; a wedged-but-renewing holder needs an external fence at the infrastructure layer (documented limit).
  • All measurements are from this build on this cluster; capacity numbers are worst-case single-shard figures, not guarantees.

How the numbers were obtained

browserCDP (Chrome DevTools Protocol) against the live page — the same harness that gates releases
loadpython websockets, N concurrent real clients, server RSS/cputime sampled around the window
secretssops -d piped to sha256; values are never printed to any log
geometryrendered bounding boxes from the live page, pairwise intersection tests
determinismproperty tests comparing bit-exact states across identical input streams