FIELD NOTES
NOTE 04 · EVIDENCE
Evidence, gates, and why I don't trust my own screenshots
In this project every “done” has to arrive with evidence. Here is what that process looks like, and what it has actually stopped.
What follows is a developer's personal record, including internal milestones, technical decisions and failures. It describes the current state — not a final form, and not a release commitment.
Working alone, the easiest person to fool is yourself
No colleagues. No code review. No QA. The only person who will say "this isn't finished" is you — and you are precisely the person least inclined to say it.
So in this project, "done" is not a feeling. It is a state that has to arrive carrying evidence.
1. Evidence is append-only
Accepted evidence_* directories are immutable. New work writes to a new isolated directory,
and the capture and validator harnesses refuse to overwrite an existing evidence directory —
not a warning, a hard failure.
Failed attempts are not deleted. They are disclosed.
In practice that produces some uncomfortable results:
- A pre-decision sealed manifest keeps its fields exactly as sealed even after approval, so
p1_a_approved=falseandp1_wp2_authorized=falsestayfalsein that artefact. The approval is a separate later record, not a rewrite of the seal. - The WP7 acceptance checklist is permanently
0/16inside its sealed artefact, because after playing the slice through I never wrote per-item notes.0/16looks bad; it is also true. The later approval lives in a different document.
I thought this was excessive at first. I now think it is the most valuable rule in the process.
2. Machine bands are necessary, not sufficient
Every new candidate ships in the same shape: a *_v2 file, a runtime option, its own validator
and its own evidence directory. Machine bands are a necessary-but-not-sufficient check.
Only a named human gate — Gate-Style-X, S10, P1-A/B — changes a default.
Which is why the repository sits for long stretches in a peculiar state: a lighting candidate passes all six of its bands, meets its performance budget and has complete evidence — and the runtime default is still the old one. Passing a metric and passing a taste judgement are different things, and I do not want the first to become the second automatically.
Concretely: lighting V3 at desktop_high passed all six bands — mean 0.359717, standard
deviation 0.209336, dark 31.2903%, bright 4.0923%, deep black 5.1532%, dark saturation
0.387926. All green. It is still an opt-in option, not the default.
3. I do not trust my own screenshots
An audit once found a static-capture depth-of-field refresh defect in a historical formal packet — meaning the screenshots used to make an approval judgement had the wrong depth-of-field state.
The packet had already been approved. What I did was not fix that packet, but add a corrected six-preset packet with its own hashes. The old 60-file slot stays byte-identical; not one byte moved.
My attitude to screenshots changed after that: screenshots are for people to look at, not for
proving things. Proof comes from the *_OK markers a validator prints, the SHA-256 manifests,
and hashes you can recompute.
That is also why every image on this site is explicitly labelled a re-rendered gameplay capture. They come from in-development builds, they have been re-rendered, and they exist to convey mood and framing — not raw screenshots, and not final quality. Saying that clearly matters more than making the images look impressive.
4. Real things run in real environments
Local development uses a locked local stack: Nakama bound to loopback, PostgreSQL private, and only a TLS ingress exposed. LAN deployment uses the same strict connection profile, the same HTTPS/WSS transport, and the same protocol, client and content identities.
On the remote side, a service restore verification in the Tokyo region (ap-northeast-1)
failed: the older stable image failed SSL verification against the private CA chain, so the
restore did not succeed.
The automatic rollback after that failure was a PASS. The safe terminal state is a different
image with the A-068 certificate, ECS 1/1/0, HTTPS 200, task definition revision 27.
The record says it plainly: a successful rollback is not "the old stable version was restored." They are different things, and calling them the same is the easiest self-deception available.
5. Three languages were there from the start
Every player-facing string goes through one catalogue (zh_CN authoritative, ja and en
complete), with identical named placeholders. The current three-language static catalogue holds
590 keys.
Locale is presentation-only and never enters online identity — there is no locale field in the protocol, and a dedicated test enforces that.
I put three languages in at the beginning rather than "adding them later" because later never arrives.
6. What this process has actually caught
- A wrong orientation yaw: probes found the Goblin's ear occupancy at zero before any expensive formal capture, killing an invalid candidate chain early.
- A zero-Event online run: the finaliser rejected and scrubbed it without inventing a cause.
- An unreadable release ACK: closure rejected the whole run; I added only a minimal, targeted hardening and wrote down explicitly that the generic failure does not prove the low-level cause.
- A depth-of-field refresh defect: handled with an additive corrected packet, with no history rewritten.
- A failed Tokyo restore: recorded as a failure plus a successful rollback, not as "restored."
In a process that only records successes, every one of those disappears.
7. The cost
This process is slow. Writing validators often takes longer than writing the feature. After something is "finished" I still have to run the chain before I am willing to say so.
What it buys is the scarcest thing available to someone working alone: I can trust the conclusions I wrote three months ago.
The game is still in development. This note is about how I know where it has got to.