How we verify our chess

The openings answer to an oracle

We never wrote our own move generator — that is how chess software goes quietly wrong. Every one of the 3,803 lines in our opening map is replayed move by move through an independent legality oracle. All 3,803 replay legally, and the move list matches the PGN on every row.

The set is then diffed, row by row, against the reference dataset the chess world already trusts: the open lichess-org/chess-openings database. 3,802 lines are byte-identical to upstream; the single exception is a deliberate, chess-valid Italian Game transposition. On top of the diff, 48 famous lines — the Najdorf, the Sveshnikov, the Marshall, the Winawer, the Grünfeld Exchange — were checked by hand across the full A00–E99 range.

Every line's ECO code, checked against its family's canonical range — scripted over the whole set, not sampled.

The PGN parser gets the ugliest legal chess we can construct: underpromotion (cxb8=N), en passant, promotion with check, three-way knight disambiguation (Ngf3 / Ndf3 / Ndxe5), castling written as 0-0, embedded clock comments, annotation glyphs, nested variations. All of it replays to the correct ply count.

The engine is checked, not just cited

Puzzle solutions are not taken on faith from their source. We re-verify a stratified sample against Stockfish 18 at depth 18 with two principal variations: the stated move must be the engine's best at every ply of the solve, with at least a 150-centipawn gap to the second-best move. Current sample: 22 of 22 pass. A sample is a sample — we say so below.

Accuracy is computed with Lichess's published win-percentage model — the same logistic curve, the same coefficients — rather than a house-brand score you can't compare to anything.

And we test the tests. The nastiest bug an eval pipeline can have is a silent sign flip — White's advantage read as Black's. So we inject exactly that bug on purpose and confirm the suite goes red. A check that cannot fail is not a check.

The statistics are attacked before they ship

A style label is a strong claim about a person. So the thresholds behind our style axes are run against simulated players first — 20,000 full records of a player with the trait switched off — to measure how often the label would fire on pure noise.

Win rates ship with Wilson 95% intervals, and a pattern that doesn't clear its interval doesn't render. Silence is a result. The full statistical doctrine, stat by stat, lives on Show the math.

We hire our own opposition

Our chess is audited continuously, and the audit is adversarial. Every internal finding is handed to a second review with one instruction: break it. Re-check the chess, re-run the position, look for the mitigation the first pass missed. In the most recent cycle, 12 of 46 findings were refuted and withdrawn — we keep score on ourselves in both directions, because a review that only ever confirms is a rubber stamp.