Textbook

The engine's content lane for curriculum books. Say it in prose — "convert the grade 12 textbooks", "verify the biology twin", "textbook bench" — and the /textbook runbook activates with the transcribe and adjudicate agents and two saved workflows.

kun holds the harness, not the books. The books live in the catalog repo (~/catalog, databayt/catalog), whose root is cdn.databayt.org/catalog/; kun holds the contract template, the agents, the scorers, the workflows, the benchmark corpus and the measured record. (Until the 2026-09-27 cutover they sat in hogwarts' gitignored curriculum/ tree — that tree is now dead, and authoring there diverges silently.) A book must pass the catalog's book gate before it earns a twin: cd ~/catalog && pnpm gate sd --grade g12. Nothing in the harness knows a subject: headings come from the book's structure.json, the language decides direction and the illegible marker, and the benchmark is per book. The Grade 12 Sudanese biology book is the first benchmark, not a special case.

The problem is verification, not transcription

A vision model reads a scanned Arabic page well — on sd/g12/biology it recovered 180,323 Arabic characters, 7,324 Latin ones, 276 table rows and 144 captioned figures where tesseract produced 143,153 / 0 / 6 / 0. The hard part is knowing which of those characters are wrong. Two failures shaped the design:

• A coverage grade certified garbage. The old grader counted whether output looked like Arabic letters, so a 47 %-complete OCR twin scored "A".
• Transcribers repair the source. One "corrected" two printed figure numbers by cross-reference and normalised ڤ to ف, reporting the fixes as improvements. Another added interpretive words to raw diagram labels; another composed a caption for an uncaptioned figure; another rendered a Punnett cross as a Markdown table.

So the twin is graded by agreement between two independent reads — and, because one number hides the defect, by six axes and a failure class per page.

The pipeline

The harness is .claude/workflows/textbook.js (Prepare → Transcribe → Assemble → Verify → Adjudicate → Persist); the benchmark runner is .claude/workflows/textbook-bench.js. Both are Workflow-tool scripts: deterministic control flow, agents only where a page has to be read.

Predictable URLs

Every subject serves exactly two textbook files, and both are derived from the subject's id — no URL fields, no override table, no per-book exception:

https://cdn.databayt.org/catalog/<curriculum>/<grade>/<subject>/textbook.pdf
https://cdn.databayt.org/catalog/<curriculum>/<grade>/<subject>/textbook.md

sd-g12-biology is …/catalog/sd/g12/biology/textbook.pdf and …/textbook.md. The catalog builds them with catalogKey + catalogUrl; hogwarts stores only Subject.pdf and derives the twin with catalogSibling(pdfKey, "textbook.md"). textbook status checks the contract live:

Copycd ~/catalog && pnpm twins sd --grade g12 --cdn
# 25 subjects — pdf 25 · md 25 · vision 8 · twin gate 8 · live 25/25 · md drift 0   (2026-10-10)

It exits non-zero when a subject lacks either file, or when the CDN disagrees with the repo — on its first run it caught biology's adjudicated twin (0.952) sitting unpublished behind the 0.948 copy. CI runs the offline half on every push.

Run it

Three calls cover every use. The book path is absolute; every other argument has a default that comes from engine.json.

Copy// a new book — render, transcribe, verify, adjudicate, persist
Workflow({ name: "textbook", args: { book: "/Users/abdout/catalog/.work/sd/g12/biology", mode: "full", date: "2026-09-11" } })
 
// a twin already on disk — verify and repair only
Workflow({ name: "textbook", args: { book: "/Users/abdout/catalog/.work/sd/g12/biology", mode: "verify", seed: 11, date: "2026-09-11" } })
 
// the benchmark, then the transcript audit — always both
Workflow({ name: "textbook-bench", args: { book: "sd-g12-biology", label: "contract-v2", date: "2026-09-11" } })
// node ~/kun/.claude/scripts/audit-textbook-run.mjs --latest --out-dir <run dir>

One measured gotcha: the typed agents can fail to resolve in the session that created them. Both workflows then fall back to a general-purpose agent that reads the agent file first and is forbidden to consult anything — a general-purpose adjudicator once reached the right verdict on a page and then stalled for twenty minutes inside its own advisor call, while the typed adjudicate agent (Read, Write, Glob only) settled the same twelve hunks in six minutes.

What the numbers mean

Every sampled page also gets a class, and the class decides the repair, not the mean: structure (tables, dims, figures, captions or label sets differ) · omission (a run of six or more words in one read only) · numeric · latin · illegible (one read marked, the other wrote text — a guess suspect) · ordering (same words, different order — noise unless the contract defines an order) · wording · clean. Only the first six enter the queue.

The headline is the random sample's seq agreement, nothing else. The risk sample is the tail and is reported beside it; post-adjudication text is not re-scored as independent, because the adjudicator saw both reads. Thresholds A ≥ 0.95 · B ≥ 0.90 · C ≥ 0.80 are provisional — one book, n = 21, opus — and are re-derived once the benchmark carries three books.

What "structure fidelity" means

Structure is scored on the reader's own grammar — the block regexes of hogwarts textbook/parse.ts are ported into md-structure.py, so a "table" is exactly what a student sees rendered as a grid. Three structural defects are detected without a model:

• diagram-as-table — a Markdown table on a page with no ruled or shaded grid. A PIL long-run detector finds rules and shaded bands; a boxed diagram has two rules per axis, a table has more.
• composed caption — alt text that carries no printed figure number.
• mirrored table — column 1 must be the rightmost printed column under dir="rtl". Decided by cropping the right 45 % of the page and asking only what is in the crop; on the first book 8 of 31 tables were mirrored and 23 were right, so nothing is ever blanket-swapped.

Unreadable labels

Figures are embedded at native resolution but painted small: biology page 60 holds a 1237 px flower diagram drawn at ~420 px in the render, which is why six labels read as [غير مقروء]. textbook-crop.py native reports the gain; zoom re-renders each figure at native density (2.95× there) and the adjudicator reads the crop. When the native image is no larger than the render, the tool says so and the harness keeps the mark — marking uncertainty is never penalised.

The benchmark

.claude/evals/textbook/manifest.json — 30 pages of sd-g12-biology chosen for the failure classes: dense labels, unreadable labels, diagram-as-table, RTL and notation tables, multi-figure pages, ordering and interpretation, repair traps, edge and control pages. Each case carries objective assertions a person can check against the scan: raw mustContain / mustNotContain ((الصورة 51) must survive; العمود الأيمن must not appear), numeral and Latin sets, an order-insensitive label set, table dimensions and the crop-verified rightmost column, noTable, figure count, illegibleMin/Max. Gold text sits beside them as a secondary signal.

A run is a blind transcription into bench-runs/<label>/; textbook-bench.py score is deterministic; audit-textbook-run.mjs scans the transcripts and voids a run that opened an earlier read. Every entry records the gold hash, the contract hash and the model, so "the contract change helped" is a comparable number — and only under the same gold hash and the same contract hash. Anything else starts a new baseline, and the entry says so.

Adding a book to the corpus

1. textbook-bench.py propose seeds cases from two reads on disk: what both reads contain becomes assertions, the rest is listed as disputed.
2. Promote a case from candidate to adjudicated — manual until wired: run the adjudicate agent on the disputed page with its output directed to gold/<N>.md, edit the assertions with the page image open.
3. freeze hashes the gold; add the book id to textbook.benchmark.books in engine.json.

Only adjudicated cases count toward the headline (gold_status_for_headline). A candidate case still runs; it just cannot certify.

A new book

What a book directory needs before textbook <dir>:

The harness creates the rest: pages/, pages-md/ with _CONTRACT.md, pages-md-audit/ (the blind reads), pages-md-verify/ (verdicts, queue, score, run), tables-audit/ (right-edge crops), crops/ (native-density figures) and textbook.md with provenance front matter. The contract's "Observed in this book" section starts empty and is appended by the adjudicator; a second book in the same curriculum starts from the template, not from the first book's cases — repair drift is book-specific, and a clause that names a case the book does not have is noise.

What does generalise travels through presets (skills/textbook/scripts/presets.json): rules by subject kind — all, math-science, arabic-language, religious, latin-language, humanities — appended to every new contract under "Learned across books". A clause is promoted only when two different books observed it (each cites its book:page evidence: tatweel is typography, hamza-less spellings stay as printed, run-in labels are not headings, stacked math stays \frac, matrices read right-to-left…); kinds with no transcribed book yet carry seed clauses. textbook-contract.py --promote ~/catalog lists the observed cases no clause cites — the loop turns them into proposals every Sunday, and a proposal is adopted only after a bench run.

The language is the only thing the harness reads about the text: it decides direction, the illegible marker and the normalisation the scorers apply. Right-to-left languages are listed in textbook-contract.py.

The loop

scripts/textbook/loop.sh (launchd com.databayt.textbook-loop, daily 01:30) keeps going without being asked. One tick:

1. sense — pnpm twins <cur> --grade <g> --cdn writes .work/twins/<cur>-<g>.json
2. pick — a book already in progress, else OCR twins before text-layer ones, fewer pages first; a grade whose subjects all pass the twin gate hands over to the next (g12 → g11 → g10)
3. stage — the PDF, structure.json and page renders cloned into ~/catalog/.work/
4. run — claude -p runs the textbook workflow with maxPages: 120, sample: 24; a complete book is verified and adjudicated in the same call
5. land — scripts/textbook/land.sh copies, locks, validates and publishes only when the twin gate passes, reverting otherwise
6. tell — one comment on databayt/catalog "textbook loop: sd twins"

It never compares a new agreement with the live one: two random 12-page samples of the same chemistry twin scored 0.946 and 0.893 — a coin flip, not a regression. That is also why loop books sample 24. --status, --next, --weekly, --tick --force [--pages N] [--subject S], --pause / --resume (kill switch ~/catalog/.work/.textbook-off).

Configuration

Declared policy lives in engine.json under textbook; measured state lives in .claude/memory/textbook-scores.json and never in the engine file — the same split the dispatch bench uses.

The record

Sudan grade 12, 2026-10-10 — 8 of 25 subjects carry a vision twin that passes the twin gate, all live at their URLs (headline = random 12-page sample):

The other 17 are tesseract or text-layer twins with no agreement score — the loop's queue.

Benchmark, the blind pair (2026-10-10). Same 30 pages, same contract (d6a20313b1cc), both transcripts audited CLEAN: sonnet 96.4 % (481/499) vs opus 93.8 % (468/499). Sonnet won numerals (226/226 vs 209/226) and RTL tables (172/178 vs 157/178); opus won only figure labels (12/15 vs 9/15). The earlier 97.4 % and 96.2 % entries scored shipped twins — opus plus adjudication — and were never a blind baseline. Hence the split: sonnet transcribes, opus re-reads blind and adjudicates.

Honesty

• The headline is the random sample only. The risk-weighted tail and the post-adjudication scores are reported beside it and never folded in — the adjudicator saw both reads.
• A dead agent leaves a page as it was and flags it unverified; a run with missing pages is DEGRADED and is persisted as such — never as a clean grade.
• Blind means blind: a bench read that opens pages-md/, gold/ or the manifest is voided by the transcript audit and persisted invalid.
• Measured cost: ~9k tokens per transcribed page on opus (sonnet's tokens cost about a fifth); verify + adjudicate ~2–3.5M per book; ~400k per benchmark run.

Files

Runbook: .claude/skills/textbook/SKILL.md. The standard both converters are held to: .claude/skills/md/SKILL.md. The words: Keywords · Workflows.