step.13.txt -- b3ubot U2.4: Retro-pair machinery + ledger CLI -- closes P2 (the ledger engine, Track A), clearing P3's stated entry condition alongside the already-done DAG substrate (U3.9/U3.9a). Builds the templating machinery on top of U2.1-U2.3's parser/emitter/table APIs, deliberately stopping short of fabricating narrative content. Step: 13 Parent: end_to_end.md §4 U2.4 bullet verbatim ("Retro-pair machinery + ledger CLI -- narrative-retro templating, cdiff generation (the claude_diff pattern), b3ubot ledger surface. Done when: a synthetic UoW walks draft -> close leaving the full artifact set."), corrected in this same step (the cdiff/claude_diff wording is stale -- see (1)). app/ledger/parser.py + emitter.py (U2.1/U2.2, StepFile/emit_step_file) and table_parser.py/table_emitter.py/table_transitions.py (U2.3) for the APIs this step exposes. UoW: U2.4 Date: 2026-07-20 Origin: Operator directive 2026-07-20: draft AND execute steps/step.13.txt for U2.4 in a single pass, closing P2 and clearing P3's stated entry condition alongside the already-done DAG substrate. ## (0) Status EXECUTED in this pass. Not a skeleton-then-later-execute split -- the retro machinery (app/ledger/retro.py), the `ledger` CLI subcommand (app/cli.py), the three new test files, the docs, the end_to_end.md/ design.md/README.md corrections, and this step file were built together, per the operator's explicit "draft AND execute (single pass)" direction, matching step.7/9/10/11/12's rhythm. This file itself was DOGFOODED: `steps/step.13.txt`'s title/header/section skeleton was generated via `.venv/bin/python -m app.cli ledger draft-step` (the tool this very step built), then hand-filled section by section -- an interesting, worth-reporting fact this step's own brief asked for explicitly. `steps/step.13.diff.txt` was ALSO drafted via `ledger draft-retro` and hand-filled, for the same reason. This is different from steps 7-12, which were all written by hand from scratch (there was no templating tool yet). ## (1) Ground truth probed at drafting (outputs -> named writers) READ end_to_end.md §4 U2.4 bullet. Found it stale on arrival, same "verify, don't copy forward" discipline steps 10/12 already established for other numbers/section-refs: "cdiff generation (the claude_diff pattern)" names a mechanism RETIRED repo-wide (scripts/claude_diff -> .cdiff.txt, replaced by scripts/step_gdiff -> .gdiff.txt, for an off-by-one range + self-referential growth bug -- project history, not something this step needed to re-derive). Confirmed via `ls steps/*.gdiff.txt` (12 files, step.1-step.12) and `ls steps/*.cdiff.txt` (0 files) that this repo's own steps 1-12 have used step_gdiff/.gdiff.txt exclusively, never claude_diff/.cdiff.txt. Corrected the U2.4 bullet in this same step (see (2)). GREPPED design.md and README.md for the SAME stale pattern, not just the one instance the brief named (steps 10/12's own "a named example is a floor, not the whole check" lesson, applied here to a different kind of staleness): found THREE more -- design.md line 47 ("`.cdiff.txt` automated" -- describing the CURRENT retro-pair mechanism, so genuinely wrong, not a historical mention like the ones left alone in end_to_end.md:195/docs/step_file_schema.md:27, which name `.cdiff.txt` only as an EXCLUDED artifact-family glob, accurate as written); design.md line 48 and line 137 ("the end_to_end §9 table"/"§9 UoW table" -- stale, actually §11); README.md lines 98 and 170 (same "§9 ledger" staleness). All five corrected in this same step, alongside the U2.4 bullet. A SIXTH, unrelated but adjacent slip found in the same pass: end_to_end.md's own U2.3 *Delivered:* line said "205p/11s -> 245p/11s" -- arithmetically wrong (205+42=247, and step.12.diff.txt's own Numbers table already said 247, and a live `pytest -q` run before any edit this step confirmed 247 passed/11 skipped) -- a hand-authoring typo, corrected in the same line. steps/step.12.txt/step.12.diff.txt themselves were NOT touched (historical retro artifacts, step.12's own documented precedent for not falsifying the record of what was believed true when written). READ steps/step.7.diff.txt through steps/step.12.diff.txt IN FULL (all six, not a 3-4 sample) to find the STRUCTURAL template empirically, the same evidence-based method U2.1 used for the step-file schema. Found genuinely invariant across all six: title line ("step.N.diff.txt -- narrative retro for step.N (UoW: ...)"), the INTERNAL banner, an "Executed: DATE, ..." line naming the companion .gdiff.txt artifact, four section headings in a FIXED order (## Plan vs actual / ## Gate evidence / ## Findings for later steps / ## Numbers), and a closing "End of retro." line. NOT invariant, and therefore NOT templated: the actual plan-vs-actual/gate-evidence/ findings/numbers CONTENT -- genuine narrative every single time, requiring the step's author to have done the work and looked at real results. This is the point 6 scoping decision (2) below builds on. READ app/ledger/parser.py, emitter.py (U2.1/U2.2) in full -- StepFile/Section, parse_step_file/parse_step_text, emit_step_file's two-tier (raw-preferred, structural- fallback) design. Confirmed a hand-built StepFile (title_raw/ header_raw/Section.raw all None) round-trips STRUCTURALLY via emit_step_file -- exactly the mechanism draft_step_skeleton() below reuses rather than reimplementing header/section formatting a second time. READ app/ledger/table_parser.py, table_emitter.py, table_transitions.py (U2.3) in full -- LedgerRow/LedgerTable, parse_ledger_table_from_doc/emit_ledger_table_text/ replace_ledger_table_in_doc, transition()/record_commit()/ add_row()/validate_row_invariants(). Confirmed the exact legal-transition rule set (todo->active/dropped, active-> done/dropped, done/dropped terminal) this step's CLI wraps but does not reimplement. READ app/cli.py's existing `ask` subcommand in full -- the argparse subparsers shape, the "propose_ask/apply_ask stay pure, main() does presentation-layer error handling only" posture. The new `ledger` subcommand follows the SAME file/ pattern (not a separate binary), same posture (ledger_* functions stay pure and raise; main()'s dispatch catches and prints). READ ~/projects/scripts/step_gdiff in full. Confirmed: it needs only a stepN.txt path inside SOME git repo (`git rev-parse --show-toplevel` -- no dependency on being run inside the real b3ubot repo specifically); START = parent of the skeleton commit (earliest commit adding the file), END = newest commit touching the file. This is why run_step_gdiff() (2) below can be tested against a REAL, throwaway tmp-dir git repo instead of needing a mock -- and why it shells out rather than re-deriving any of this range logic itself. DECIDED (point 6 of the operator's brief, deliberately, not by default) the mechanical/narrative boundary for the retro templating: MECHANICAL = step number, UoW id/title, date, companion-artifact filename, the section-heading/marker skeleton itself, and (for step.N.txt specifically) the canonical (0)-(7) section numbering+titles and the exact two-line Hash-backfill shape every corpus retro uses verbatim. NARRATIVE, left as a single greppable placeholder (`app.ledger.retro.FILL_IN`) = literally everything else, INCLUDING file-changed lists and test-count deltas -- both LOOK mechanical but aren't available at draft time (before the work exists) without either re-deriving git-diff logic step_gdiff already owns, or actually running the suite twice (a thing a template generator has no business doing on its own initiative). See app/ledger/retro.py's own module docstring and docs/ledger_cli_and_retro.md for the full reasoning. CONFIRMED `git log --oneline -30` for exact commit-message phrasing, and re-read steps/step.11.txt (2)/(7) directly for the EXACT 2-commit hash-backfill shape (work commit leaves PENDING; the immediately-following gdiff commit backfills both end_to_end.md's ledger row AND this file's own (7)) -- the reference step.11 itself names, correcting step.12's own 3-commit drift (work / gdiff / a SEPARATE hash-backfill commit) back to this repo's established precedent, per the operator's explicit point 7 instruction. ## (2) What U2.4 delivers 2.1 **app/ledger/retro.py** (new): `draft_step_skeleton()` (builds a StepFile, renders via app/ledger/emitter.py's emit_step_file -- U2.2, no reimplemented formatting), `generate_diff_skeleton()` (pure string templating, no StepFile involved), `run_step_gdiff()` (shells to the SHARED v3_claude/scripts/step_gdiff via `default_scripts_home()`, overridable via `$V3_CLAUDE_SCRIPTS_HOME` -- mirrors $B3UBOT_HOME/ $B3U_SIBLING_HOME's convention), `RetroError`. `FILL_IN` is the one, greppable, narrative-placeholder constant every generated slot uses. 2.2 **app/cli.py** gains the `ledger` subcommand (a SEPARATE top-level command from `ask`, same file/pattern): `show` / `transition` / `add-row` / `record-commit` / `close` / `draft-step` / `draft-retro` / `gdiff`. `ledger_close()` picks `transition(..., "done", commit=...)` for an active row or `record_commit()` for an already-done row (the step.7/9/10/11 PENDING-then-real-hash shape), mirroring table_transitions.py's own documented rationale for keeping the two operations separate. Every `--doc` defaults to this repo's real end_to_end.md but is an explicit, overridable argument everywhere -- no ledger_* function or CLI path hardcodes the real file. 2.3 **tests/test_ledger_retro.py** (new, 8 tests): draft_step_skeleton round-trips through the REAL U2.1 parser; mechanical facts present verbatim, narrative facts FILL_IN; generate_diff_skeleton has the invariant structural shape; run_step_gdiff against a REAL synthetic tmp-dir git repo (not mocked -- confirmed feasible per (1)'s step_gdiff read), plus its missing-script and never-committed-step-file refusal paths. 2.4 **tests/test_ledger_cli.py** (new, 19 tests): every `ledger_*` library function AND the `main()` argparse wiring, against an isolated synthetic ledger doc under tmp_path -- never the real end_to_end.md. 2.5 **tests/test_synthetic_uow_arc.py** (new, 1 test): the done-when, literally -- see (3) G8. 2.6 **docs/ledger_cli_and_retro.md** (new): the CLI surface, the mechanical/narrative scoping decision and its reasoning, and the synthetic-UoW done-when's shape. 2.7 **end_to_end.md** same-commit edits: the cdiff/claude_diff correction + the 245->247 typo fix in U2.3's own Delivered line (both (1)); §4 U2.4 bullet -> DONE (step.13) with a *Delivered:* line; §11 ledger row -> done/step.13/PENDING (via `ledger transition`/`ledger close`, dogfooding this step's OWN new CLI -- see (3) G9). 2.8 **design.md** same-commit edits: `.cdiff.txt` -> `.gdiff.txt` in §1.3's retro-pair description, "§9" -> "§11" in §1.3 and §2.3 (both found in (1)'s grep sweep, not the brief's own named instance). 2.9 **README.md** same-commit edits: "§9" -> "§11" (x2, same grep sweep), a new "Getting started" paragraph naming the `ledger` subcommand alongside the existing `ask` examples. EXPLICITLY OUT OF SCOPE (named here per the operator's brief, not silently built): - P3's orchestrator actually driving REAL UoWs through skeleton -> ratify -> execute -> verify -> retro. This step only proves the MACHINERY on a SYNTHETIC UoW; the orchestrator doesn't exist until U3.1+. -> P3 - Auto-generating plan-vs-actual/gate-evidence/findings/numbers NARRATIVE content. Named explicitly as a deliberate non-goal in (1)/ (2.1), not a silent gap -- fabricating reflective prose about work that hasn't happened would be dishonest content, not a shortcut. ## (3) Verification gates G1 All 8 tests/test_ledger_retro.py cases PASS live, including the REAL scripts/step_gdiff invocation against a synthetic tmp-dir git repo (asserts the real widget.py content appears in the real step.1.gdiff.txt it produces) -- verified live: 8/8 PASS. G2 All 19 tests/test_ledger_cli.py cases PASS live, both the direct library-function calls and the main() argparse-wired end-to-end invocations, against an isolated synthetic doc -- verified live: 19/19 PASS. G3 tests/test_synthetic_uow_arc.py's single test walks the FULL draft->ratify(trivial)->execute(trivial)->verify(trivial)->close arc and asserts the complete artifact set exists: step.N.txt- shaped file (0)-(7) present, step.N.diff.txt-shaped file with FILL_IN placeholders present, a REAL step.N.gdiff.txt (produced by the actual scripts/step_gdiff, containing the real diff content), and the ledger row at done/step.N/PENDING -- verified live: 1/1 PASS. Also asserts the synthetic UoW's id/title never appear in the REAL end_to_end.md (isolation, not inferred). G4 draft_step_skeleton()'s output round-trips through the REAL app/ledger/parser.py's parse_step_text (not a hand-rolled checker) -- verified live (test_draft_step_skeleton_round_trips_ through_the_real_parser), and again live via the CLI dogfooding of steps/step.13.txt itself (this file parses clean under the same parser -- confirmed by running `.venv/bin/python -c "from app.ledger.parser import parse_step_file; parse_step_file('steps/step.13.txt')"` after every edit to this file). G5 Every stale cdiff/claude_diff/§9 reference found in (1) is fixed in end_to_end.md, design.md, and README.md -- verified live via `grep -rn '§9\|claude_diff' design.md end_to_end.md README.md`: every remaining hit is CORRECTION NARRATIVE (naming the old, now-fixed value inline, e.g. "corrected ... from '§9'") not a live stale pointer -- confirmed by reading each hit, not just counting them. The two accurate `.cdiff.txt`-as-excluded-glob mentions in end_to_end.md:195/docs/step_file_schema.md:27 are deliberately left alone, named explicitly in (1) as NOT stale. G6 grep -rn "FILL_IN\|generate_diff_skeleton\|draft_step_skeleton" app/ledger/retro.py app/cli.py -- confirms the mechanical/ narrative split is implemented exactly as named in (1)/(2): no code path in either module synthesizes plan-vs-actual/gate- evidence/findings/numbers prose. G7 Full suite before this step's edits: 247 passed/11 skipped (confirmed live, matching step.12's own corrected baseline). After: 277 passed/11 skipped -- +30 (28 genuinely new + 2 from the step-file parser's/emitter's own self-inclusive corpus parametrization picking up step.13.txt, the same growth pattern steps 10-12 documented for their own predecessors). A REAL, live-run finding along the way, not silently worked around: G9's dogfooding (U2.4 todo->active->done through the real ledger) broke a PRE-EXISTING test (test_ledger_table_roundtrip.py's test_structural_round_trip_after_mutating_a_row, from step.12) that had picked U2.4 as its "a real todo row" example -- confirmed live (the suite went red immediately after the dogfooding commands ran), fixed by pointing that test at U3.1 (the next real todo row, P3, not yet started) instead of loosening or deleting the test. See steps/step.13.diff.txt for the full account. G8 end_to_end.md's own U2.4 done-when, LITERALLY: "a synthetic UoW walks draft -> close leaving the full artifact set" -- G3 IS this gate, not an approximation of it. G9 §11 ledger row for U2.4 updated THROUGH this step's OWN new `ledger` CLI (not the raw table_transitions API directly) -- `.venv/bin/python -m app.cli ledger transition U2.4 active --step step.13` then `... ledger close U2.4 PENDING`, against the REAL end_to_end.md -- confirmed via `git diff --stat` before committing. G10 b3ubot porcelain clean after each commit; retro pair via the SHARED v3_claude/scripts/step_gdiff (steps/step.13.gdiff.txt); exactly 2 commits (work + gdiff-with-hash-backfill-folded-in), not 3 -- step.12's own drift, corrected per the operator's point 7 instruction and step.11's own precedent. ## (4) LOCKs C-13-A b3ubot porcelain clean after each commit. C-13-B NO FABRICATED NARRATIVE: neither draft_step_skeleton() nor generate_diff_skeleton() ever writes plan-vs-actual/gate- evidence/findings/numbers CONTENT -- every such slot is the single greppable `FILL_IN` constant (G6). This is the central design constraint of this whole step, not an afterthought. C-13-C RETRO MACHINERY SHELLS OUT, NEVER REIMPLEMENTS: run_step_gdiff() calls the SHARED scripts/step_gdiff; no git-diffing/commit-range logic is duplicated in app/ledger/retro.py (verified: grep -n "git diff\|git log" app/ledger/retro.py -> empty, the only `git`-adjacent code is the subprocess.run call to the external script itself). C-13-D ISOLATION: every ledger_* CLI function takes `doc`/`out` as an explicit argument; tests/test_ledger_cli.py and tests/test_synthetic_uow_arc.py never write to this repo's own steps/ or end_to_end.md (G3's own isolation assertion). C-13-E NO SILENT SCOPE CREEP: P3's orchestrator (real UoWs, real skeleton->ratify->execute->verify->retro) is named out of scope in (2), not silently begun. This step's `close` CLI command and the synthetic arc test do not simulate an AI-driven ratify/execute/verify -- those steps are literally trivial placeholders in the test, named as such. C-13-F standing locks inherited: B-3 (no disclosure/no push); B-6 (deterministic pytest oracles gate this step; no AI review tier). B-1/B-2/B-4/B-5 are moot -- this step invokes no cppcc, no provider, no egress. ## (5) Open questions Q-13-A Should `run_step_gdiff()`/the `ledger gdiff` CLI command AUTOMATICALLY run after `ledger close`, folding the retro- pair generation into one CLI invocation? LEAN (built this way): no -- step_gdiff's own END marker is "the newest commit that touches the step file," which in this repo's real 2-commit shape is the GDIFF commit itself (it backfills §7); running gdiff automatically inside `close` (which runs BEFORE that commit exists) would compute the wrong range. Keeping them as separate CLI invocations lets the caller sequence them correctly around the real commit boundary, exactly as steps 7-12 already do it by hand. Revisit only if a future step finds a commit shape where this ordering constraint no longer applies. Q-13-B Should draft_step_skeleton()'s canonical (0)-(7) section skeleton be configurable (a different section count/order) for a UoW that doesn't fit that shape? LEAN (built this way): not yet -- every one of this repo's own steps 7-12 uses exactly this (0)-(7) shape (verified live while surveying in (1)); U2.1's own parser doesn't require it (only section (0)=="Status" and strictly-increasing numbers are enforced), so a future step with a genuinely different shape can build its own StepFile and call emit_step_file directly instead of this convenience wrapper. Revisit if a real step needs that. ## (6) Acceptance - [x] app/ledger/retro.py: draft_step_skeleton, generate_diff_ skeleton, run_step_gdiff, RetroError, FILL_IN (2.1, G1, G4, G6). - [x] app/cli.py: `ledger` subcommand, 8 sub-subcommands, all thin wrappers (2.2, G2, G9). - [x] tests/test_ledger_retro.py -- 8/8 PASS (2.3, G1). - [x] tests/test_ledger_cli.py -- 19/19 PASS (2.4, G2). - [x] tests/test_synthetic_uow_arc.py -- 1/1 PASS, the literal done-when (2.5, G3, G8). - [x] docs/ledger_cli_and_retro.md (2.6). - [x] end_to_end.md: cdiff/claude_diff correction, 245->247 typo fix, §4 U2.4 bullet -> DONE with *Delivered:*, §11 ledger row -> done/step.13/PENDING via the new CLI (2.7, G5, G9). - [x] design.md: .cdiff.txt->.gdiff.txt, §9->§11 x2 (2.8, G5). - [x] README.md: §9->§11 x2, new `ledger` Getting-started paragraph (2.9, G5). - [x] Zero regressions: 247p/11s -> 277p/11s, one pre-existing test fixed (not loosened) after a real live-data collision with this step's own dogfooding (G7). - [x] Retro pair: this file + step.13.diff.txt + step.13.gdiff.txt via the SHARED scripts/step_gdiff (G10). - [x] Exactly 2 commits (work + gdiff-with-backfill), not 3 (G10). ## (7) Hash backfill WORK commit (retro machinery + ledger CLI + tests + docs + this file): f6fd996 GDIFF commit (step.13.gdiff.txt via scripts/step_gdiff): (HEAD) End of step.