step.10.diff.txt -- narrative retro for step.10 (U2.1: step-file parser) Executed: 2026-07-20, same session as the skeleton (no separate skeleton-then-execute split -- the parser, schema doc, tests, and this step file were built together per the operator's explicit "step file and its execution" direction). Companion artifact: step.10.gdiff.txt (automated, scripts/step_gdiff). This is the FIRST P2 UoW and the first genuinely new subsystem either repo's steps 7-9 didn't already resemble -- it needed real schema-design work grounded in the actual b3u corpus, not an incremental extension of app/pal/ or app/cli.py. ## Plan vs actual ``` +---+----------------------------------+------------------------------------------+ | # | Planned (§) | Actual | +---+----------------------------------+------------------------------------------+ | 1 | §1: corpus count "34" (the | Verified live BEFORE any code was | | | objective's own wording, and | written (not assumed from steps 7-9's | | | the operator brief's opening | own step.7.txt/step.8.txt/step.9.txt, | | | line) needed live | which mirror the convention but are 9 | | | verification, not a copy | files, not 50): `ls b3u/steps/step.*.txt | | | forward | | grep -vE '\.(diff|cdiff|gdiff)\.txt$'` | | | | -> 50 files, step.1.txt..step.50.txt, | | | | contiguous. Corrected in end_to_end.md's | | | | U2.1 bullet same-commit, with the | | | | verification command named inline -- | | | | exactly the small, real, worth-recording | | | | finding the operator's brief called it | | | | out as in advance. | | 2 | §1: the brief's own "recent" | Those files DO NOT EXIST -- b3u's step | | | corpus examples (step.140.txt,| corpus tops out at step.50.txt (b3u's | | | step.145.txt, step.146.txt) | own P6 production-launch arc, steps | | | | 45-50, is the most recent slice). The | | | | "step.146" figure in the brief's own | | | | framing was cppcc's step numbering | | | | (a DIFFERENT repo, DIFFERENT corpus -- | | | | cppcc's CLAUDE.md git-log preamble | | | | mentions "step.146" for CPPCC, not b3u), | | | | not b3u's. Substituted step.45/48/49/50 | | | | as the actual "recent" slice and | | | | verified the corpus range with `ls` | | | | before reading a single file, rather | | | | than trusting the guessed filenames. | | 3 | §3: assumed the trailing 5 | The relative ORDER (Verification gates / | | | sections would be in a fixed | LOCKs / Open questions / Acceptance / | | | relative order (matching | Hash backfill) holds in 49/50 files but | | | design.md §2.3's prose list) | b3u/steps/step.8.txt puts Hash backfill | | | | BEFORE Acceptance -- found by grepping | | | | `## (` markers across all 50 files, not | | | | by eyeballing 3-4 samples. Same file | | | | also titles its gate section "Oracle | | | | (gates G1-G3)" instead of "Verification | | | | gates". Decision: the parser enforces | | | | ONLY the marker shape and section (0) == | | | | "Status" -- NOT a fixed closing-section | | | | title vocabulary or order -- because | | | | enforcing either would make a real, | | | | already-closed step in the GRADED | | | | fixture corpus fail to parse. See | | | | step.10.txt (1) and | | | | docs/step_file_schema.md. | +---+----------------------------------+------------------------------------------+ ``` Every other part of the plan held: the header-field set (Step/Parent/ UoW/Date/Origin, UoW optional per step.1.txt's own absence of it), the `AdapterError`-family exception shape mirrored into `StepFileError`/`UnknownSectionError`/`MalformedStepFileError`, and the `$B3U_SIBLING_HOME` env-var-with-sane-default pattern all landed exactly as scoped in the operator's brief. ## Gate evidence ``` +------+---------------------------------------------------------------------+ | G1 | pytest -q tests/test_ledger_parser.py -k b3u_corpus_file_parses -- | | | 50/50 PASS, one assertion per real b3u corpus file, live. | | G2 | pytest -q tests/test_ledger_parser.py -k b3ubot_own_step_file_parses| | | -- 10/10 PASS (bonus, not graded; self-inclusive of step.10.txt | | | once written), live. | | G3 | Six synthetic fixtures under tests/fixtures/ledger/, each asserted | | | via pytest.raises(, match=): unknown | | | header field ("Sponsor:") -> UnknownSectionError; unknown section | | | marker ("## Extra Notes") -> UnknownSectionError; missing header | | | block entirely -> MalformedStepFileError (missing "Step:"); wrong | | | section-0 title ("Overview" not "Status") -> MalformedStepFileError;| | | out-of-order section numbers (2 then 1) -> MalformedStepFileError | | | ("strictly increase"); zero section markers anywhere -> | | | MalformedStepFileError ("no numbered sections"). 6/6 PASS, plus a | | | 7th empty-string case for good measure -- 7/7 PASS total on the | | | refuse-loudly leg. | | G4 | test_b3u_corpus_found_and_sized_correctly: asserts len == 50 AND | | | the number sequence == range(1, 51) -- PASS live; this is the | | | corpus-size tripwire named in step.10.txt G4. | | G5 | Full suite before this step: 55 passed / 11 skipped (post-step.9 | | | baseline, confirmed by re-running before any edit). After: 125 | | | passed / 11 skipped. Delta: +70 passed, 0 skipped delta, 0 | | | failures -- purely additive, confirmed by running the suite twice. | | G6 | end_to_end.md §4 U2.1 bullet -> DONE (step.10) with the 34->50 | | | correction and a *Delivered:* line; §11 ledger row -> done/step.10/ | | | PENDING in the work commit, both confirmed present via `git diff | | | --stat` before committing. | | G7 | b3ubot porcelain clean confirmed before committing; retro pair via | | | the SHARED v3_claude/scripts/step_gdiff (steps/step.10.gdiff.txt). | +------+---------------------------------------------------------------------+ ``` ## Findings for later steps 1. **A corpus survey has to actually run `ls`, not trust a number carried forward in planning prose.** The operator's own brief said the "34" figure was "already confirmed this session" as stale and handed over the exact verification command -- this step ran it again anyway (rather than trusting the brief's own restated count), and it's a good thing it did: the brief's own guessed "recent" filenames (step.140/145/146) turned out not to exist in b3u/steps/ at all (that numbering belongs to a DIFFERENT repo, cppcc). Every number and filename this step used in its final artifacts was independently `ls`-verified, not copied from any instruction text, including the operator's own. The general lesson for U2.2+: corpus facts age between when a plan is written and when a step executes them, even within the SAME planning document, even within the SAME task description -- verify live, every time, cheap as it is. 2. **A schema derived from 3-4 familiar examples will overfit; a schema derived from grep across the FULL corpus will find the real exceptions.** step.7.txt/step.8.txt/step.9.txt (this repo's own files) all follow the trailing-section order design.md §2.3's prose describes without exception -- reading only those three would have produced a parser that hard-rejects b3u/steps/step.8.txt (a real, closed, ACCEPTED step in the graded fixture corpus) for putting Hash backfill before Acceptance and for calling its gate section "Oracle" instead of "Verification gates". The fix wasn't picking the "right" template -- it was recognizing that no single template covers 50/50 files, and building the parser's hard requirements (marker shape, section 0) to match exactly what genuinely IS universal, while treating the rest as informational. Worth the same lens on U2.2's emitter: whatever round-trip byte-identity it targets has to hold against ALL 50 files' real shapes, not an idealized one. 3. **"Refuse loudly" needed a real answer to "refuse on WHAT, exactly," and the corpus itself supplied it.** The objective's wording ("unknown-section and malformed inputs refuse loudly") is compatible with several designs of varying strictness. The corpus survey turned that from a judgment call into an empirical one: the header-field label set (Step/Parent/UoW/Date/Origin) IS closed (verified 50/50 + 9/9, one exception explained), so an unrecognized label is a legitimate "unknown" refusal; body/closing-section TITLE text is NOT closed (step.8.txt's "Oracle" proves it), so refusing on unrecognized title text would be refusing on something the real corpus itself doesn't treat as an error. The exception boundary ended up exactly at the boundary the data supports, not the boundary that felt symmetrical or complete. ## Numbers ``` +---------------------------------------+----------------------+ | new files | 9 (app/ledger/ | | | __init__.py+parser. | | | py+errors.py, docs/ | | | step_file_schema.md, | | | tests/ | | | test_ledger_parser. | | | py + 6 fixtures | | | under tests/ | | | fixtures/ledger/, | | | steps/step.10.txt) | | parser + errors LOC | 232 + 45 = 277 | | new tests | 70 (50 corpus + 10 | | | bonus corpus + 1 | | | corpus-size + 1 | | | bonus-count + 7 | | | refuse-loudly + 1 | | | known-good shape) | | b3u corpus files parsed clean | 50/50 | | b3ubot own step files parsed clean | 10/10 (bonus, self- | | | inclusive of | | | step.10.txt) | | files edited | 1 (end_to_end.md; | | | plus this step's own | | | step.10.txt backfill | | | at the gdiff commit) | | corpus count corrected | 34 -> 50 (verified | | | live, not assumed) | | suite before -> after | 55p/11s -> 125p/11s | | regressions | 0 | +---------------------------------------+----------------------+ ``` U2.1 done -- opens P2 (the ledger engine, Track A). Unblocks U2.2 (the emitter + byte round-trip oracle), which needs this step's `StepFile` return shape to emit from. Ledger updated in the work commit per the §12 execution contract; commit hash of that same commit backfilled into both end_to_end.md §11 and this file's own steps/step.10.txt §7 in the immediately following gdiff commit -- the same 2-part resolution step.7/step.9 used (confirmed by inspecting 4f24b22's own diff via `git show --stat 4f24b22` before repeating the pattern here rather than assuming it). End of retro.