step.10.diff.txt -- narrative retro for step.10 (U2.1: step-file parser)

Executed: 2026-07-20, same session as the skeleton (no separate
skeleton-then-execute split -- the parser, schema doc, tests, and this
step file were built together per the operator's explicit "step file
and its execution" direction). Companion artifact: step.10.gdiff.txt
(automated, scripts/step_gdiff). This is the FIRST P2 UoW and the first
genuinely new subsystem either repo's steps 7-9 didn't already resemble
-- it needed real schema-design work grounded in the actual b3u corpus,
not an incremental extension of app/pal/ or app/cli.py.

## Plan vs actual

```
+---+----------------------------------+------------------------------------------+
| # | Planned (§)                      | Actual                                   |
+---+----------------------------------+------------------------------------------+
| 1 | §1: corpus count "34" (the       | Verified live BEFORE any code was        |
|   |   objective's own wording, and   | written (not assumed from steps 7-9's    |
|   |   the operator brief's opening   | own step.7.txt/step.8.txt/step.9.txt,    |
|   |   line) needed live              | which mirror the convention but are 9    |
|   |   verification, not a copy       | files, not 50): `ls b3u/steps/step.*.txt |
|   |   forward                        | | grep -vE '\.(diff|cdiff|gdiff)\.txt$'`  |
|   |                                  | -> 50 files, step.1.txt..step.50.txt,    |
|   |                                  | contiguous. Corrected in end_to_end.md's |
|   |                                  | U2.1 bullet same-commit, with the        |
|   |                                  | verification command named inline --     |
|   |                                  | exactly the small, real, worth-recording |
|   |                                  | finding the operator's brief called it   |
|   |                                  | out as in advance.                       |
| 2 | §1: the brief's own "recent"     | Those files DO NOT EXIST -- b3u's step   |
|   |   corpus examples (step.140.txt,| corpus tops out at step.50.txt (b3u's    |
|   |   step.145.txt, step.146.txt)   | own P6 production-launch arc, steps      |
|   |                                  | 45-50, is the most recent slice). The    |
|   |                                  | "step.146" figure in the brief's own     |
|   |                                  | framing was cppcc's step numbering       |
|   |                                  | (a DIFFERENT repo, DIFFERENT corpus --   |
|   |                                  | cppcc's CLAUDE.md git-log preamble       |
|   |                                  | mentions "step.146" for CPPCC, not b3u), |
|   |                                  | not b3u's. Substituted step.45/48/49/50  |
|   |                                  | as the actual "recent" slice and         |
|   |                                  | verified the corpus range with `ls`      |
|   |                                  | before reading a single file, rather     |
|   |                                  | than trusting the guessed filenames.     |
| 3 | §3: assumed the trailing 5       | The relative ORDER (Verification gates / |
|   |   sections would be in a fixed  | LOCKs / Open questions / Acceptance /    |
|   |   relative order (matching      | Hash backfill) holds in 49/50 files but  |
|   |   design.md §2.3's prose list)  | b3u/steps/step.8.txt puts Hash backfill  |
|   |                                  | BEFORE Acceptance -- found by grepping   |
|   |                                  | `## (` markers across all 50 files, not  |
|   |                                  | by eyeballing 3-4 samples. Same file     |
|   |                                  | also titles its gate section "Oracle     |
|   |                                  | (gates G1-G3)" instead of "Verification  |
|   |                                  | gates". Decision: the parser enforces    |
|   |                                  | ONLY the marker shape and section (0) == |
|   |                                  | "Status" -- NOT a fixed closing-section  |
|   |                                  | title vocabulary or order -- because     |
|   |                                  | enforcing either would make a real,      |
|   |                                  | already-closed step in the GRADED        |
|   |                                  | fixture corpus fail to parse. See        |
|   |                                  | step.10.txt (1) and                      |
|   |                                  | docs/step_file_schema.md.                |
+---+----------------------------------+------------------------------------------+
```

Every other part of the plan held: the header-field set (Step/Parent/
UoW/Date/Origin, UoW optional per step.1.txt's own absence of it), the
`AdapterError`-family exception shape mirrored into
`StepFileError`/`UnknownSectionError`/`MalformedStepFileError`, and the
`$B3U_SIBLING_HOME` env-var-with-sane-default pattern all landed exactly
as scoped in the operator's brief.

## Gate evidence

```
+------+---------------------------------------------------------------------+
| G1   | pytest -q tests/test_ledger_parser.py -k b3u_corpus_file_parses --  |
|      | 50/50 PASS, one assertion per real b3u corpus file, live.           |
| G2   | pytest -q tests/test_ledger_parser.py -k b3ubot_own_step_file_parses|
|      | -- 10/10 PASS (bonus, not graded; self-inclusive of step.10.txt     |
|      | once written), live.                                                 |
| G3   | Six synthetic fixtures under tests/fixtures/ledger/, each asserted  |
|      | via pytest.raises(<SpecificType>, match=<substring>): unknown       |
|      | header field ("Sponsor:") -> UnknownSectionError; unknown section   |
|      | marker ("## Extra Notes") -> UnknownSectionError; missing header    |
|      | block entirely -> MalformedStepFileError (missing "Step:"); wrong   |
|      | section-0 title ("Overview" not "Status") -> MalformedStepFileError;|
|      | out-of-order section numbers (2 then 1) -> MalformedStepFileError   |
|      | ("strictly increase"); zero section markers anywhere ->             |
|      | MalformedStepFileError ("no numbered sections"). 6/6 PASS, plus a   |
|      | 7th empty-string case for good measure -- 7/7 PASS total on the     |
|      | refuse-loudly leg.                                                   |
| G4   | test_b3u_corpus_found_and_sized_correctly: asserts len == 50 AND    |
|      | the number sequence == range(1, 51) -- PASS live; this is the       |
|      | corpus-size tripwire named in step.10.txt G4.                       |
| G5   | Full suite before this step: 55 passed / 11 skipped (post-step.9    |
|      | baseline, confirmed by re-running before any edit). After: 125      |
|      | passed / 11 skipped. Delta: +70 passed, 0 skipped delta, 0          |
|      | failures -- purely additive, confirmed by running the suite twice.  |
| G6   | end_to_end.md §4 U2.1 bullet -> DONE (step.10) with the 34->50       |
|      | correction and a *Delivered:* line; §11 ledger row -> done/step.10/  |
|      | PENDING in the work commit, both confirmed present via `git diff    |
|      | --stat` before committing.                                          |
| G7   | b3ubot porcelain clean confirmed before committing; retro pair via  |
|      | the SHARED v3_claude/scripts/step_gdiff (steps/step.10.gdiff.txt).  |
+------+---------------------------------------------------------------------+
```

## Findings for later steps

1. **A corpus survey has to actually run `ls`, not trust a number
   carried forward in planning prose.** The operator's own brief said
   the "34" figure was "already confirmed this session" as stale and
   handed over the exact verification command -- this step ran it
   again anyway (rather than trusting the brief's own restated count),
   and it's a good thing it did: the brief's own guessed "recent"
   filenames (step.140/145/146) turned out not to exist in b3u/steps/
   at all (that numbering belongs to a DIFFERENT repo, cppcc). Every
   number and filename this step used in its final artifacts was
   independently `ls`-verified, not copied from any instruction text,
   including the operator's own. The general lesson for U2.2+: corpus
   facts age between when a plan is written and when a step executes
   them, even within the SAME planning document, even within the SAME
   task description -- verify live, every time, cheap as it is.
2. **A schema derived from 3-4 familiar examples will overfit; a
   schema derived from grep across the FULL corpus will find the real
   exceptions.** step.7.txt/step.8.txt/step.9.txt (this repo's own
   files) all follow the trailing-section order design.md §2.3's prose
   describes without exception -- reading only those three would have
   produced a parser that hard-rejects b3u/steps/step.8.txt (a real,
   closed, ACCEPTED step in the graded fixture corpus) for putting
   Hash backfill before Acceptance and for calling its gate section
   "Oracle" instead of "Verification gates". The fix wasn't picking
   the "right" template -- it was recognizing that no single template
   covers 50/50 files, and building the parser's hard requirements
   (marker shape, section 0) to match exactly what genuinely IS
   universal, while treating the rest as informational. Worth the same
   lens on U2.2's emitter: whatever round-trip byte-identity it
   targets has to hold against ALL 50 files' real shapes, not an
   idealized one.
3. **"Refuse loudly" needed a real answer to "refuse on WHAT,
   exactly," and the corpus itself supplied it.** The objective's
   wording ("unknown-section and malformed inputs refuse loudly") is
   compatible with several designs of varying strictness. The corpus
   survey turned that from a judgment call into an empirical one: the
   header-field label set (Step/Parent/UoW/Date/Origin) IS closed
   (verified 50/50 + 9/9, one exception explained), so an unrecognized
   label is a legitimate "unknown" refusal; body/closing-section TITLE
   text is NOT closed (step.8.txt's "Oracle" proves it), so refusing
   on unrecognized title text would be refusing on something the real
   corpus itself doesn't treat as an error. The exception boundary
   ended up exactly at the boundary the data supports, not the
   boundary that felt symmetrical or complete.

## Numbers

```
+---------------------------------------+----------------------+
| new files                              | 9 (app/ledger/       |
|                                         | __init__.py+parser.  |
|                                         | py+errors.py, docs/  |
|                                         | step_file_schema.md, |
|                                         | tests/               |
|                                         | test_ledger_parser.  |
|                                         | py + 6 fixtures      |
|                                         | under tests/         |
|                                         | fixtures/ledger/,    |
|                                         | steps/step.10.txt)   |
| parser + errors LOC                    | 232 + 45 = 277       |
| new tests                              | 70 (50 corpus + 10   |
|                                         | bonus corpus + 1     |
|                                         | corpus-size + 1      |
|                                         | bonus-count + 7      |
|                                         | refuse-loudly + 1    |
|                                         | known-good shape)    |
| b3u corpus files parsed clean          | 50/50                |
| b3ubot own step files parsed clean     | 10/10 (bonus, self-  |
|                                         | inclusive of         |
|                                         | step.10.txt)         |
| files edited                           | 1 (end_to_end.md;    |
|                                         | plus this step's own |
|                                         | step.10.txt backfill |
|                                         | at the gdiff commit) |
| corpus count corrected                 | 34 -> 50 (verified   |
|                                         | live, not assumed)   |
| suite before -> after                  | 55p/11s -> 125p/11s  |
| regressions                            | 0                    |
+---------------------------------------+----------------------+
```

U2.1 done -- opens P2 (the ledger engine, Track A). Unblocks U2.2 (the
emitter + byte round-trip oracle), which needs this step's `StepFile`
return shape to emit from. Ledger updated in the work commit per the
§12 execution contract; commit hash of that same commit backfilled into
both end_to_end.md §11 and this file's own steps/step.10.txt §7 in the
immediately following gdiff commit -- the same 2-part resolution
step.7/step.9 used (confirmed by inspecting 4f24b22's own diff via
`git show --stat 4f24b22` before repeating the pattern here rather than
assuming it).

End of retro.
