step.11.diff.txt -- narrative retro for step.11 (U2.2: emitter + THE round-trip oracle) Executed: 2026-07-20, same session as the skeleton (no separate skeleton-then-execute split -- app/config/, the parser extension, the emitter, the tests, the docs updates, and this step file were built together, per the operator's explicit "step file and its execution" direction, matching step.7/step.9/step.10's rhythm). Companion artifact: step.11.gdiff.txt (automated, scripts/step_gdiff). Distinguishing feature of this step versus its predecessors: a mid-task operator design-change message arrived AFTER the investigation had already found the banner was a real capture problem, and BEFORE any emitter code was written -- absorbed as a redesign, not a patch. ## Plan vs actual ``` +---+----------------------------------+------------------------------------------+ | # | Planned (brief) | Actual | +---+----------------------------------+------------------------------------------+ | 1 | Investigate whether the SPECIFIC | Confirmed real, and WORSE than the | | | risk (StepFile doesn't retain | brief's own framing in one respect: the | | | enough for byte-identity) is | brief's risk section focused on the | | | real, resolve via (a) fixed | banner and section-boundary blank lines. | | | template / (b) StepFile | A fresh 60-file resurvey found a THIRD, | | | extension / (c) re-scope to | unnamed gap: 52/60 files wrap their | | | structural-only | title onto indented continuation lines | | | | that title_line (first line only) drops | | | | entirely -- never mentioned in the brief,| | | | found by grepping line 2 of every file. | | | | Resolved via (b), StepFile extension, | | | | exactly as the brief's own "let the | | | | evidence decide, in order of preference" | | | | framing anticipated for THIS class of | | | | gap once evidence showed corpus | | | | uniformity elsewhere was strong enough | | | | to make (a)'s fixed-template shortcuts | | | | safe for everything EXCEPT the 2/534 | | | | section-boundary anomalies and the | | | | 52/60 multi-line titles -- both of which | | | | got real, additive fields instead of a | | | | template guess. | | 2 | Investigate the banner-capture | SUPERSEDED mid-investigation by an | | | risk: does the parsed model | operator design-change message: the | | | need to store the banner text | banner isn't captured OR reproduced from | | | losslessly to reproduce it? | source text at all -- it's a fresh, | | | | per-deployment runtime value | | | | (app/config/, ${HOME}/BANNER.md). This | | | | actually SIMPLIFIED the design versus | | | | the original ask (no banner field, no | | | | banner-preservation logic in the parser | | | | at all) while still achieving byte- | | | | identity for the CURRENT corpus by | | | | pointing a test fixture's $HOME at a | | | | BANNER.md matching the corpus's own | | | | (verified uniform) banner text. Absorbed | | | | before any emitter code was finalized, | | | | not bolted on after -- the blank-line/ | | | | trailer investigation below was already | | | | under way and unaffected by the message. | | 3 | (not in the original brief at | Building the emitter's reconstruction | | | all -- found DURING | formula and hand-testing it against a | | | implementation) | synthetic StepFile (before writing the | | | | pytest suite around it) surfaced that | | | | U2.1's `Section.body` for the LAST | | | | section of a file silently embedded the | | | | document trailer ("End of step.") as | | | | trailing content -- `body_end` ran to | | | | end-of-file, an accidental side effect | | | | never documented or exercised by U2.1's | | | | own tests. Structural round-trip on a | | | | hand-built StepFile failed on exactly | | | | this (the trailer showed up TWICE: | | | | embedded in body AND appended via | | | | StepFile.trailer). Fixed by excluding | | | | the trailer from `_parse_sections`'s | | | | last-section content_end for BOTH `body` | | | | and the new `raw` -- verified safe | | | | against U2.1's one existing body-content | | | | assertion (the "2cf5c0f" substring | | | | check) before landing it. | +---+----------------------------------+------------------------------------------+ ``` Every other part of the plan held: the two-tier emitter design (verbatim `raw` fields preferred, `title_line`/`header_fields`/`body` fallback for hand-built StepFiles) worked on the first implementation once the three findings above were folded in; the $B3U_SIBLING_HOME test convention and the two-commit retro shape carried over from U2.1 unchanged. ## Gate evidence ``` +------+---------------------------------------------------------------------+ | G1 | pytest -q tests/test_ledger_roundtrip.py -k | | | test_b3u_corpus_file_round_trips_byte_identical -- 50/50 PASS, | | | live, against a fixture BANNER.md matching the corpus's own banner | | | text exactly. | | G2 | pytest -q ... -k test_b3ubot_own_step_file_round_trips_byte_ | | | identical -- 11/11 PASS, live, self-inclusive of step.11.txt once | | | written (same growth pattern step.10 documented for itself). | | G3 | 3 dedicated tests: banner-configured emits the block | | | (test_emit_includes_configured_banner_block), no-BANNER.md emits | | | none at all (test_emit_omits_banner_block_when_none_configured -- | | | asserts the internal-banner strings are absent AND the structural | | | shape: title, one blank line, straight into "Step:"), and a THIRD | | | deployment's own (non-corpus) banner text round-trips correctly | | | (test_emit_with_a_different_deployments_banner) -- the actual point | | | of the mechanism, not just the corpus's own fixture. 3/3 PASS live. | | G4 | 4 hand-built "generated skeleton" StepFiles (minimal single- | | | section; full 7-section with UoW; UoW ABSENT, mirroring b3u's | | | founding step.1.txt; a body with internal blank-line paragraph | | | breaks) each round-trip structurally via | | | parse(emit(skeleton)) -- header_fields/trailer/section numbers+ | | | titles+bodies all compared, live. 4/4 PASS. | | G5 | tests/test_config_loader.py: key/value parsing + ${VAR} expansion, | | | comment/blank-line tolerance, non-KV-line tolerance, missing-file | | | tolerance, the shipped config.txt's own banner key, banner_path() | | | default expansion, banner_text() present/absent/empty-file. 9/9 | | | PASS live. | | G6 | tests/test_ledger_parser.py (U2.1's own file, test BODIES unedited) | | | -- grew from 70 to 71 collected items purely because its own | | | self-inclusive corpus parametrization now also picks up | | | step.11.txt at collect time (not a new test written by this step). | | | 71/71 PASS live. Full suite before this step: 125 passed/11 skipped | | | (post-step.10 baseline, re-confirmed live before any edit). After: | | | 205 passed/11 skipped. Delta: +80 passed (79 in the two new test | | | files + 1 from the self-inclusive growth), 0 skipped delta, 0 | | | failures. | | G7 | end_to_end.md §4 U2.2 bullet -> DONE (step.11) with a *Delivered:* | | | line naming the byte-identical/structural two-guarantee split and | | | the 61/61 + 4/4 pass counts; §11 ledger row -> done/step.11/PENDING | | | in the work commit, confirmed present via `git diff --stat` before | | | committing. | | G8 | b3ubot porcelain clean confirmed before committing; retro pair via | | | the SHARED v3_claude/scripts/step_gdiff (steps/step.11.gdiff.txt). | +------+---------------------------------------------------------------------+ ``` ## Findings for later steps 1. **A risk investigation should re-derive its OWN scope from live evidence, not just answer the questions the brief asked.** The operator's brief named the banner and section-boundary blank lines as the risk to check. Running the actual survey (not just checking those two named things) found a THIRD gap -- multi-line titles, 52/60 files -- that nobody had named yet. The lesson isn't "the brief was wrong" (it wasn't, both named risks were real and confirmed); it's that "investigate whether the model retains enough" is itself the actual task, and a named risk list is a floor, not a ceiling, on what that investigation needs to check. 2. **A mid-task design-change message should be absorbed as a redesign, not patched onto whatever was already half-built.** The banner redesign arrived after the multi-line-title and section- blank-line findings were already made but before any emitter code existed. Because no emitter code existed yet, there was nothing to unwind -- the new design (banner sourced from app/config, never from StepFile) was simply the design used from the first line of emitter.py onward. Worth noting for any FUTURE step that receives a redesign message mid-flight: check what's already been BUILT (irreversible) versus merely INVESTIGATED (cheap to redirect) before deciding whether to unwind or simply proceed differently. 3. **Testing the reconstruction formula against a REAL hand-built example, before writing the pytest suite around it, caught a bug the byte-identity tests against real files would NOT have caught.** The trailer-embedded-in-last-section's-body bug was invisible to the byte-identical corpus round-trip (parsed `body`'s trailer- embedding and the emitter's trailer-handling were both derived from the SAME source text, so they happened to agree even while both were "wrong" relative to a clean model) -- it only surfaced once a hand-built StepFile (no parsed source to accidentally agree with) exposed the double-trailer. This is the general argument for why U2.2's OTHER done-when clause ("over generated skeletons") isn't redundant with the corpus check -- it exercises a different failure mode entirely. ## Numbers ``` +---------------------------------------+----------------------+ | new files | 7 (app/config/ | | | __init__.py+config. | | | txt+loader.py, | | | app/ledger/ | | | emitter.py, tests/ | | | test_ledger_ | | | roundtrip.py+test_ | | | config_loader.py, | | | steps/step.11.txt) | | files edited | 3 (app/ledger/ | | | parser.py additive | | | extension, docs/ | | | step_file_schema.md, | | | end_to_end.md) | | emitter + config loader LOC | ~115 + ~85 = ~200 | | new tests | 79 (61 byte-identical | | | corpus round-trip + | | | 3 banner-path + 4 | | | generated-skeleton + | | | 1 corpus-count-check | | | + 1 own-corpus-count- | | | check + 9 config | | | loader) + 1 from | | | U2.1's own self- | | | inclusive growth | | b3u corpus files round-tripped clean | 50/50 byte-identical | | b3ubot own step files round-tripped | 11/11 byte-identical, | | clean | self-inclusive | | generated-skeleton structural | 4/4 | | round-trips | | | suite before -> after | 125p/11s -> 205p/11s | | regressions | 0 | +---------------------------------------+----------------------+ ``` U2.2 done -- closes the byte-round-trip half of P2's exit criterion ("b3ubot parses and emits its own ledger byte-stably, with the b3u step corpus as the acceptance fixture"). Opens U2.3 (UoW ledger operations), which can now read AND write through a proven emit path. Ledger updated in the work commit per the §12 execution contract; commit hash of that same commit backfilled into both end_to_end.md §11 and this file's own steps/step.11.txt §7 in the immediately following gdiff commit -- the same 2-part resolution step.7/step.9/step.10 used. End of retro.