step.10.txt -- b3ubot U2.1: step-file parser -- the canonical step-file
               shape as a schema, corpus-surveyed against the FULL b3u
               step corpus (not assumed from step.7-9's own shape),
               refuse-loudly on unknown sections and malformed input.

Step:     10 -- promote UoW **U2.1** (end_to_end.md §4, P2) to execution
                depth: the FIRST P2 UoW and a genuinely new subsystem
                (not an incremental PAL/CLI extension like steps 7-9) --
                a parser module that turns a `step.N.txt` file into a
                structured `StepFile` (header fields + numbered
                sections with raw body text), grounded in an empirical
                survey of the b3u corpus across its full age range, not
                an assumption that step.7/step.8/step.9's own shape is
                representative of the whole 50-step history.
Parent:   end_to_end.md §4 P2 entry/exit ("b3ubot parses and emits its
                own ledger byte-stably, with the b3u step corpus as the
                acceptance fixture") and the U2.1 bullet verbatim ("the
                canonical sections as a schema; the 34 b3u step files
                (read-only fixtures) must parse. Done when: corpus
                parses; unknown-section and malformed inputs refuse
                loudly" -- the "34" is corrected to 50 in this step,
                see (1)). design.md §2.3 (The ledger engine: "canonical
                sections (Status / ground truth / deliverables / gates
                / LOCKs / open questions / acceptance / hash
                backfill)") and §1.2 ("Honest interims: partial
                mechanisms are named at creation... never silently
                deferred") for the refuse-loudly framing. design.md §3
                Data Model's `Step (UoW)` entity is a DIFFERENT, LATER
                Track B/P4 concern (SQLite/OO-API layers) -- explicitly
                NOT touched by this step, named to avoid the confusion
                the operator's own brief flagged up front.
UoW:      U2.1 -- "Step-file parser (b3u corpus)". First UoW of P2 (the
                ledger engine, Track A). Depends on nothing built so
                far (P2 has no prerequisite UoW inside itself); P2 as a
                phase depends on P1 (done, U1.1-U1.6 all closed).
                Unblocks U2.2 (emitter + byte round-trip oracle), which
                needs this step's `StepFile` shape to emit FROM.
Date:     2026-07-20  (drafted and executed in the same pass, at the
                operator's explicit request -- the step.7/step.9
                single-pass rhythm, not step.8's SKELETON-then-execute
                split.)
Origin:   Operator directive 2026-07-20: "go with U2.1, step file and
                its execution" -- single pass.

## (0) Status

EXECUTED in this pass. Not a skeleton-then-later-execute split like
step.8 -- the parser module, the schema doc, the tests, and this step
file were built together per the operator's explicit direction.

## (1) Ground truth probed at drafting (outputs -> named writers)

    READ    end_to_end.md §4 (P2 entry/exit + all four U2.x bullets)
            and §11 ledger (U2.1 row: `todo`, no step number -- confirmed
            before starting) and §12 maintenance rules (same-commit
            ledger update; step.7/step.9's PENDING-then-backfill
            resolution of the "this commit's own hash is unknown while
            writing it" problem, confirmed via `git show --stat 4f24b22`
            and `git show 4f24b22 -- end_to_end.md steps/step.7.txt`
            before repeating it here rather than assuming it).
    VERIFIED  `ls b3u/steps/step.*.txt | grep -vE '\.(diff|cdiff|gdiff)
            \.txt$'` (relative to the v3_claude parent) -> 50 files,
            `step.1.txt` through `step.50.txt`, contiguous, no gaps
            (`sort -V` then `wc -l` = 50; range check against
            `seq 1 50` matched exactly). CORRECTS end_to_end.md's U2.1
            bullet, which said "34" -- that count was accurate when P2
            was originally scoped (2026-07-12, repo founding); the b3u
            corpus grew from 34 to 50 by 2026-07-20 (b3u's own P6
            production-launch arc, steps 45-50, landed since). Fixed in
            end_to_end.md §4 in this same commit (small, real, worth
            recording per the operator's own framing).
    READ    a SPREAD of the b3u corpus across its full age range, not
            just the steps this session already knew: EARLY --
            b3u/steps/step.1.txt (founding, no UoW: field -- predates
            UoW numbering), step.5.txt (U3.3, OO API core), step.10.txt
            (U3.7, the b3ucli REPL -- b3u's OWN step.10, a DIFFERENT
            file in a DIFFERENT repo from this one); MID -- step.20.txt
            (U3.10), step.30.txt (UL.3), step.45.txt (U6.1); RECENT --
            step.48.txt (U6.4), step.49.txt (U6.5), step.50.txt (U6.6,
            the FINAL P6 UoW -- b3u's corpus tops out at 50, not the
            140s the task brief's own "recent examples" guess named;
            those numbers don't exist in b3u/steps/, confirmed by `ls`
            before reading anything, not assumed). This superseded an
            assumption carried in from steps 7-9 (that step.7.txt/
            step.8.txt/step.9.txt's OWN shape, all drafted THIS repo,
            THIS week, is representative of the full 50-step b3u
            history) -- it mostly held, but two real deviations
            surfaced (next item).
    FOUND   (grep-driven, across all 50 files, not eyeballed) two
            genuine era-variations the "read 3 files and assume" path
            would have missed:
              (a) section (0) is titled exactly "Status" in 50/50
                  files -- the one true universal invariant;
              (b) the gate-enumeration section is titled "Verification
                  gates" in 49/50 files but "Oracle (gates G1-G3)" in
                  b3u/steps/step.8.txt -- same role, different title;
              (c) the trailing-section run (Verification gates / LOCKs
                  / Open questions / Acceptance / Hash backfill) is in
                  that RELATIVE ORDER in 49/50 files, but
                  b3u/steps/step.8.txt has Hash backfill BEFORE
                  Acceptance;
              (d) header field `UoW:` is present in 49/50 (absent only
                  from step.1.txt, the founding step, which predates
                  UoW numbering) while `Step:`/`Parent:`/`Date:`/
                  `Origin:` are 50/50;
              (e) the `End of skeleton.`/`End of step.` trailer is
                  present in 49/50 (b3u/steps/step.44.txt does not end
                  that way).
            Full table in docs/step_file_schema.md. Design decision
            flowing from this: the parser enforces the MARKER SHAPE
            (`## (N) <title>`, N strictly increasing) and section (0)
            == "Status" as hard requirements, but does NOT enforce a
            fixed closing-section title vocabulary, a fixed relative
            order beyond section 0, or the trailer -- the real corpus
            does not support that rigidity, and force-fitting it would
            make step.8.txt itself fail to parse, which would be
            absurd (it is a real, accepted, closed step in the source
            corpus).
    READ    steps/step.7.txt + step.7.diff.txt and steps/step.9.txt +
            step.9.diff.txt (the two prior single-pass steps in THIS
            repo) for the retro-pair format and the exact 2-commit
            resolution of the ledger-row self-reference problem.
    READ    steps/step.1.txt through step.9.txt (this repo's own step
            files) -- confirmed they mirror the SAME b3u convention
            (Status-first, numbered sections, LOCKs/Open questions/
            Acceptance/Hash backfill trailing run), a second, smaller,
            already-familiar reference point -- NOT the graded fixture
            set (end_to_end.md's own wording names the b3u corpus), but
            worth parsing too as a bonus check (§2 below).
    READ    design.md §3 (Data Model, Step (UoW) entity) in full and
            confirmed it is Track B/P4 (the SQLite/OO-API five-layer
            pipeline) -- a DIFFERENT, LATER, independent track from
            this step's Track A plaintext-file parsing. Not touched.
    READ    design.md §1.2 ("Honest interims") and §2.7 (the tool
            surface's "refuse with a named next action, no --force"
            posture) for the refuse-loudly exception-message framing.
    READ    app/pal/errors.py in full: the `AdapterError` base +
            `AdapterUnavailableError`/`AdapterRunError` subclass shape
            (each with a docstring naming the LOCK/precedent it
            enforces) -- the template this step's
            `app/ledger/errors.py` mirrors exactly (`StepFileError`
            base + `UnknownSectionError`/`MalformedStepFileError`).
    READ    app/egress.py in full: the `$B3UBOT_HOME` env-var-with-
            default-path resolution pattern, and scripts/smoke.sh's
            `$CPPCCHOME` sibling-repo-path pattern (`HERE=$(cd
            "$(dirname ...)/.." && pwd)`, `CPPCCHOME="${CPPCCHOME:-
            $HERE/../cppcc}"`) -- the template
            tests/test_ledger_parser.py's `$B3U_SIBLING_HOME`
            (default `../b3u` relative to this repo's root) mirrors.
    READ    app/tools.py + tests/test_tools.py for this repo's pytest
            style (plain functions, `pytest.raises`, `tmp_path`,
            docstring citing the UoW/LOCK it proves) and
            app/pal/__init__.py / app/__init__.py (both 0 bytes) for
            the empty-`__init__.py` convention `app/ledger/__init__.py`
            follows.
    CONFIRMED  `git log --oneline -25` for exact commit-message
            phrasing: step.7/step.9's two-commit shape (`b3ubot step.N:
            UX.Y -- ...` then `b3ubot step.N: add step.N.gdiff.txt via
            scripts/step_gdiff`) is this step's template.

## (2) What U2.1 delivers

  2.1 **app/ledger/parser.py** (new): `parse_step_file(path)` /
      `parse_step_text(text, source=...)` -> `StepFile` (`source`,
      `title_line`, `header_fields: dict[str,str]`,
      `sections: list[Section]`, `trailer: str | None`); `Section`
      (`number`, `title`, `body` -- raw text verbatim, outer blank
      lines only trimmed). `StepFile.section(n)` / `.section_titled(t)`
      convenience lookups. Header-field parsing recognizes exactly the
      five labels the corpus survey found (Step/Parent/UoW/Date/
      Origin), continuation lines by indentation under the label,
      matching the observed shape byte-for-byte. Section parsing
      requires the `## (N) <title>` marker shape and strictly
      increasing N; does NOT validate title text past section (0) (per
      (1)'s findings).
  2.2 **app/ledger/errors.py** (new): `StepFileError` base,
      `UnknownSectionError` (an unrecognized header-field label, or a
      `## ` line that isn't the numbered-marker shape),
      `MalformedStepFileError` (missing required header field, no
      section markers at all, section (0) not titled "Status", or
      section numbers not strictly increasing) -- mirrors
      app/pal/errors.py's `AdapterError` family shape exactly; no
      generic `ValueError` anywhere in this module.
  2.3 **docs/step_file_schema.md** (new): the live schema doc --
      corpus-surveyed shape (title line / INTERNAL banner / header
      fields / numbered sections / optional trailer), the header-field
      presence table, the closing-section title/order era-variation
      table, and what U2.2/U2.3/U2.4 will build on top of this return
      shape (named, not built).
  2.4 **tests/test_ledger_parser.py** + **tests/fixtures/ledger/**
      (new): (a) all 50 real b3u corpus files parse without raising,
      parametrized per-file (`$B3U_SIBLING_HOME`-resolved, default
      `../b3u`, honest `pytest.skip` naming the override if the
      sibling isn't checked out); a live tripwire asserting the corpus
      is exactly `step.1.txt`..`step.50.txt` (50, contiguous) so a
      FUTURE corpus-size drift re-triggers a survey rather than going
      stale silently a second time; (b) this repo's own step files
      parse too (bonus, not graded -- self-inclusive: this collects
      whatever `steps/step.N.txt` files exist on disk at test-collect
      time, which includes step.10.txt itself once it's written, so
      the count is 10/10 this run, `>=9` asserted rather than a fixed
      number); (c) six synthetic malformed/
      unknown-section fixtures (unknown header field, unknown section
      marker, missing header entirely, wrong section-0 title,
      out-of-order section numbers, no section markers at all), each
      asserted against the SPECIFIC exception type AND a message
      substring (not "it raised something"); (d) one known-good shape
      test against steps/step.9.txt asserting header-field/section/
      trailer content, not just non-raising.
  2.5 **end_to_end.md** same-commit edits: §4 U2.1 bullet -> DONE
      (step.10), 34 corrected to 50 with the verification method named,
      a `*Delivered:*` line (mirrors U1.5/U1.6's own style); §11 ledger
      row -> done/step.10/PENDING (backfilled at the gdiff commit, the
      step.7/step.9 resolution of the self-reference problem).

EXPLICITLY OUT OF SCOPE (named here per the operator's brief, not
silently built):
  - The EMITTER / byte-round-trip oracle (U2.2) -- `emit(parse(x)) ==
    x`. This step's `StepFile` shape is designed to make that
    achievable (raw body text preserved verbatim) but no emitter
    exists yet.                                                   -> U2.2
  - UoW ledger-table operations / status transitions (§9-table model:
    todo -> active -> done/dropped) (U2.3).                        -> U2.3
  - Retro-pair machinery, cdiff generation, the `b3ubot ledger` CLI
    surface (U2.4).                                                -> U2.4
  - design.md §3's Data Model (`Step (UoW)` SQLite entity, the five-
    layer verbal->SGDL->SQLite->OO-API->CLI pipeline) -- Track B, P4,
    unrelated to this step's plaintext-file parsing.                -> P4
  - Fixing design.md §2.3's own stale "34 b3u step files" reference
    (line ~142) -- the operator's brief scoped the correction to
    end_to_end.md's U2.1 bullet specifically; design.md's instance is
    left as a named, deliberate follow-up, not silently inconsistent
    without comment.                                          -> next touch
    of design.md §2.3

## (3) Verification gates

    G1  All 50 real b3u corpus files (step.1.txt..step.50.txt) parse
        without raising, verified live via pytest parametrization, one
        assertion per file (verified live: 50/50 PASS).
    G2  This repo's own step files parse too (bonus check, not graded;
        self-inclusive of step.10.txt once written) (verified live:
        10/10 PASS).
    G3  Each of the six synthetic malformed/unknown-section fixtures
        raises the SPECIFIC exception type named for it
        (UnknownSectionError vs MalformedStepFileError) with a message
        containing the concrete defect, asserted via `pytest.raises(...,
        match=...)` (verified live: 6/6 PASS, wrong-exception-type
        would fail the same as wrong-message-content).
    G4  Corpus-size tripwire: the b3u corpus is asserted to be exactly
        `step.1.txt`..`step.50.txt` (50 files, contiguous); this is a
        DELIBERATE early-warning assertion, not incidental (verified
        live: PASS at 50).
    G5  NO REGRESSION: the pre-existing 55p/11s baseline (post-step.9)
        is additive only -- no existing test file touched, no existing
        `app/`/`adapters/` file touched except the new `app/ledger/`
        package (verified live: `pytest -q` -> `125 passed, 11 skipped`,
        delta = +70 passed, 0 skipped delta, 0 failures).
    G6  end_to_end.md §4/§11 updated same commit as the work (§2.5);
        ledger row carries `PENDING` for the commit hash (the self-
        reference problem), backfilled at the gdiff commit.
    G7  Hygiene: b3ubot porcelain clean after each commit; retro via
        the SHARED `v3_claude/scripts/step_gdiff` (not a b3ubot-local
        copy -- the step.8/step.9 finding, confirmed again rather than
        rediscovered).

## (4) LOCKs

    C-10-A  b3ubot porcelain clean after each commit.
    C-10-B  READ-ONLY CORPUS: the b3u step files are read from
            `../b3u/steps/` IN PLACE via `$B3U_SIBLING_HOME`
            (default) -- never copied into this repo, never written
            to, never modified by any test (end_to_end.md's own "the
            50 b3u step files (read-only fixtures)" wording, and this
            step's own tests/test_ledger_parser.py only ever calls
            `Path.read_text`, never `write_text`, against that tree).
    C-10-C  NO SILENT SCOPE CREEP: U2.2 (emitter/round-trip oracle),
            U2.3 (ledger ops), U2.4 (retro-pair/CLI machinery), and
            design.md §3's Track B/P4 SQLite layer are named out of
            scope in §2 above, not silently begun.
    C-10-D  NO GENERIC EXCEPTIONS: every refusal in app/ledger/
            raises `UnknownSectionError` or `MalformedStepFileError`
            (never a bare `ValueError`/`Exception`), each with a
            message naming the concrete defect -- the "refuse with a
            named next action" posture (design.md §2.7) applied to
            parsing, not just to the tool surface / CLI it was
            originally written for.
    C-10-E  SCHEMA NOT OVER-FIT: the parser's structural requirements
            (marker shape, section (0) == "Status", strictly-
            increasing section numbers, five known header-field
            labels) are exactly what the FULL 50-file corpus survey
            supports -- no requirement was added that b3u/steps/
            step.8.txt (or any other real file) would fail. This is
            the load-bearing decision the whole step turns on; see
            (1)'s findings and docs/step_file_schema.md.
    C-10-F  standing locks inherited: B-3 (no disclosure/no push);
            B-6 (this step's gates are T0/T1 deterministic pytest
            assertions; no AI review tier involved). B-1/B-2/B-4/B-5
            are moot -- this step invokes no cppcc, no provider, no
            egress.

## (5) Open questions

    Q-10-A  Should "unknown section" cover unrecognized BODY-section
            titles (e.g. a numbered section titled something the
            schema has never seen), not just header-field labels and
            non-numbered `## ` markers? LEAN (built this way): no --
            the corpus survey (1) found real, legitimate title
            variation in the body/closing run (e.g. "Oracle (gates
            G1-G3)" instead of "Verification gates" in
            b3u/steps/step.8.txt); enforcing a title vocabulary there
            would reject a real, already-accepted, already-closed step
            in the source corpus, which would falsify the parser
            against its own graded fixture set. Revisit only if a
            FUTURE corpus survey (U2.2+ or beyond) finds title drift
            that actually breaks something downstream (e.g. U2.3's
            ledger ops needing to KNOW which section is "the gates
            section" reliably) -- at that point a `kind:` classifier
            with a small synonym table (not a hard reject) is the
            likely shape, not a schema tightening.
    Q-10-B  Should the parser enforce the `End of skeleton.`/`End of
            step.` trailer, given it's 49/50? LEAN (built this way):
            no -- same reasoning as Q-10-A; captured as `.trailer`
            when present (informational), `None` otherwise, never a
            parse failure. b3u/steps/step.44.txt is a real, accepted
            step that doesn't end that way.
    Q-10-C  `$B3U_SIBLING_HOME` vs mirroring `$B3UBOT_HOME`'s exact
            name pattern more closely (e.g. `$B3U_STEPS_DIR` pointing
            straight at the steps/ dir rather than the sibling repo
            root)? LEAN (built this way): sibling-repo-root env var
            (`$B3U_SIBLING_HOME`, `/steps` appended in code) --
            mirrors `$CPPCCHOME`'s shape (points at a repo root, not a
            specific subdirectory inside it) more closely than
            `$B3UBOT_HOME`'s shape (points at a per-developer DATA
            home, a different kind of thing); a future U2.2+ that
            needs OTHER b3u paths (not just steps/) reuses the same
            var without a second env var being invented.

## (6) Acceptance

    - [x] app/ledger/parser.py + app/ledger/errors.py + app/ledger/
          __init__.py (2.1, 2.2, G1-G4).
    - [x] docs/step_file_schema.md (2.3).
    - [x] tests/test_ledger_parser.py + tests/fixtures/ledger/ (6
          fixtures) (2.4, G1-G4).
    - [x] All 50 b3u corpus files parse clean -- verified live, 50/50
          (G1).
    - [x] This repo's own step files parse clean -- verified live,
          10/10, self-inclusive of step.10.txt (G2, bonus).
    - [x] Six refuse-loudly fixtures raise the specific exception type
          + message content -- verified live, 6/6 (G3).
    - [x] Corpus-size tripwire asserted at exactly 50 -- verified live
          (G4).
    - [x] Full suite: 125 passed / 11 skipped (was 55p/11s) -- +70
          passed, 0 skipped delta, 0 failures, 0 files outside
          app/ledger/, docs/, tests/ touched except end_to_end.md
          (G5).
    - [x] end_to_end.md §4 U2.1 bullet -> DONE (step.10), 34 -> 50
          corrected with method named, *Delivered:* line added; §11
          ledger row -> done/step.10/PENDING (2.5, G6).
    - [x] Retro pair: this file + step.10.diff.txt + step.10.gdiff.txt
          via the SHARED `v3_claude/scripts/step_gdiff` (G7).

## (7) Hash backfill

    WORK commit (parser + errors + docs + tests + this file + ledger):  424f7b2
    GDIFF commit (step.10.gdiff.txt via scripts/step_gdiff):            (HEAD)

End of step.
