step.20.txt -- b3ubot U3.7: RETRO closure -- retro pair + ledger row +
               hash backfill + the execution DAG FROZEN as the step's
               evidence graph. design.md §4's `CLOSED <--retro pair--
               RETRO` arrow ("Everything between the gates is machine:
               ... retro drafting"), §2.13's third lifecycle row ("the
               instance is FROZEN as evidence and pointed at by the
               Step's WorkflowDag row"), and §2.3's last ledger op
               ("close(retro) -> done row + hash backfill") -- as
               code. Stops here -- does NOT continue into U3.8 or any
               later P3 UoW.

Step:     20
Parent:   end_to_end.md §5 U3.7 bullet verbatim ("RETRO closure:
          retro pair + ledger row + hash backfill + the execution
          DAG FROZEN as the step's evidence graph"). design.md §4
          (retro drafting is MACHINE work; ratify + escalation are
          the only human gates), §2.13 (the RETRO->CLOSED freeze
          row), §2.3 (the close(retro) ledger op), §2.6 ("clean tree
          per closed UoW; commit messages carry the step trail"),
          §3 (the WorkflowDag entity's kind/dag_uri/status/frozen_at
          fields). app/orchestrator/ (U3.1-U3.6) is the package this
          completes P3's per-state row of; app/ledger/retro.py
          (U2.4) + table_* (U2.3) are the machinery this composes
          rather than duplicates; app/tools/gitops.py is the ONLY
          git path.
UoW:      U3.7
Date:     2026-07-21
Origin:   Operator directive 2026-07-21 (same dispatch shape as
          step.14-19's): draft AND execute steps/step.20.txt for
          U3.7 in a single pass (the step.7/9/10/11/14-19
          convention, NOT a skeleton-split), with the SAME explicit
          HARD scope boundary -- stop after U3.7, do not continue
          into U3.8 or any later UoW; the operator directs P3 one
          UoW at a time with strict per-UoW checkpoints.

## (0) Status

EXECUTED in this pass. Not a skeleton-then-later-execute split --
app/orchestrator/closing.py, the CloseError addition, the
instance_is_frozen/refuse_if_frozen freeze wall grown into
executing.py (+ its verifying.py call sites), the `b3ubot close` CLI
subcommand, tests/test_orchestrator_closing.py, the design.md
(§2.3/§2.13) / end_to_end.md / docs/orchestrator_state_machine.md
edits, the stale "U3.7 pending" docstring sweep, and this step file
were all built together, per the operator's explicit "draft AND
execute (single pass)" direction. Per the operator's explicit scope
boundary (Origin, above), this session STOPS after U3.7 -- U3.8 (the
M1 oracle, incl. automatic stage continuation) is the next UoW but
is NOT started here: a closed UoW is terminal and the engine returns
control to the human between every phase.

## (1) Ground truth probed at drafting (outputs -> named writers)

    READ    design.md §4 in full (retro drafting sits on the MACHINE
            side of the gates; the diagram draws NO human gate
            between RETRO and CLOSED -- ratify and escalation are
            the only two; "freezes the instance at CLOSED as the
            step's evidence graph"), §2.13 (the lifecycle table's
            RETRO->CLOSED row; the WorkflowDag ROW is Track B),
            §2.3 (the ledger-ops list; "close(retro) -> done row +
            hash backfill"), §2.6 (clean tree per closed UoW --
            commit messages carry the step trail), §3 (the
            WorkflowDag entity's field list: kind, dag_uri, status,
            frozen_at -- the precursor row uses it verbatim).
    READ    app/orchestrator/ in full: executing.py (the evidence
            stamps, _load_instance/roundtrip_verify -- reused, not
            re-derived; the resume path the freeze wall had to
            guard), verifying.py (the verification DAG + the
            verifications.jsonl rows the retro transcribes; the
            per-pass instance rewrite the freeze wall had to guard;
            _reset_execution_for_repair -- guarded too),
            state_machine.py (RETRO->CLOSED exists with a U3.7
            rationale string -- NO state-machine change needed, a
            checked finding; CLOSED is terminal, which already
            refuses every LEGAL re-entry -- the freeze wall is for
            the ILLEGAL ones), persistence.py (the history list the
            retro's state-path line and repair count read),
            ratify.py/drafting.py (the legality-first lesson).
    READ    app/ledger/retro.py in full (U2.4): the surveyed corpus
            .diff.txt structural template (title/banner/Executed/4
            headings/closing line) and its OWN scoping rule --
            narrative content must never be FABRICATED; FILL_IN
            placeholders for what a template cannot know. The U3.7
            drafter fills that same structure with MECHANICAL
            evidence that DOES exist at closure time -- the honest
            complement, not a contradiction. run_step_gdiff shells
            to the SHARED scripts/step_gdiff (read in full: START =
            parent of the skeleton commit -- the commit that first
            ADDS step.N.txt; END = newest commit touching it) --
            so a toy workspace whose closure work commit IS the
            skeleton commit gets exactly the closure's own diff.
    READ    app/ledger/table_parser.py/table_transitions.py (U2.3:
            regex-heading table location -- workspace-ledger
            detection rides it; transition("done") requires
            commit=; done is terminal, so a re-run backfills via
            record_commit -- app/cli.py's ledger_close precedent),
            app/tools/gitops.py (status/diff/add/commit exist,
            repo-top verified, git_add requires NAMED paths -- no
            add-everything verb; push/rewrite still refuse),
            trail.py (logged_call -- the step_gdiff run and the
            frozen refusals record through it).
    CONFIRMED dag-level kv attrs round-trip through the REAL
            bin/dagModel BEFORE designing the freeze on them: a Dag
            with (frozen_at = "...") parses and re-renders
            byte-identically (live probe, this session) -- the
            frozen kv is substrate-legal, not hoped-so.
    CONFIRMED `git log --oneline -30` for the 2-commit phrasing
            (step.19's precedent, 529834f/0e9e608).
    CONFIRMED baseline: full suite 517 passed / 11 skipped before
            any edit (matches step.19's reported final).
    CONFIRMED live CLI smoke before committing (scratch workspaces +
            isolated $B3UBOT_HOME), two legs: (a) git-init toy ws
            driven to RETRO, then `b3ubot close` -> both instances
            frozen, evidence-filled retro, WORK commit + REAL short
            hash backfilled into §(7), real step_gdiff gdiff, GDIFF
            commit, porcelain CLEAN, state CLOSED, rc=0; re-close ->
            the terminal refusal verbatim, rc=1; (b) non-git toy ws
            -> "(not a git repository)" markers in §(7), gdiff
            skipped and NAMED, state CLOSED, rc=0. One cosmetic
            live-smoke finding fixed on sight: the retro title
            doubled the UoW prefix (title_line already carries it).

## (2) What U3.7 delivers

  2.1 **app/orchestrator/closing.py** (new; the package's per-state
      peer of drafting/ratify/executing/verifying): `close_uow(
      workspace_root, uow, *, output_fn, home=, scripts_home=)` --
      the full arc: state legality FIRST (validate_transition(state,
      "CLOSED") admits exactly RETRO; zero reads on an illegal
      move), both evidence instances REQUIRED (a RETRO-state UoW has
      been executed and verified; a missing instance is a named
      CloseError), freeze both + WorkflowDag pointer rows, resolve
      the workspace ledger, draft the evidence-filled retro, stamp
      §(0) + rewrite §(7), the 2-commit git shape with real-hash
      backfill (git workspaces), advance RETRO -> CLOSED. Plus
      CloseError in errors.py; all re-exported through
      app/orchestrator/__init__.
  2.2 **Retro drafting V0 (THE core decision, made and owned):
      DETERMINISTIC, evidence-filled -- not a PAL call.** Why: (1)
      closure must be reliable/cheap at the exact moment evidence is
      sealed -- a provider failure/misnarration there is the worst
      possible place for one; (2) every fact a V0 retro needs
      already exists as a machine record (engine-state history, both
      instances' node kvs, verifications.jsonl, the trails) --
      transcribing them IS what makes the retro evidence-based; (3)
      U2.4's own no-fabricated-reflection rule: a PAL-narrated retro
      is a genuine QUALITY upgrade (real reflection, not
      transcription) that rides AFTER M1 proves the arc, as policy,
      not V0 plumbing. FORM matches the corpus convention:
      `draft_closure_retro()` fills retro.py's surveyed structural
      template (title/banner/Executed line/Plan vs actual/Gate
      evidence/Findings/Numbers/closing line) with mechanical
      evidence -- per-node outcomes, per-pass gate verdicts with
      exit codes, the engine-state path, freeze/ledger/gdiff facts,
      node/gate/pass/repair counts -- and its Executed line NAMES
      itself MACHINE-DRAFTED with reflection explicitly not
      synthesized (asserted in-suite: content, not just existence).
  2.3 **The freeze (V0 semantics, decided and justified):** a
      `frozen_at` dag-level kv (design §3's WorkflowDag field name
      VERBATIM, so Track B lifts it unchanged) stamped into BOTH
      instances -- `freeze_instance()` is idempotent (a re-run after
      a partial close keeps the original stamp), written through the
      trail-recorded tool surface, round-trip verified through the
      real bin/dagModel -- PLUS the code-enforced no-writes wall:
      `executing.instance_is_frozen()` (a TEXT sniff on the rendered
      kv, deliberately -- a guard that itself required the substrate
      parser could not protect evidence on a parser-less machine)
      and `refuse_if_frozen()` (refusal trail-recorded as
      `frozen_instance_refusal`, C-15-B: the trail must show what
      the engine REFUSED to do). Call sites: execute_uow (resume AND
      fresh-overwrite paths), verify_uow (the per-pass instance
      write), _reset_execution_for_repair. NOT filesystem read-only
      bits: chmod is neither portable policy nor honest protection
      (any process chmods back); the engine's own modules are the
      writers the design needs to bind, and terminal CLOSED already
      refuses the LEGAL routes -- the kv wall is for the ILLEGAL
      ones (a hand-repaired/divergent engine record -- proven
      in-suite by tampering one).
  2.4 **The WorkflowDag pointer precursor (design §2.13 "pointed at
      by the Step's WorkflowDag row"; Track B not built):**
      `<data_home>/workflow_dags.jsonl` (record_workflow_dag/
      read_workflow_dags) -- the established file-based-precursor
      pattern (egress runs / tool_calls.jsonl / verifications.jsonl),
      one append-only row per frozen instance: ts / uow / step /
      kind / dag_uri / status / frozen_at -- design §3's own field
      list, literally.
  2.5 **Git commit at closure (decision, justified):
      COMMIT-AS-MACHINE-WORK, no approval gate.** design §2.6's
      "clean tree per closed UoW" makes committing PART of closure;
      the in-loop human gate already approved the applied change
      byte-for-byte and the ratify gate approved the plan;
      git_commit sits on §2.7's NON-destructive side (push/history-
      rewrite still refuse unconditionally -- nothing leaves the
      machine); a confirm prompt here would be a third gate design
      §4 does not draw. The 2-commit corpus shape, machine-run:
      WORK commit (retro + frozen instances + §(0) closure stamp +
      §(7) "PENDING" + workspace ledger row -> done/PENDING; paths
      NAMED from git status --porcelain -- no add-everything verb
      exists on the surface, deliberately), then the REAL short hash
      backfilled into §(7) AND the ledger row, the SHARED
      scripts/step_gdiff run (through logged_call -- outcome
      recorded AFTER the fact), and the GDIFF commit. Porcelain
      clean after closure, asserted in-suite. **A NON-git workspace
      closes honestly**: "(not a git repository)" markers in §(7) --
      NEVER a fabricated hash (asserted: no 7-hex token appears) --
      and a gdiff skip NAMED in the retro (step_gdiff needs git);
      closure still completes to CLOSED.
  2.6 **Which ledger (decision, justified): the TARGET workspace's.**
      `<workspace>/end_to_end.md` detected via U2.3's regex-heading
      machinery with a row for the closing UoW; the row goes
      active -> done with commit=PENDING before the work commit and
      gets the real short hash backfilled in the gdiff commit (a
      row already `done` -- a re-run after a partial close -- takes
      record_commit, the ledger_close precedent). No doc / no
      parseable table / no row => a NAMED skip in the retro, never
      an error (a toy workspace without a ledger is legitimate) and
      NEVER this repo's own ledger for a target-workspace UoW (the
      engine only opens the ledger under workspace_root; own-repo
      untouched asserted in-suite).
  2.7 **Human confirmation (decided against, honestly):** `b3ubot
      close` runs to completion once invoked -- no --yes, no
      prompt, no Q-8-B guard, because closure READS NO INPUT.
      Justified against the diagram's own "gates are few" stance:
      design §4 draws no human gate between RETRO and CLOSED
      (ratify + escalation are the only two), and every input
      closure consumes was human-approved upstream. The human act
      IS invoking the command -- V0's separate-command convention
      (draft/ratify/execute/verify/close) keeps the human at the
      wheel BETWEEN phases; automatic continuation is U3.8's M1.
  2.8 **CLI: top-level `b3ubot close`** (the sibling row completed):
      workspace_root + uow + --scripts-home. Presentation-layer
      error handling only; close_uow stays pure and raises;
      unattended runs are honest by construction (no input).
  2.9 **tests/test_orchestrator_closing.py** (17 new): the git-init
      toy-workspace happy path (state CLOSED with the full history;
      retro pair BOTH files; REAL 7-hex hash in §(7) matching the
      actual work commit; founding+work+gdiff = exactly 3 commits;
      porcelain CLEAN); the gdiff records the closure range; retro
      CONTENT asserted (real node kvs incl. applied=hello.py, real
      per-pass verdicts with exit codes, the state path, the
      MACHINE-DRAFTED honesty line, all 4 corpus headings); both
      instances frozen IN THE FILE; pointer rows with matching
      frozen_at; §(0) closure stamp once with the U3.5/U3.6 stamps
      preserved; the non-git honest-marker path; ledger closed with
      the REAL hash AND the named-skip variant; illegal close
      refuses with ZERO read_file trail entries and no retro file;
      close-twice terminal; frozen execute AND verify refusals
      against a TAMPERED engine record with `frozen_instance_
      refusal` trail rows; own steps/ + dag/instances/ untouched;
      CLI unattended green + error presentation. Hermetic
      $B3UBOT_HOME throughout (conftest).
  2.10 **Docs (same commit):** end_to_end.md §5 U3.7 row -> DONE in
      the U3.2-U3.6 row style; §11 ledger THROUGH the ledger CLI
      (transition U3.7 active --step step.20; close U3.7 PENDING;
      hash backfilled in the gdiff commit -- the step.13-19 shape).
      design.md §2.3 delivered-state (all four ledger ops now
      exist), §2.13 delivered-state (the freeze row is live; the
      pointer precursor named). docs/orchestrator_state_machine.md
      (U3.7 delivered; edge-table annotation). Every stale "U3.7
      pending"/"nothing drafts the retro pair yet" claim updated
      (verifying.py stamp + completion message + docstring;
      state_machine.py rationale string; app/orchestrator/
      __init__.py). No new docs/*.md file -- this file is the
      writeup.

EXPLICITLY OUT OF SCOPE (named here per the operator's brief, not
silently built -- and NOT continued into after this step, per the
operator's explicit stop-here scope boundary):
  - U3.8 M1 (the one-complete-UoW arc, mock AND real provider;
    automatic stage continuation lives there).               -> next
  - A PAL-narrated retro (the named post-M1 quality upgrade
    of 2.2's deterministic V0).                              -> later
  - T2+/T3/T4 tiers (unchanged from U3.6's boundary).        -> later
  - Auto re-draft on amendments (Q-17-A stays open).         -> later
  - Track B / P4 (WorkflowDag stays the JSONL pointer
    precursor; no DB row).                                   -> later
  - Pushing anything anywhere (git_push still refuses; B-3). -> never
    (without explicit operator direction)

## (3) Verification gates

    G1  tests/test_orchestrator_closing.py 17/17 PASS live: happy
        path on a git-init toy workspace, gdiff range, retro
        evidence CONTENT, freeze in-file, pointer rows, stamp-once,
        non-git honest markers, both ledger variants, legality,
        terminality, missing-instance refusal, frozen execute/verify
        refusals, own-repo hygiene, CLI -- scripted adapters,
        injected I/O, zero network.
    G2  The REAL closure artifacts proven, not mocked: the .gdiff.txt
        comes from the SHARED v3_claude/scripts/step_gdiff run
        against the toy repo's actual commits; the §(7) hash is the
        actual `git rev-parse HEAD` short form of the work commit
        (asserted equal); porcelain clean after closure (design
        §2.6's own rule, asserted).
    G3  Frozen-instance enforcement proven at the FILE level, not
        just the state level: the engine record tampered back to
        EXECUTING/VERIFYING (the illegal route the state machine
        cannot see) -> execute_uow/verify_uow refuse loudly naming
        FROZEN evidence, with `frozen_instance_refusal` rows on the
        tool trail (C-15-B).
    G4  Substrate round-trip: dag-level `(frozen_at = "...")` kv
        confirmed byte-round-trip-safe through the REAL bin/dagModel
        BEFORE the design committed to it (§(1) CONFIRMED); every
        frozen instance re-verified through roundtrip_verify at
        freeze time.
    G5  Live CLI smoke (not just tests), two legs -- §(1) CONFIRMED:
        git-init scratch ws close rc=0 (2 commits, real hash, clean
        porcelain, evidence-filled retro) + terminal re-close rc=1;
        non-git scratch ws close rc=0 with honest markers. Confirmed
        live before committing.
    G6  Full suite before this step's edits: 517 passed / 11 skipped
        (confirmed live, matching step.19's reported final). After:
        534p/11s pre-step-file (+17 closing); 536p/11s once this
        step.20.txt lands in the corpus (+2 self-inclusive pickups
        -- the steps 10-19 growth pattern). Zero regressions; zero
        existing tests edited.
    G7  Scope boundary held: no U3.8 arc, no automatic
        verify->close continuation, no PAL retro, no Track B row,
        no T3/T4 flow, nothing pushed anywhere; this repo's own
        steps/ and dag/instances/ untouched by every test (asserted
        in-suite); this session stops after U3.7.
    G8  §11 ledger row updated THROUGH the existing `ledger` CLI
        (transition -> active --step step.20, close -> done/PENDING),
        against the REAL end_to_end.md -- dogfooding, the step.13-19
        precedent.
    G9  The closure convention matches this repo's own: the toy
        workspace's machine-run closure leaves the SAME artifact
        set (step.N.txt with §(0) stamps + §(7) real hash,
        step.N.diff.txt, step.N.gdiff.txt, done ledger row, 2
        commits) that steps 13-19 left by hand -- the U3.7 bullet's
        "indistinguishable in form" bar, checked artifact by
        artifact in G1/G2.
    G10 b3ubot porcelain clean after each commit; retro pair via the
        SHARED v3_claude/scripts/step_gdiff (steps/step.20.gdiff.txt);
        exactly 2 commits (work + gdiff-with-hash-backfill-folded-in),
        the step.7-19 precedent.

## (4) LOCKs

    C-20-A  b3ubot porcelain clean after each commit.
    C-20-B  FROZEN EVIDENCE IS NEVER REWRITTEN: after RETRO->CLOSED,
            executing.py and verifying.py refuse loudly (trail-
            recorded) before touching either instance -- even under
            a tampered/divergent engine record. The freeze kv is
            design §3's `frozen_at`, verbatim, so Track B lifts it
            unchanged.
    C-20-C  THE RETRO TRANSCRIBES, NEVER INVENTS (U2.4's rule
            carried into the machine): the V0 closure retro contains
            ONLY mechanical facts drawn from the real evidence
            records, names itself MACHINE-DRAFTED, and marks
            reflective narrative as not synthesized. A PAL-narrated
            retro is a post-M1 policy upgrade, never smuggled in.
    C-20-D  NO FABRICATED HASHES: a non-git workspace gets the
            honest "(not a git repository)" marker in §(7) and a
            NAMED gdiff skip -- never an invented 7-hex token
            (asserted in-suite).
    C-20-E  COMMIT-AS-MACHINE-WORK STAYS INSIDE §2.7's LINE: closure
            commits via gitops' non-destructive verbs only, with
            NAMED paths; git_push/git_rewrite_history still refuse
            unconditionally; nothing leaves the machine (B-3/B-5).
    C-20-F  THE TARGET WORKSPACE'S LEDGER ONLY: closure never opens
            this repo's own end_to_end.md for a target-workspace
            UoW; absence is a NAMED skip, not a fallback.
    C-20-G  NO SILENT SCOPE CREEP: U3.8, the PAL retro, T3/T4,
            Track B and Q-17-A named out of scope in (2) and
            confirmed not started (G7); the operator's explicit
            "stop after U3.7" direction is honored.
    C-20-H  standing locks inherited: B-3 (no disclosure/no push);
            B-4/B-5 (no provider call in closure -- nothing new
            leaves the machine; the trails record everything);
            C-15-B (every closure file op + the step_gdiff run +
            every frozen refusal on the tool trail); C-18-B/C-19-E
            (the instance files stay the truth -- the freeze makes
            them IMMUTABLE truth).

## (5) Open questions

    Q-20-A  Should the freeze also snapshot per-pass verification
            graphs (step.19's Q-19-C deferred exactly this to U3.7)?
            DECIDED HERE for V0: no -- the frozen verification
            instance holds the FINAL pass, verifications.jsonl holds
            every pass's verdicts, and the retro transcribes both;
            a per-pass graph-file family would multiply files for
            evidence the JSONL already carries. Recorded as decided,
            not left open -- revisit only if Track B's real
            WorkflowDag rows want per-pass graph pointers.
    Q-20-B  Should `b3ubot close` verify the workspace suite one
            last time before committing (a paranoid re-check of
            U3.6's green)? LEAN: no -- VERIFYING->RETRO already
            required green gates and re-running them would make
            closure slow and double-spend the oracle; the engine
            record's history IS the proof the gates ran. Revisit at
            U3.8 if the M1 arc surfaces a real staleness window
            between verify and close.
    Q-20-C  Where should a CLOSED UoW's engine-state record
            eventually retire to (it stays in ~/.b3ubot/orchestrator/
            forever today)? LEAN: nowhere until Track B -- the
            file IS the durable record and P4's Step row will
            subsume it; premature archival would break the resume
            reader for no consumer.

## (6) Acceptance

    - [x] app/orchestrator/closing.py: legality-first close_uow --
          freeze + pointer rows + evidence retro + §(0)/§(7) stamps
          + 2-commit git shape + RETRO->CLOSED (2.1, G1, G2, G5).
    - [x] Retro drafting V0 decided and owned: deterministic
          evidence-filled template, corpus FORM, MACHINE-DRAFTED
          honesty line; PAL retro named post-M1 (2.2, C-20-C).
    - [x] Freeze semantics decided and enforced: frozen_at dag-level
          kv (substrate-verified) + refuse_if_frozen in executing
          AND verifying, refusals trail-recorded, proven against a
          tampered record (2.3, G3, G4, C-20-B).
    - [x] WorkflowDag pointer precursor: workflow_dags.jsonl rows
          with design §3's field list (2.4).
    - [x] Git-at-closure decided: commit-as-machine-work justified
          against §2.6/§2.7/§4; real-hash backfill into §(7) AND the
          ledger row; honest non-git markers, no fabricated hashes
          (2.5, G2, C-20-D, C-20-E).
    - [x] Which-ledger decided: the target workspace's, detected;
          named skip otherwise; own repo never a fallback (2.6,
          C-20-F).
    - [x] Closure keeps NO human confirmation once invoked --
          justified against design §4's own gate set; `b3ubot close`
          is the no-input sibling (2.7, 2.8, G5).
    - [x] 17 new tests, zero regressions, zero edited tests
          (517p/11s -> 534p/11s -> 536p/11s with this file) (2.9,
          G1, G6).
    - [x] design.md §2.3/§2.13; docs/orchestrator_state_machine.md;
          end_to_end.md §5 row + §11 ledger via the `ledger` CLI;
          every stale "U3.7 pending" docstring (2.10, G8).
    - [x] Scope boundary held: U3.8 + PAL retro + T3/T4 + Track B
          not started (G7, C-20-G) -- this session stops after U3.7.
    - [x] Retro pair: this file + step.20.diff.txt + step.20.gdiff.txt
          via the SHARED scripts/step_gdiff (G10).
    - [x] Exactly 2 commits (work + gdiff-with-backfill), not 3 (G10).

## (7) Hash backfill
WORK commit (app/orchestrator/closing.py + errors + the freeze wall
in executing.py/verifying.py + state_machine rationale + __init__ +
cli + tests + design.md/end_to_end.md/docs + this file):  7cbee41
GDIFF commit (step.20.gdiff.txt via scripts/step_gdiff):  (HEAD)

End of step.
