step.5.txt -- b3ubot U3.9 (first slice): DAG substrate adoption -- rebuild the dag/ fork's whole toolchain (cppcc frontend, cctdag CLI, 4-language OO APIs) for the FIRST time since it was cloned in at founding, fix a directory-depth bug that silently broke 12 inherited scripts in their new location, and get the upstream regression suite (byte- identity matrix + 2-pass round-trip) GREEN IN-REPO. Pulled forward ahead of P2 at the operator's explicit direction -- self-contained, no P1/P2 dependency. Step: 5 -- promote UoW **U3.9** (end_to_end.md §5, P3) to execution depth, as a FIRST SLICE: everything design.md §2.13 and end_to_end.md's own U3.9 text calls "rebuild the dag/ toolchain... the upstream regression suite green IN-REPO" -- explicitly NOT the same UoW's third clause ("first b3ubot-kind instance (`execution`) authored + round-tripped"), which is real NEW authoring work on top of a substrate this step only PROVES still works, and gets its own follow-on UoW (2.6, below). Parent: end_to_end.md §5 U3.9 ("DAG substrate adoption... Precedes every other P3 UoW") + design.md §2.13 (the workflow DAG substrate) + dag/PROVENANCE.md (the fork, cloned 2026-07-12 from bet3/bot/dag @ 4f54126, commit f1ebb47 -- "163 files 1.3M," purely additive, NEVER built or tested since). UoW: U3.9 -- "DAG substrate adoption" (first slice). No hard dependency on U1.x/U2.x (design.md's own "Track B can run in parallel with P1-P3" framing) -- self-contained within dag/, pulled forward at the operator's explicit direction 2026-07-18 ("create a step... and rebuild b3ubot/dag including its all tests"), following the earlier b3u.dev/b3ubot sequencing precedent (work reordered on explicit instruction, not silently). Date: 2026-07-18 (SKELETON -- not executed. Execute on explicit request.) Origin: User directive 2026-07-18: fix the wc_bot_dag_*.sh path bug (found during the wc_b3ubot work) and "rebuild b3ubot/dag including its all tests." ## (0) Status SKELETON. Nothing built yet. Ground truth below comes from a dedicated research pass (not assumption) -- including a CORRECTION to my own prior claim: I originally told the operator the fork sits "one level deeper" than the original and needs one MORE `../`. That was backwards. Verified directly by resolving the path chain from an actual script location: the fork sits one level SHALLOWER (`b3ubot/dag` is 2 path segments; the original `bet3/bot/dag` is 3, via an intermediate `bot/` layer b3ubot's fork never had) -- the fix removes one `../`, not adds one. ## (1) Ground truth probed at drafting EXISTS (the fork, untouched since cloning): `f1ebb47`'s diffstat is purely additive (163 files) with an explicit commit- message note: "build outputs excluded... regenerable via makealls" -- i.e. NOTHING was ever built, only source copied. `end_to_end.md`'s own ledger row for U3.9 is `todo`, no step file, no commit -- confirmed this was never done, not once, not even a throwaway build at founding. EXISTS (the master driver): `dag/tests/run_tests.sh` loops a fixed, dependency-correct order -- `cppcc sql dagModelCpp dagCommands dagModelPy dagModelJs dagModelRust instances bootstrap regression` -- invoking each subproject's OWN `tests/run_tests.sh`. EVERY per-subproject runner is SELF-BUILDING (checked directly: dagModelCpp's builds `libdagModelCpp.a` if missing, dagModelRust's builds the `dagModel` parser via `cppcc/build/makeall` if missing, dagCommands' builds `cctdag` if missing, bootstrap's builds BOTH if missing) -- a single `bash dag/tests/run_tests.sh` cascades build-then-test through the whole toolchain in one command, given the fixes below. EXISTS (CPPCCHOME, an ALREADY-established b3ubot convention): step.1's `scripts/smoke.sh:24` already does `CPPCCHOME="${CPPCCHOME:- $HERE/../cppcc}"` (default `v3_claude/cppcc`, confirmed present + executable there). This step's driver exports the SAME value once, up front, for every `cppcc/build/ makeall` invocation the cascade triggers. EXISTS (all 4 language toolchains, confirmed present this dev box): cargo 1.96.0, node 18.19.1 + npm 9.2.0, python3 3.12.3 + uv, g++ (Ubuntu 13.3.0), make. NOTHING is missing -- a clean run should show ZERO toolchain-driven skips. EXISTS (the regression suite's real scope, `regression/bin/ run_regression` + `regression/lib/*.sh`, read in full): 6 corpora (`sample`, `inst_d1H` -- both static fixtures already in-repo -- plus `proj_d1H`/`proj_d1Dep`/ `proj_d1Iso`/`proj_d1TODO`, generated fresh each run by `bootstrap/bin/bot_init` into `$VERSION_HOME/dags/`) x 4 renderers (`cctdag` CLI, dagModelPy, dagModelJs, dagModelRust -- each SELF-CONTAINED, no external sibling- repo dependency: Python/JS load their OWN `dag_model` package via a direct path, Rust builds a throwaway render binary from ITS OWN crate) = a 24-cell byte-identity matrix; PLUS a 2-pass round-trip oracle (load->DB1-> unload->assert byte-identical->load again->DB2->assert counts match) via `cctdag`; PLUS a quirk-landscape regression (`check_scb_quirks.py`, E1/E2/E3 baseline). `run_regression` self-builds the `cppcc` parser + `cctdag` CLI if missing, matching the per-subproject runners. FOUND (the corpora path is SHARED, workspace-scoped, on purpose): `$VERSION_HOME/dags/` is NOT b3ubot-private -- `bot_init`'s generators build workspace-wide dependency/hierarchy/ TODO graphs describing `v3_claude` as a whole (the "bootstrap kind lineage," design.md §2.13), and the comment says explicitly "Idempotent -- running twice produces byte-identical outputs." The SAME original bet3/ bot/dag repo, if ever run, would resolve to the identical `v3_claude/dags/` path -- this is intentional sharing of a workspace-level artifact, not cross-repo contamination, and needs no isolation. FOUND (the ACTUAL bug, corrected from my own prior wrong claim): all 12 affected scripts share ONE root cause -- each computes its target path by counting a FIXED number of `cd ..`/`.parent` hops from its own location, a number that was correct for the ORIGINAL `bet3/bot/dag//...` (3 path segments between the script and `v3_claude`) but is now WRONG for the fork's `b3ubot/dag//...` (only 2 segments -- no intermediate `bot/` layer). Every one of them OVERSHOOTS by one level, landing at `projects` (the grandparent of `v3_claude`) instead of `v3_claude` itself. Verified empirically (not just by reading the arithmetic) by running `cppcc/tests/wc_bot_dag_cppcc.sh` unmodified: 9 "No such file or directory" errors, "0 total." The 12 files, confirmed by direct read of each: - 10x `/tests/wc_bot_dag_.sh` (bootstrap, cppcc, dagCommands, dagModelCpp, dagModelJs, dagModelPy, dagModelRust, instances, regression, sql) -- IDENTICAL pattern: `HERE=.../tests`, `V3="$(cd "$HERE/../../../../../" && pwd)"` (5x `../`), then a `wc -l` file list hardcoded under `bet3/bot/dag// ...`. BOTH the hop-count (5 -> 4) AND the hardcoded path prefix (`bet3/bot/dag/` -> `b3ubot/dag/`) need fixing in each -- fixing only one leaves the other broken (a 4-hop climb to the right `v3_claude` directory still 404s on a `bet3/bot/dag/...` path that doesn't exist there). - `regression/bin/run_regression` (NOT one of the originally-reported 10 -- found during this step's own research): its `VERSION_HOME="$(cd "$BET3/.." && pwd)"` line (one hop past a variable ALREADY correctly named `BET3` that, in the fork, actually holds `v3_claude`) is the same one-hop overshoot. Left broken, 4 of the 6 regression corpora (`proj_d1H` etc.) would resolve under the wrong root and fail to be found. - `bootstrap/bootstrap/discovery.py`'s `_DEFAULT_VERSION_HOME` (6 `.parent`s) has the identical miscount, but ships a WORKING escape hatch (`os.environ.get("VERSION_HOME")`, checked first) -- so it is a non-issue as long as this step's driver exports `VERSION_HOME` explicitly before invoking anything that reaches `bot_init`. Fixed anyway (2.3): leaving a known-broken fallback as a landmine for a future run that forgets the export is exactly the kind of gap this project's own discipline says to close while already in the code, not defer again. MISSING: none of the 12 fixes exist yet; the toolchain has never been built in this repo; `dag/dags/` (or rather `v3_claude/ dags/`) doesn't exist yet; no regression report has ever been produced for the b3ubot fork. ## (2) What this step delivers 2.1 **Fix the 10 `wc_bot_dag_*.sh` scripts**: `../../../../../ ` (5x) -> `../../../../ ` (4x) AND `bet3/bot/dag/` -> `b3ubot/dag/` in every hardcoded `wc -l` file-list entry, in each of the 10 files. Same mechanical edit, applied 10 times. 2.2 **Fix `regression/bin/run_regression`'s `VERSION_HOME` line**: drop the extra `cd "$BET3/.."` hop -- `VERSION_HOME="$BET3"` directly, since `$BET3` (kept as-is; renaming it is out of scope, a cosmetic change unrelated to the bug) already holds the correct `v3_claude` path in the fork's context. 2.3 **Fix `bootstrap/bootstrap/discovery.py`'s `_DEFAULT_VERSION_HOME`**: 6 `.parent`s -> 5, matching the same one-level correction, as a defensive close of the fallback path (2.2's "why fix it while already here" reasoning). 2.4 **Build the whole toolchain, for real, for the first time**: a small driver script (or documented command sequence -- Q-5-A) exports `CPPCCHOME` (the step.1 convention) and `VERSION_HOME` once, then runs `bash dag/tests/run_tests.sh` -- cascading build-then-test through cppcc -> sql -> dagModelCpp -> dagCommands -> dagModelPy -> dagModelJs -> dagModelRust -> instances -> bootstrap -> regression, self-building every missing artifact along the way (2.1's ground truth). 2.5 **The regression suite green IN-REPO** (end_to_end.md's own done-when clause, verbatim): the byte-identity matrix (24 cells, 4 renderers x 6 corpora), the 2-pass round-trip oracle, and the quirk-landscape check all PASS -- with ZERO renderer- level skips (every toolchain is present, G4). 2.6 **Re-plan U3.9 in end_to_end.md** (the project's own established convention: "split UoWs that turn out >1 day (suffix letters)"): U3.9 closes as DONE for exactly what this step delivers (2.1- 2.5); a NEW row **U3.9a** -- "first b3ubot-kind instance (`execution`) authored + round-tripped" -- is added `todo`, carrying forward the one clause of the original U3.9 text this step deliberately does not touch (see EXPLICITLY OUT OF SCOPE). 2.7 **Closure**: design.md untouched (this step proves existing design true, changes none of it); memory. Retro pair via step_gdiff. EXPLICITLY OUT OF SCOPE (per the operator's own narrower ask -- "fix the bug... and rebuild... including its all tests," not "author a new workflow instance"): - Authoring a b3ubot-specific `execution`-kind DAG instance and round-tripping it -- real NEW content on top of a substrate this step only verifies, not extends. Split into U3.9a (2.6). -> U3.9a - Fixing the ORIGINAL `bet3/bot/dag`'s own copies of these scripts -- a different repo, not touched here, not requested. -> never (out of this repo's scope entirely) - The orchestrator's actual USE of the DAG substrate (design.md §2.13's "RATIFIED -> EXECUTING compiles the skeleton into an execution DAG") -- that's P3's later orchestrator UoWs (U3.1+), which this step's substrate now sits ready for, unused. -> P3 - Any change to `app/`, `adapters/`, `tests/` (the P1 work from steps 1-4) -- this step touches ONLY `dag/`. -> n/a ## (3) Verification gates G1 ALL 12 FIXED FILES VERIFIED INDIVIDUALLY: each of the 10 `wc_bot_dag_*.sh` scripts, run standalone from its actual location, produces a real non-zero `wc -l` total with ZERO "No such file" errors; `run_regression`'s computed `$VERSION_HOME` prints as the real `v3_claude` path (grep/ echo-verified, not assumed); `discovery.py`'s `_DEFAULT_VERSION_HOME` matches `v3_claude` when `$VERSION_HOME` is unset (a direct Python check). G2 `bash dag/tests/run_tests.sh` EXITS 0: the master driver's own pass/fail tally shows fail==0 across all 10 subprojects. G3 ZERO SKIPS, NOT JUST GREEN: given every toolchain is confirmed present (§1), a truly clean run has NO renderer- or subproject-level SKIP lines -- a skip anywhere this run would mean something is silently not being exercised despite having no excuse not to be; investigated and named if any appears, never waved through as "still green." G4 THE REGRESSION REPORT ITSELF, READ: the byte-identity matrix (24/24 cells PASS), the 2-pass round-trip oracle (byte- identical snap1==snap0, snap2==snap1, counts(DB1)== counts(DB2)), and the quirk-landscape check (E1/E2/E3 baseline) all pass -- the report's own text is inspected, not just its exit code. G5 THE ORIGINAL bet3/bot/dag REPO IS UNTOUCHED: this step edits ONLY files under `b3ubot/dag/` -- confirmed via `git status` in the (separate) bet3 repo showing no changes, if reachable. G6 NO REGRESSION ON b3ubot's OWN PRIOR WORK: `make venv && make smoke && make test` (steps 1-4's suite) still passes, unmodified -- this step never touches `app/`/`adapters/`/ `tests/`. G7 Hygiene: b3ubot porcelain clean after each commit; retro via step_gdiff. ## (4) LOCKs C-5-A b3ubot porcelain clean after each commit. C-5-B SCOPE CONFINED TO dag/ (+ this step's own end_to_end.md re-plan, C-5-D): no edit to `app/`, `adapters/`, `tests/`, or `design.md`'s own content this step. C-5-C THE ORIGINAL bet3/bot/dag IS NEVER WRITTEN: every fix in this step targets the FORK (`b3ubot/dag/...`) exclusively; the upstream repo is read-only from this step's perspective, matching the fork-not-mirror decision (d11) that this content is now independently maintained. C-5-D RE-PLANNING IS RECORDED, NOT SILENT: splitting U3.9 into this step's slice + the new U3.9a happens in the SAME commit as the rest of this step's closure, per end_to_end.md §12's own maintenance rule ("re-planning is normal... in the same commit that records the decision"). C-5-E standing locks inherited: B-3 (no disclosure / no push), B-6 (this step's whole verification IS a deterministic T0/T1-class oracle -- no AI-review tier involved). ## (5) Open questions Q-5-A Driver shape: a small, throwaway shell one-liner in the retro (export the two env vars, run the master script), NOT a new committed `scripts/*.sh` file (LEAN -- this is a one-time verification pass over already-existing tooling, not a new piece of b3ubot's own product surface; the EXISTING `dag/tests/run_tests.sh` is the durable, reusable entry point, and it needs no b3ubot-side wrapper) vs adding a `scripts/rebuild_dag.sh` that exports the env vars and calls the master driver (a real, reusable convenience, but invents a new file for something one documented command sequence already covers). LEAN: no new script; document the exact command sequence in the runbook-equivalent (this step's own retro + a brief dag/README.md note if one is warranted at closure). Q-5-B `VERSION_HOME` export scope: export it for this step's manual verification run only (LEAN -- nothing in b3ubot's OWN `Makefile`/`scripts/smoke.sh` currently needs it; the dag/ toolchain is the only consumer, and Q-5-A already leans against adding new committed tooling) vs adding it to a b3ubot-wide env convention now. LEAN: scoped to this step's verification only; revisit if/when U3.9a or later P3 work needs `VERSION_HOME` as a standing b3ubot convention. Q-5-C U3.9a's exact wording in end_to_end.md: carry the original U3.9 text's third clause forward verbatim ("first b3ubot- kind instance (`execution`) authored + round-tripped," LEAN -- it was already precisely scoped, no reason to rewrite it) vs redrafting it now that the substrate is actually proven live (more context available, but risks scope-creeping this step's own closure into planning a DIFFERENT step's content). LEAN: carry forward verbatim, let U3.9a's own future skeleton do any redrafting. ## (6) Acceptance - [x] 10x wc_bot_dag_*.sh fixed (hop count + path prefix), each individually re-run and confirmed producing a real count with zero errors (2.1, G1). - [x] regression/bin/run_regression's VERSION_HOME fixed + verified (2.2, G1). - [x] bootstrap/bootstrap/discovery.py's _DEFAULT_VERSION_HOME fixed + verified (2.3, G1). - [x] Full toolchain built + tested via `bash dag/tests/run_tests.sh`, exit 0, zero skips (2.4, G2, G3). 4 MORE instances of the same bug class found live (not planned -- see step.5.diff.txt findings #1-2) in dagModelRust/Cargo.toml, dagModelJs/loader.mjs, dagModelPy/loader.py, dagModelPy/tests/check_scb_quirks.py -- all fixed the same way. One missing build step found + resolved (the dagCommands cppcc frontend, never triggered by the cascade -- finding #4). - [x] Regression report read directly: byte-identity matrix 24/24, 2-pass round-trip 6/6, quirk check 6/6, all PASS (2.5, G4). - [x] end_to_end.md re-planned: U3.9 -> done (this slice); U3.9a added todo, same commit (2.6, C-5-D). - [x] G5/G6 verified: sibling SCB repos + b3ubot's own make venv/ smoke/test all confirmed clean/unchanged (38 passed/8 skipped, step.4's exact baseline). - [x] Closure: memory; retro pair via step_gdiff (2.7, G7). ## (7) Hash backfill SKELETON commit (this file + ledger row U3.9 -> active): 89104c3 BUILD commit (16 fixes + full toolchain build+test green): edbe4ed RETRO commit (retro pair + U3.9/U3.9a re-plan + ledger done): (HEAD) End of skeleton.