Design docs: the drift PR #58 missed, and a Grid re-walk now that realignment shipped - #59
Merged
Merged
Conversation
…cs with no header at all A pass over all 38 design documents, one day after the PR #58 audit, looking for anything that pass re-affirmed in place without checking. Three claims were wrong, all three in the same direction as PR #58's: the doc claims *less* than the tree has. - `RustRewrite_Plan.md` §4e said "not yet built". It was built on 2026-06-03 (`04728ba`), two days after the plan was drafted. The section's *body* already carried the measured result — 13.9 Mb callable chrY short-read vs 1.1 Mb HiFi, the three private-Y bucket sizes — so the section disagreed with itself, which is how two audits read past it. The sub-heading and the "Missing" list are corrected, and the one genuinely unbuilt piece (the BED is never persisted as a per-alignment artifact) is now stated where a reader will find it. - `project-block-tree.md` §11.4 and `BACKLOG.md` §3.3 both said the private-Y batch has no GUI trigger. The workspace chore table (`maintenance.rs` + `central.rs::maintenance_section`) gave all three CLI-only chores one shared surface in `331e8cb` (PR #47) — which is exactly the "one answer, not a third separate button" §11.4 asked for. Like PR #42 before it, that PR touched no design doc at all; this is now the second confirmed instance of the pattern. - `project-block-tree.md` §11 and `BACKLOG.md` still listed the `project_report` artifact-payload debt. `c0994d0` paid it the day before. BACKLOG §1.2 is re-verified genuinely open — `genotype_panel_for_subject` still has exactly two callers — and now names the chore table as its obvious home. Four documents carried no status header. Three are Scala-era and were missed by PR #58 because they live in `documents/` rather than `documents/design/`; each now says what survived into Rust and where the current design lives. The fourth, `DecodingUsTreeProvider.md`, had no status line of any kind despite being fully implemented with DecodingUs as the default provider and all three of its "Decisions to confirm" resolved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SequryftjcFkLnjQRMKCx3
…ent has shipped The Grid was drafted while realignment was still a plan, and it depended on that module. Realignment merged 2026-08-14, which unblocked the Grid and, in the same stroke, overtook one of its four locked decisions. §11 is the reconnaissance refresh; D1 and §7.3 are corrected in place rather than annotated, because a locked decision the tree has already overruled will otherwise be read as still binding. The design's shape survives intact — staged phases, adaptive replication, digest comparison, the lease state machine. What moved is the cost, and it moved down. - D1 locked minimap2 through the `minimap2-rs` FFI. `navigator-align` ships `minimap2-pure-rs` instead: pure Rust, no C toolchain, 99.74 % byte-identical with zero disagreements at MAPQ > 0. §7.3 built a two-tier fleet on that FFI assumption — macOS+Linux realign, Windows passthrough-only until a spike lands — and none of it is true. Windows nodes take FASTQ units the day P2 opens, `/grid/claim` should filter on RAM/disk/threads rather than OS, and the P4 Windows milestone is gone. - §7.1 asked someone to lift `run_full_analysis_streaming` out of the UI worker. `App::analyze_biosample` already is that: PR #47 built it for batch analysis, headless, cancellable mid-sample, resumable, preflighted before any I/O. - It deliberately omits mtDNA, because that value "is not final on CHM13" — and the digest in §5.2 wants an exact `mt_terminal` match while the Grid realigns *to* CHM13. That is a contradiction, not a wiring gap, and it is the one decision worth settling before P1 code. §11.3 lays out three ways out and recommends the cheapest. - §2 said the AppView pulls ENA study metadata only. `EnaClient::run_files` resolves the run-level filereport with precisely the columns a work unit needs, and `du-jobs/crawl_project.rs` already groups runs by sample and materializes files, splitting BAM/CRAM from the index files — which is D3's passthrough selector. Two traps recorded: SV was in the §7.2 driver at 2–5 h per WGS against ~1 h for the whole rest of the stack, for a signal the digest never compares (removed); and `ena.rs` cannot mirror `refgenome::download`, which sends no `Range` and hashes SHA-256 where ENA publishes md5 on multi-GB files. Still greenfield, re-verified: the `grid` schema (`0075`, not the `0059` guessed), the claim SQL, `Provenance`, subject≠contributor, credit, the validator. The `fed.pds_*` scaffold is still dormant. The critical path is the AppView half. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SequryftjcFkLnjQRMKCx3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documentation only — no code changed, so nothing here alters behavior.
Two threads, in two commits.
1. The drift the last audit missed
PR #58 audited all 32 design docs and corrected five headers. A pass the following
day found three more claims it re-affirmed in place without checking. All three run
the same direction as PR #58's findings — the doc claims less than the tree has.
RustRewrite_Plan.md§4e04728ba, 2026-06-03 — two days after the plan was draftedproject-block-tree.md§11.4 +BACKLOG.md§3.3331e8cb(PR #47)project-block-tree.md§11 +BACKLOG.mdproject_reportdebt unpaidc0994d0, the day beforeThe §4e case is the interesting one: the section's body already carried the
measured result (13.9 Mb callable chrY short-read vs 1.1 Mb HiFi, the three
private-Y bucket sizes). Only the sub-heading and the "Missing" list contradicted it.
The section disagreed with itself, which is how two consecutive audits read past it.
A status marker that sits away from the evidence is the thing that goes stale.
PR #47 is also the second confirmed instance of a substantial PR shipping and
touching no design doc at all — PR #42 was the first, found by the last audit.
Four docs carried no status header. Three are Scala-era and were missed by PR #58
because they live in
documents/, notdocuments/design/. The fourth,DecodingUsTreeProvider.md, had no status line of any kind despite being fullyimplemented with DecodingUs as the default provider.
2. The Grid re-walk
distributed-compute-grid.mdwas drafted while realignment was still a plan, anddepended on it. Realignment merged 2026-08-14 — which unblocked the Grid and
overtook one of its four locked decisions. New §11 is the reconnaissance
refresh; D1 and §7.3 are corrected in place, because a locked decision the tree
has already overruled otherwise reads as still binding.
The design's shape survives intact. The cost moved down:
navigator-alignshipsminimap2-pure-rs— pureRust, no C toolchain, 99.74 % byte-identical, zero disagreements at MAPQ > 0. §7.3
split the fleet by OS on that assumption. Windows nodes can take FASTQ units on day
one;
/grid/claimshould filter on RAM/disk/threads; the P4 Windows milestone is gone.App::analyze_biosamplefromPR Make batch analysis tractable, and fix the Y re-placement bugs it uncovered #47 is already the headless, cancellable, resumable, preflighted per-unit driver.
mt_terminalis a contradiction, not a gap. The digest demands an exact match;analyze_biosampledeclines to assign mtDNA because it "is not final on CHM13" —and the Grid realigns to CHM13. §11.3 gives three ways out and recommends one.
This is the decision to settle before any P1 code.
EnaClient::run_files+du-jobs/crawl_project.rs, already splitting BAM/CRAM fromindex files, which is D3's passthrough selector.
Two traps recorded: SV sat in the §7.2 driver at 2–5 h per WGS (vs ~1 h for the whole
rest of the stack) for a signal the digest never compares; and
ena.rscannot mirrorrefgenome::download, which sends noRangeand hashes SHA-256 where ENA publishesmd5 on multi-GB files.
Net: the critical path is the AppView half —
grid.work_unit, theSKIP LOCKEDclaim, the validator.
Checks
scripts/ste-check.py(no arguments, per the known gotcha): 0 files with violationsfmt/clippy/testare unaffected🤖 Generated with Claude Code
https://claude.ai/code/session_01SequryftjcFkLnjQRMKCx3