Skip to content

Quarantine reviewed MicrobeDecoder crosswalk contradictions with retained evidence - #1228

Merged
realmarcin merged 1 commit into
masterfrom
fix/microbedecoder-crosswalk-quarantine-1224
Sep 29, 2026
Merged

realmarcin merged 1 commit into
masterfrom
fix/microbedecoder-crosswalk-quarantine-1224

Conversation

@realmarcin

@realmarcin realmarcin commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Implements the user's approved conservative disposition for #1224: quarantine only individually reviewed, authority-contradicted MicrobeDecoder NCBI/GOLD links. Preserve the complete original claim, source record and scientific evidence outside the assertion graph. Do not infer a replacement identity; preserve assembly/IMG and insufficiently supported links.

  • Pin one exact 27,010-record raw CSV and 10,066 approved claims across 5,033 records (5,033 NCBI + 5,033 GOLD).
  • Validate saved independent LPSN/BacDive/native-NCBI/GOLD witnesses, complete lineage declarations/ranks, retired-ID paths, and content-addressed evidence. Evidence gzip has both compressed and decoded hash/length bounds.
  • Preflight the entire immutable consumed CSV before graph output. Match the exact record/field/token/subject/target before retired-ID or GOLD-fold conversion. Every approved rule must dispatch exactly once.
  • Require reversible crosswalk_quarantine.tsv evidence at producer completion, staged finalization, repeat finalization and merge admission. Missing, changed, symlinked or omitted evidence fails closed.
  • Adapt existing hermetic tests with explicit, normally validated fixture policies; no silent test/runtime bypass.

Independent adversarial review

Two reviewers independently checked scientific admission/package evidence and runtime/finalization behavior. Review findings were filed and fixed:

Final independent runtime probes: 20 passed. Existing-test adaptations: 226 passed. Focused consumer/audit/witness tests pass; formatting, lint, spelling and 100% docstring checks pass.

Completed local/source acceptance — not release approval

Exact committed head: 5637c2678dbcd4342c0b927439d755e97eebe54d.

  • All four required local gates passed: poetry run pytest (4,761 passed, 56 skipped), poetry run tox (all environments passed), poetry check --lock, and generated merge-config check. Final gate receipt SHA-256: b534c7450720abe0aa0e43753485567c52db79041acd38c6be4b592f1bd37a20. Candidate HEAD and tracked/untracked cleanliness guards passed; the only explicitly allowed untracked file was the pinned local .venv symlink.
  • After the gates completed, copied only the three existing pinned schema files into the isolated worktree's ignored raw directory and reran test_biolink_hierarchy.py and test_biolink_schema_pin.py: 34 passed, no skips. This is supplemental coverage, not a replacement or inflated total for the full-suite result above.
  • Actual isolated producer/finalizer replay and independent full-row comparison passed: 521,217 old edges minus exactly 10,066 approved claims = 511,151 byte-identical surviving edge rows. All 17,855 retained node rows are unchanged; removed declarations are approved targets, with four still-incident targets explicitly supported by the consumed native NCBI ontology.
  • Strict inventory passed, including all 240 reviewed local-material assertions and unchanged phenotype normalization evidence. The 10,066-row audit retains full original claims and source records and matches both producer-time and finalizer audit receipts.
  • Actual full-source modeling review: zero ERRORs, zero WARNINGs, with house-profile exceptions explicitly reported. Self-loop/subclass-cycle/family checks passed. Strict source-only orphan checking remains non-green: a complete configured-supplier scan leaves the same 18,374 IMG references as the prior bound review (Track and reconcile 696 unexpected anonymous merged nodes with per-ID source evidence #1073), with no newly unresolved IDs. No falsely clean merged/path claim is made.
  • Record 10902 retains its corroborated assembly link, unresolved IMG links and original source-qualified prediction evidence; only reviewed NCBI/GOLD contradictions are quarantined.

Exact-head GitHub CI is green on Python 3.10, 3.11 and 3.12: run 36538573179. The last matrix job completed successfully at 2026-09-29 08:10:57 UTC. Local/source evidence is under data/issue1224-quarantine-20260929.xjuS9H/, including exact-head-review.md, final-gates-t515_ncy/results.json, and source-replay/attempt-61wnnzqd/graph-review/TRIAGE.md.

Separate local production-content helper review found and fixed #1229/#1230/#1231 (archive padding, exact evidence types, mixed primary providers). Final independent and main-agent reruns both pass all 78 tests. Those helpers are not product changes in this PR and have not been used to accept a production candidate.

No production transform output, merged archive or user-modified statistics file was changed. Full production rebuild and merged modeling/path/KGXVal acceptance remain tracked under #650. Remaining unadjudicated crosswalks are not certified correct by this PR.

Fixes #1224.
Fixes #1225.
Fixes #1226.
Related to #650 and #1227.

@realmarcin

Copy link
Copy Markdown
Collaborator Author

Exact-head acceptance for 5637c2678dbcd4342c0b927439d755e97eebe54d:

Evidence root: data/issue1224-quarantine-20260929.xjuS9H/. This is a code/source acceptance, not production or release approval. The full 15-source production rebuild, candidate merge, quarantine evidence companion, CAS reconciliation and actual merged modeling/path/KGXVal reviews remain under #650. Existing production archive and user statistics remain protected.

The user explicitly authorized admin bypass of the required collaborator approval after independent review and green CI. Those prerequisites are now satisfied for this exact head; the planned squash merge will use --match-head-commit to prevent merging a changed head.

@realmarcin
realmarcin merged commit 6c8f360 into master Sep 29, 2026
3 checks passed
@realmarcin
realmarcin deleted the fix/microbedecoder-crosswalk-quarantine-1224 branch September 29, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment