Repository navigation
Registry-backed mapping engine and marker-resolution policies (part 6/9) - #20
Open
s28ale-maker wants to merge 4 commits into
Open
s28ale-maker wants to merge 4 commits into
s28ale-maker wants to merge 4 commits into
Conversation
…aluation harness Move curated mapping decisions from Python into mappings/curated_mappings.tsv and mappings/soulcap_entities.tsv, with phenotype.py providing a shared AST for marker validation, token extraction, and scoring. Add soulcap-audit (a self-contained HTML dashboard), soulcap-evaluate (offline matcher retrieval metrics against a reviewed benchmark), and opt-in alias/protein-aware matching via candidate_index.py. Fix mapping_evidence.assess() double- counting bare/unqualified required markers as unscorable gaps, which incorrectly suppressed the required_markers_supported status even when every checkable required marker was directly asserted in CL; also cache per-row CL indexes instead of rebuilding one per matched clause. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…d evidence-reviewed follow-up Add marker_resolution.py + marker_mappings/marker_resolution.tsv: every registry marker token gets an explicit, SHA-256-fingerprinted policy (single_protein/protein_family/complex/reagent_gate/unresolved) with an allow/withhold protein-equivalence call, rationale, and source ref, wired into candidate scoring, mapping evidence, SSSOM export, and the audit via --marker-map. Add soulcap-resolution-audit (resolution_audit.py) to diff legacy vs. policy-driven resolution per marker. Add regression_triage.py, an offline diagnostic that isolates enhanced token-evidence additions, legacy token-evidence removals, and semantic- clause deduplication across 8 combinations without re-running lexical search, to explain matcher ranking changes without altering any mode. Add a hand-reviewed regression follow-up (reports/regression-followup/) covering the three top-5 targets lost under the stricter resolver: cites CL/PRO axioms and OMIP-060 for context, and implements one narrowly-scoped correction (whole_cd8_surface_positive_v1) rather than loosening policy broadly. Add benchmark scaffolding (marker_assertion_benchmark.json as regression tests, benchmark_reserve.json as a documented five-entity prospective reserve) with results reported as provisional retrieval metrics, not validated biological accuracy. No source-sheet values, marker identifiers, or curated mapping decisions changed; legacy matcher mode remains the default. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Break the Fall 2026 (Oct-Dec) working plan into concrete sub-tasks with acceptance criteria, grounded in ROADMAP.md, gaps.tsv, and issues #5-#14, flagging what's blocked on Dr. Diehl/sheet admin/upstream reviewers vs. independently actionable. Linked from CLAUDE.md, README.md, and ROADMAP.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…a/ cache test_reserve_is_reproducible_and_separate_from_development asserted a hash for data/marker_combinations.csv, a gitignored sync-only cache that CI never has (no Sheets credentials to regenerate it) — this broke CI on every Python version while passing locally only because this machine has a synced data/. Match the existing convention elsewhere (evaluation.py's dashboard_evaluation) of treating a missing data/ file as unverifiable rather than a hard failure; tracked mapping files under mappings/ still fail hard if missing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2 of 3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Commits
f1e0772..c6343c1:f1e0772).4dfc726).8a1094e).c6343c1): the benchmark-reserve test no longer hard-fails when the gitignoreddata/cache is absent, as it always is in CI. It's included here so this part passes CI on its own.About the size (~1M added lines): almost all of it is generated JSON and TSV report output, e.g.
regression_triage.json,audit_data.jsonandmatcher_evaluation.json. The code to review is insrc/soulcap_cl_mapping/andtests/.Archived later, no close review needed: part 8 (the tidy-up) moves the experimental outputs added here (
reports/resolver-refinement/,reports/regression-triage/, the regression follow-up, and the matcher evaluation variants) intoarchive/2026-09_resolver_experiments/. You don't need to review those files closely.Test plan
Checked in isolation with the same steps as CI (
uv sync --frozen --extra dev):uv run pytest: 537 passed, 96.5% coverageuv run ruff check .,ruff format --check ., anduv run mypy src/soulcap_cl_mappingpass🤖 Generated with Claude Code