PubMed search CLI, Fall plan, cell_type_reviews scaffold (part 7/9)

CLAUDE.md

@@ -61,7 +61,8 @@ AST in `marker_syntax.py`, compiled/evaluated by `phenotype.py`.
 | [Notes.md](Notes.md) | Scratch notes on data sources + a marker-syntax quick reference. |
 | [MARKER_SYNTAX.md](MARKER_SYNTAX.md) | **Canonical spec** of the marker expression language (human-readable guide + EBNF). Cite this for anything parsing/validating marker strings. |
 | [reports/](reports/) | Analysis reports. `marker_string_issues.md` is the curated review; `marker_validation.md` is **auto-generated** by the EBNF validator on each sync. `cl_pro_relationships.md`/`.tsv` are **auto-generated** by `soulcap-cl-pro` from Ubergraph. `cl_term_issues.md` tracks proposed CL corrections found during mapping work. `candidate_cl_mappings.md` is the Milestone 4 deliverable — proposed SOULCAP→CL mappings with rationale and evidence. `candidate_cl_mappings_batch.tsv`/`candidate_cl_mappings_agreement.tsv` are **auto-generated** by `soulcap-match --batch [--lexical]` (issue #6) — a marker-axiom-only pass and a marker/lexical agreement pass across every SOULCAP cell type; treat as an unreviewed draft shortlist (conflicts must be checked), not curated output. `candidate_cl_mappings.sssom.tsv` is **auto-generated** by `soulcap-sssom` from the curated `CURATED_MAPPINGS` table in `sssom_export.py` (kept in sync by hand with `candidate_cl_mappings.md`) — standard SSSOM format, with `confidence`/`comment` derived from whether the match is backed by a directly-asserted CL marker axiom, an inferred-only one, or none at all (lexical/name match only). `gaps.tsv` is a curated, hand-maintained log of cases where SOULCAP has no reasonable CL match, CL conflicts with a marker panel, CL has an axiom gap, or the sheet itself has a data problem — separate from the mapping table; matching GitHub issues use the `gap` label (`.github/ISSUE_TEMPLATE/mapping_gap.yml`). `pro_marker_species_support.tsv` is the issue #11 deliverable — for each (CL cell type, general PRO marker) pair SOULCAP's mappings actually touch, whether literature supports the marker as human, mouse, or both, or `insufficient_evidence`; built by the `pro-marker-species-support` skill, every quote verified verbatim against its source before writing. `fall_2026_sprint_backlog.md` is the sprint-level breakdown of [ROADMAP.md](ROADMAP.md)'s Fall 2026 (Oct–Dec) working plan, prepared for Dr. Diehl — hand-maintained, not regenerable; update it as items complete or scope is renegotiated, don't treat it as a generated report. New reports go here. `reports/citation_traversal/` holds gitignored, regenerable snippet caches + summaries from the `citation-traversal` skill. |
-| `src/soulcap_cl_mapping/` | All Python code. `sync_sheets.py` → `soulcap-sync`; `marker_syntax.py` → `soulcap-validate` (EBNF validator); `snippet_cache.py` → `soulcap-cache`; `report_validator.py` → `soulcap-validate-report`; `cl_pro.py` → `soulcap-cl-pro` (CL→PR relationships via Ubergraph SPARQL); `cl_match.py` → `soulcap-match` (marker-axiom + lexical CL candidate scoring, single-profile or `--batch`); `oak_match.py` → `soulcap-oak-match` (OAK sqlite-backed lexical/synonym CL search — downloads/caches a local CL database on first use, ~100MB); `sssom_export.py` → `soulcap-sssom` (curated mappings → SSSOM TSV, with confidence derived from CL marker-axiom assertion status); `pro_species_support.py` (scopes CL PRO markers needing species-specificity literature support — no CLI, called as a library from the `pro-marker-species-support` skill); `europepmc_search.py` → `soulcap-europepmc` (free, keyless Europe PMC search — fallback literature source when the Asta MCP server isn't reachable). |
+| [cell_type_reviews/](cell_type_reviews/README.md) | **One review file per SOULCAP cell type** (tracked, curated — not a regenerable cache): definition, marker→protein, verbatim literature support, CL comparison, proposed verdict, uncertainty flags. Start from `_template.md` and keep the folder README's index in sync. Only reviews flagged **uncertain** go to Dr. Diehl for expert review. Scaffolded 2026-09-23, no reviews written yet; its relationship to the existing `reports/literature/` Milestone 2 output is still being confirmed with Dr. Diehl — see ROADMAP.md before assuming one supersedes the other. |
+| `src/soulcap_cl_mapping/` | All Python code. `sync_sheets.py` → `soulcap-sync`; `marker_syntax.py` → `soulcap-validate` (EBNF validator); `snippet_cache.py` → `soulcap-cache`; `report_validator.py` → `soulcap-validate-report`; `cl_pro.py` → `soulcap-cl-pro` (CL→PR relationships via Ubergraph SPARQL); `cl_match.py` → `soulcap-match` (marker-axiom + lexical CL candidate scoring, single-profile or `--batch`); `oak_match.py` → `soulcap-oak-match` (OAK sqlite-backed lexical/synonym CL search — downloads/caches a local CL database on first use, ~100MB); `sssom_export.py` → `soulcap-sssom` (curated mappings → SSSOM TSV, with confidence derived from CL marker-axiom assertion status); `pro_species_support.py` (scopes CL PRO markers needing species-specificity literature support — no CLI, called as a library from the `pro-marker-species-support` skill); `europepmc_search.py` → `soulcap-europepmc` (free, keyless Europe PMC search — fallback literature source when the Asta MCP server isn't reachable); `pubmed_search.py` → `soulcap-pubmed` (NCBI E-utilities search — reads `PUBMED_API_KEY` from `.env` if set for the higher rate limit, otherwise works keyless; use when a task specifically needs NCBI's own index/query syntax rather than Europe PMC's mirror of it). |
 | `tests/` | Unit tests (mirror `src/` layout). |
 | `.claude/skills/` | Project skills — see [citation-traversal](.claude/skills/citation-traversal/SKILL.md), `ontology-term-lookup`, [pro-marker-species-support](.claude/skills/pro-marker-species-support/SKILL.md), [gap-issue-filing](.claude/skills/gap-issue-filing/SKILL.md), [roadmap-status-sync](.claude/skills/roadmap-status-sync/SKILL.md), and [mapping-audit](.claude/skills/mapping-audit/SKILL.md). |
 | `.claude/hooks/` | Claude Code hooks. `validate_report_quotes.py` is a PreToolUse guard that blocks writing a citation-traversal report whose quotes aren't verbatim in the snippet cache. |
@@ -74,6 +75,12 @@ When you create a new artifact, put it in the established location and link it
 from here and the README so locations stay consistent. Do not scatter
 documents at the repo root.
 
+The September 21 Fall 2026 planning draft for Dr. Diehl is
+`reports/fall_2026_semester_plan.md`, with matching `.pdf` and `.html` versions.
+It summarizes recent work and proposes weekly deliverables and agent task
+instructions. It does not replace the existing sprint backlog or establish
+new scientific approvals or collaborator commitments without agreement.
+
 ## Conventions & requirements
 
 - **All code lives under `src/`** (package `soulcap_cl_mapping`).

README.md

@@ -37,6 +37,13 @@ semester's sprint-level breakdown (sub-tasks, acceptance criteria, and what's
 blocked on someone outside the repo) is
 [reports/fall_2026_sprint_backlog.md](reports/fall_2026_sprint_backlog.md).
 
+The September 21 discussion draft for Dr. Diehl expands this into a
+plain-language October–December plan, recent-work summary, weekly schedule,
+and agent task instructions: [PDF](reports/fall_2026_semester_plan.pdf),
+[editable Markdown](reports/fall_2026_semester_plan.md), or
+[browser version](reports/fall_2026_semester_plan.html).
+Its workload and review targets are proposed for approval, not confirmed commitments.
+
 ## Setup
 
 ### 1. Clone this repo
@@ -84,6 +91,21 @@ startup and expands `${ASTA_API_KEY}` into the `x-api-key` header in
 it is the only place the secret lives. Restart Claude Code after editing it so
 the key is picked up.
 
+### Optional: PubMed (NCBI E-utilities)
+
+`soulcap-pubmed` (`src/soulcap_cl_mapping/pubmed_search.py`) works keyless,
+same as `soulcap-europepmc`. An NCBI API key just raises the rate limit from
+3 to 10 requests/second. If you want one, generate it from your
+[NCBI account settings](https://www.ncbi.nlm.nih.gov/account/settings/) under
+**API Key Management**, then add it to the **local, gitignored** `.env`:
+
+```
+PUBMED_API_KEY={token}
+```
+
+Unlike `ASTA_API_KEY`, this is read directly by Python (via `python-dotenv`),
+not through Claude Code's MCP `env` substitution — it isn't an MCP server key.
+
 ## Unified audit dashboard
 
 Generate an offline dashboard from the local data and mapping registries:
@@ -420,6 +442,19 @@ own candidate mappings — those stay as SSSOM, a proposal for human review,
 not something this repo should unilaterally convert into OWL axioms and merge
 into CL as if already accepted.
 
+## Cell-type reviews
+
+[`cell_type_reviews/`](cell_type_reviews/README.md) holds one review file per
+SOULCAP cell type. Each file gathers the marker definition, marker → protein/gene
+mappings, verbatim literature support, a comparison against Cell Ontology terms
+and their marker axioms, and a proposed CL mapping. Reviews that meet an
+uncertainty trigger (weak literature, a conflict with CL axioms, no exact CL term,
+etc.) are sent for expert review; confident mappings go straight into the Google
+Sheet. See the folder README for the workflow, statuses, and template. Scaffolded
+2026-09-23 — no reviews written yet, and how it relates to the existing
+`reports/literature/` Milestone 2 reports is still being confirmed with Dr. Diehl
+(see [ROADMAP.md](ROADMAP.md)).
+
 ## Skills & literature workflows
 
 This repo ships Claude Code skills under `.claude/skills/`:

cell_type_reviews/README.md

@@ -0,0 +1,80 @@
+# Cell-type reviews
+
+One review file per SOULCAP cell type (one per `Subset name` row in the
+`Marker Combinations` sheet). Each file collects the evidence for that cell
+type's marker definition and CL mapping, and proposes a verdict:
+
+1. SOULCAP definition (markers, current mapping)
+2. Markers → protein / gene (Milestone 1)
+3. Literature support: verbatim, cited quotes (Milestone 2)
+4. Comparison with Cell Ontology terms and their PR marker axioms
+5. Proposed CL term, match type, and rationale (Milestones 3–4)
+6. Uncertainty flags
+7. Reviewer notes
+
+These files are **tracked, curated artifacts**. They are not a regenerable cache.
+The Google Sheet stays the source of truth: a verdict only takes effect once
+it has been entered in the sheet and re-synced.
+
+## Workflow
+
+Start a review by copying [`_template.md`](_template.md) to `<slug>.md`. The slug
+is the `Subset name` in lowercase with hyphens, e.g. `MAIT` → `mait.md`,
+`CD4 Con T` → `cd4-con-t.md`. Then add a row to the [index](#index).
+
+```
+draft ──▶ evidence-complete ──▶ verdict-proposed ──┬──▶ synced-to-sheet            (confident)
+                                                   └──▶ needs-expert-review
+                                                          ──▶ expert-reviewed ──▶ synced-to-sheet
+```
+
+| Status | Meaning |
+|--------|---------|
+| `draft` | File created; sections still being filled in. |
+| `evidence-complete` | Sections 1–4 done. Every marker has a literature verdict, even if that verdict is "no evidence found". |
+| `verdict-proposed` | Section 5 filled in and section 6 assessed. |
+| `needs-expert-review` | At least one uncertainty trigger applies. Waiting for the expert (Dr. Diehl). |
+| `expert-reviewed` | Expert decision recorded in section 7; `reviewer` and `reviewed_on` set. |
+| `synced-to-sheet` | Verdict entered in the Google Sheet (`OLS CL identifier`, `Type of Match`, `CL Mapping Notes`) and confirmed after `soulcap-sync`. |
+
+**Expert review happens only for uncertain cell types.** Confident mappings go
+straight from `verdict-proposed` to `synced-to-sheet`. The expert's queue is
+every row in the index with status `needs-expert-review`.
+
+## When a review is uncertain
+
+Set `uncertain: true` and move the review to `needs-expert-review` if **any** of
+these apply. List the matching trigger IDs in `uncertainty_triggers`:
+
+| Trigger ID | Applies when |
+|------------|--------------|
+| `weak-literature` | A **required** marker has no supporting quote, or is supported only indirectly (e.g. by a different tissue, species, or population). |
+| `conflicting-literature` | Sources disagree about a marker for this cell type. |
+| `cl-axiom-conflict` | The candidate CL term's PR marker axioms contradict the SOULCAP markers (e.g. CL says `CD27+`, SOULCAP says `CD27-`). |
+| `no-exact-cl-term` | Only a broad CL match exists. The cell type may be missing from CL (a candidate new-term request). |
+| `multiple-cl-candidates` | More than one CL term fits, and the markers don't decide between them. |
+| `ambiguous-definition` | The marker string is invalid or ambiguous (see `reports/marker_validation.md`), or it contains free text or placeholders. |
+| `mapping-change` | The proposed verdict would change an existing `OLS CL identifier` or `Type of Match` in the sheet. |
+
+If none apply, write "None" in section 6 and continue without expert review.
+
+## Evidence rules
+
+- Quotes must be **verbatim** from the snippet cache or retrieved full text, and
+  must cite a DOI or PMID and the citation-traversal run ID. Use the
+  [`citation-traversal`](../.claude/skills/citation-traversal/SKILL.md) skill to
+  gather them. Check quotes with `uv run soulcap-validate-report` before setting
+  `evidence-complete`.
+- Record "no evidence found" explicitly. Never invent support.
+- Look up CL terms with OLS4 or the `ontology-term-lookup` skill. Take CL marker
+  axioms from [`reports/cl_pro_relationships.tsv`](../reports/cl_pro_relationships.tsv).
+- Report data errors found during a review in the review and in
+  `reports/marker_string_issues.md`. Fix them in the sheet, not in `data/`.
+
+## Index
+
+Keep this table in sync with each file's front matter.
+
+| Cell type | File | Status | Uncertain | Proposed CL | Match |
+|-----------|------|--------|-----------|-------------|-------|
+| _none yet_ | | | | | |

cell_type_reviews/_template.md

@@ -0,0 +1,85 @@
+---
+# Copy this file to <slug>.md (lowercase Subset name, hyphens, e.g. `cd4-con-t.md`).
+subset_name: ""            # exactly as in the sheet's `Subset name` column
+sheet_row: null            # 1-based row in `Marker Combinations` at time of review
+sheet_synced: ""           # date of the `soulcap-sync` snapshot used (YYYY-MM-DD)
+status: draft              # draft | evidence-complete | verdict-proposed | needs-expert-review | expert-reviewed | synced-to-sheet
+uncertain: false           # true if any uncertainty trigger applies (see README)
+uncertainty_triggers: []   # e.g. [weak-literature, cl-axiom-conflict, no-exact-cl-term]
+proposed_cl_id: ""         # CL:NNNNNNN
+proposed_cl_label: ""
+match_type: ""             # exact | broad | none
+reviewer: ""               # set only when status is expert-reviewed
+reviewed_on: ""            # YYYY-MM-DD
+---
+
+# <Subset name>
+
+## 1. SOULCAP definition
+
+Copied from the synced `Marker Combinations` row. Don't edit these here. If a
+value is wrong, fix it in the Google Sheet and re-sync.
+
+| Column | Value |
+|--------|-------|
+| Required exclusion | `` |
+| Ideal exclusion | `` |
+| Required phenotypic markers | `` |
+| Ideal phenotypic markers | `` |
+| Current `Type of Match` | |
+| Current `OLS CL identifier` | |
+| Current `CL Mapping Notes` | |
+
+Marker-string problems (from `reports/marker_validation.md` /
+`reports/marker_string_issues.md`): _none_ / list them.
+
+## 2. Markers → protein / gene
+
+One row per distinct marker in the definition. Once Milestone 1 exists, take
+these from `marker_mappings/marker_protein_gene.csv`.
+
+| Marker | Sense | PR ID | UniProt (human) | Gene |
+|--------|-------|-------|-----------------|------|
+| | + / − / lo / hi | | | |
+
+## 3. Literature support
+
+Every quote must be **verbatim** from a cached snippet or retrieved full text,
+with its source and citation-traversal run ID. Never paraphrase something into a
+quote. Record "no evidence found" explicitly rather than leaving it blank.
+
+Seed papers (from `Citation Mgr`): 
+
+| Marker / combination | Verdict | Quote | Source | Run |
+|----------------------|---------|-------|--------|-----|
+| | supported / contradicted / no evidence found | "…" | DOI / PMID | `reports/citation_traversal/<run_id>` |
+
+## 4. Cell Ontology comparison
+
+Candidate CL term(s) (checked via OLS4 / `ontology-term-lookup`), plus their
+marker axioms from `reports/cl_pro_relationships.tsv`.
+
+| CL term | Label | CL marker axioms | Agreement with SOULCAP markers |
+|---------|-------|------------------|--------------------------------|
+| CL: | | | agrees / conflicts / not in CL |
+
+Gaps or problems in CL itself (missing term, wrong or missing axiom): _none_ / describe.
+
+## 5. Proposed verdict
+
+- **Proposed CL term:** CL:NNNNNNN (label)
+- **Match type:** exact / broad / none
+- **Rationale:** 
+- **Proposed text for `CL Mapping Notes`:** 
+- **Proposed changes upstream:** SOULCAP sheet / CL new-term or edit request / none
+
+## 6. Uncertainty
+
+List each trigger that applies (see [README](README.md#when-a-review-is-uncertain))
+and what an expert would need to decide. If none applies, write "None: confident
+mapping, no expert review needed."
+
+## 7. Reviewer notes
+
+_Filled in by the expert reviewer, and only for uncertain reviews._ Record the
+decision, any correction, and the reason.

pyproject.toml

@@ -39,6 +39,7 @@ soulcap-map = "soulcap_cl_mapping.marker_map:main"
 soulcap-oak-match = "soulcap_cl_mapping.oak_match:main"
 soulcap-sssom = "soulcap_cl_mapping.sssom_export:main"
 soulcap-europepmc = "soulcap_cl_mapping.europepmc_search:main"
+soulcap-pubmed = "soulcap_cl_mapping.pubmed_search:main"
 
 [project.optional-dependencies]
 dev = [