[NoopDog]

What changed

• src/meta_disco/rules/curator_rules.yaml (new): the first curator rule (contract 1.6, 1.7, 4.7). Its one row, hprc_parent_reads_named_by_sample_table, answers an ANVIL_HPRC samples conflict. It records its author, date, rationale and the two cited GIAB pairs.
• src/meta_disco/curator_rules.py (new): loads and checks the rules. It refuses:
  • a missing or unknown key;
  • a repeated id;
  • a column the slot map doesn't read for samples in that dataset and table (SlotMap.cells now takes an optional slot);
  • a pair identifier the identifier rules don't recognise as written;
  • a name two pairs of one row share;
  • two rows that one file could match.
• src/meta_disco/reconcile_samples.py:
  • answering_rule judges, once per file and from the file's own sources, which rule would answer it. Each table source must be a cell of the rule's own dataset, table and columns. Checking the file's dataset alone is not enough: evidence that names no target dataset joins to files in every dataset.
  • answer gives the answer each time the file settles. It sets the activity aside and settles the rest, so an agreeing parent or PLINK fileset keeps the answer.
  • An answered file gets the agreeing sources' samples, cited by each and then by the rule (source_type: wrangler_annotation). It is counted answered, and it stays in sample_conflicts.ndjson.gz with every source, outcome: answered and answered_by the rule.
• src/meta_disco/reconcile.py: loads the rules, passes them to SampleReport (now a required argument) and records curator_rules_sha256 in the envelope and the report.
• scripts/generate_reconcile_report.py: an answered column and its definition in the samples table, and an answered_by column in docs/sample-conflicts.tsv.
• scripts/generate_rules_report.py: curator rules are listed among the sample rules. Otherwise the rule would show as "undeclared".
• Schema: the description of Attribution.rule_id covers a curator rule; the generated model is regenerated.
• Docs:
  • docs/claims-contract.md's "What is not true yet" now says one curator rule exists, for samples, and 5.6 is updated to match.
  • CLAUDE.md gains a bullet on curator rules.
  • docs/anvil-data-issues.md issue 1 records the effect.
• Tests:
  • tests/test_curator_rules.py (new);
  • run-level tests in tests/test_activity_samples.py;
  • the rules report's row for the rule;
  • curator ids not colliding with any other rule id.

Why

HPRC's sample table has one row per assembled child. The row lists the parents' Illumina reads in mat_ilmn and pat_ilmn, beside maternal_id and paternal_id, and the parents have no row of their own. AnVIL makes one anvil_activity from each row, and its used_biosample_id is the child's biosample for every file in the row, the parents' reads included. So 1,314 parent files had a source naming the child against the table and, for the CRAMs, the read group naming the parent. Each was a conflict with no samples, which left their donors unreached. Examples from anvil15:

File	Listed in row	Column → id	Activity, built from	Activity names
HG01047.final.cram	HG00735	pat_ilmn → paternal_id HG01047	HG00735's row	HG00735
NA24695_CTTGTA_L002_R1_034.fastq.gz	HG005	mat_ilmn → maternal_id HG007	HG005's row	HG005
HG003_HiSeq30x_subsampled_R2.fastq.gz	HG002	pat_ilmn → paternal_id HG003	HG002's row	HG002

The contract lets only a curator rule answer a conflict (4.7), and the conflict stays recorded. The table is supporting evidence, not the decider. The rule answers only where the file's own read group or its name names the same parent:

• 94 CRAMs: the read group agrees on every one.
• 1,212 GIAB FASTQs: named NA24694_* and NA24695_*. Those Coriell ids are GIAB's HG006 and HG007, per NCBI BioSample SAMN21924297 ("NIST HG006", cell line "NA24694 at Coriell", male) and SAMN21924298 ("NIST HG007", "NA24695 at Coriell", female).
• 4 FASTQs: named HG003_* and HG004_*.

A file name is a check, never a source: the name never enters the samples or their named_by, and only decides whether the rule applies.

Assumptions I made

• Dave chose a curator rule over dropping the activity source. Dropping the activity would erase the conflict rather than answer it. It would also settle NA21309's parents' four barcode-named FASTQs from the table alone.
• The rule overrides any activity in those columns, not only one naming the row's child. The evidence lines don't carry the activity's source row, so "names the row's child" can't be checked at reconcile. I measured it instead: every one of the 1,314 files is in one parent column of one row, generated by the one activity made from that row. Dave decided to leave this as a stated limit.
• An activity in another namespace (incomparable) is not answered. No file has that shape, and the rule is scoped to conflicts. Dave decided this.
• Known limit: a derived file carrying its own wrong activity stays unanswered. Take a .crai whose only own source is an activity naming the child, under an answered CRAM. Its own sources don't conflict, so no rule is judged for it. It then takes the parent from the CRAM, conflicts with its activity, and stays a conflict. No file in the run has this shape. Answering it would mean treating a parent's inherited samples as the table evidence the rule requires, which widens the rule. Dave decided to leave it.
• "Confirmed by the file's own content" means any header source, such as a VCF's sample columns, not only a read group. For these files it is the CRAMs' read group.
• The GIAB pairs live in the rule, for its name check alone. They are a local stand-in for the crosswalk in Sample to donor: donor_of edges from the Azul manifest and public registries, pedigree as child_of #361. No record, sample list or identifier rule carries them, so giab:HG007 and coriell:NA24695 stay apart everywhere else.
• The rule's meaning is split between the YAML and the code. The row gives the dataset, table, columns and pairs. The code fixes the rest: the rule overrules the activity, and a read group or name must confirm. General curator rules are Curator rules: resolve a conflict by picking, adding, declining, or recording unknown #397's.
• Left as conflicts:
  • NA21309's parents' four FASTQs, named by a barcode alone (AGAACGAG-*, CTTGTCGA-*), and their four checksums, per Dave;
  • the two AnVIL_HPRC_R2 HiFi BAMs whose read groups name HG02451, already recorded as data issue 2;
  • HG005.paternal.GRCh38_no_alt.bam (incomparable), which is data issue 10.
• Follow-up, not done here: a curator rule is not re-checked when the source changes (Curator rules: resolve a conflict by picking, adding, declining, or recording unknown #397's re-validation).

How to verify

Definition of done, from #777:

• Each pattern-1 file names the parent the sample table names (steps 2 and 3).
• A new run's sample_conflicts.ndjson.gz no longer lists the pattern-1 files as unanswered conflicts. They are listed answered (step 1). Per Dave's ground rule, this covers 1,310 files: the 4 barcode-named FASTQs stay conflicts.
• The two HiFi BAMs stay listed as conflicts, with the finding recorded (step 4).
• A test covers the rule (step 6).

1. Count the outcomes in the PR-head run's reconciled/sample_conflicts.ndjson.gz: run output/anvil/20261111_091236, classified and reconciled at d7861b3 with a clean tree, whose reports are committed in a1f5561. ANVIL_HPRC should show 1,310 answered, each answered_by hprc_parent_reads_named_by_sample_table, plus 4 conflict and 1 incomparable. AnVIL_HPRC_R2 should show 2 conflict. The previous run, 20261109_222451, had 1,314 + 2 conflicts and 1 incomparable.
2. In the run's reconciled/bam_classifications.ndjson.gz, find ANVIL_HPRC's HG01047.final.cram (file_id abfd3a7d-9960-321b-85af-3100bad08d54; ANVIL_1000G_high_coverage_2019 has a file of the same name, whose sources all agree). Its samples.list_id points to a list in sample_lists.ndjson.gz whose one member is coriell:HG01047. Its named_by lists the read group, the sample.paternal_id cell and the curator rule, and not the activity.
3. In reconciled/fastq_classifications.ndjson.gz, NA24694_GCCAAT_L002_R2_021.fastq.gz's list has one member, giab:HG006, named by the sample.paternal_id cell and the curator rule.
4. In docs/sample-conflicts.tsv, the two m54329U_*.5mc.hifi_reads.bam rows have outcome conflict and a blank answered_by. docs/anvil-data-issues.md issue 2 records the finding.
5. In docs/reconcile-report.md, the samples table shows ANVIL_HPRC with 1,310 answered and 4 conflicts. Its "What no route ties" table shows ANVIL_HPRC's unreached donors falling from 108 to 10. The run adds three sample records, giab:HG004, giab:HG006 and giab:HG007; no dimension's count moved against 20261109_222451.
6. make test-all passes, as do make lint-all (which includes the typecheck) and make format-check. The rule's tests are tests/test_curator_rules.py and the answered tests in tests/test_activity_samples.py.

Closes #777

🤖 Generated with Claude Code

https://claude.ai/code/session_01VF7tbFa5ZTmYWZ6JWKiBBS
🐦 Built with cc-claude-tools

[Copilot]

🟡 Changes recommended

Filename-only confirmation can incorrectly assign every member of a multi-valued cell; it must require a singleton unless content confirms the complete set.

1 open issue

• Prevent filename-only confirmation from assigning every cell member · New

What changed in this PR

Adds the first curator rule to Meta-disco’s reconciliation pipeline, resolving eligible HPRC parent-read sample conflicts while preserving competing evidence for review.

Changes:

• Introduces validated curator rules with filename or content confirmation.
• Applies answers during sample reconciliation and records rule attribution and provenance.
• Updates reports, schema descriptions, documentation, and tests for answered conflicts.

File	Description
tests/​test_rules_report.py	Checks curator rules appear as declared sample rules.
tests/​test_reconcile_samples.py	Updates fixtures for required curator-rule input.
tests/​test_reconcile_report.py	Adjusts expected sample outcome columns.
tests/​test_curator_rules.py	Tests rule validation, confirmation, and exclusions.
tests/​test_code_rules.py	Checks curator rule IDs for collisions.
tests/​test_activity_samples.py	Tests answers, propagation, and retained diagnostics.
src/​meta_disco/​schema/​classification.yaml	Documents curator-rule attribution.
src/​meta_disco/​schema/​classification_model.py	Regenerates attribution descriptions.
src/​meta_disco/​rules/​curator_rules.yaml	Declares the HPRC parent-read rule and supporting citations.
src/​meta_disco/​reconcile.py	Loads curator rules and records their digest.
src/​meta_disco/​reconcile_samples.py	Applies answers and retains competing sample sources.
src/​meta_disco/​curator_rules.py	Defines and validates curator rules and filename checks.
scripts/​generate_rules_report.py	Includes curator rules in the rule inventory.
scripts/​generate_reconcile_report.py	Reports answered outcomes and rule IDs.
docs/​claims-contract.md	Documents the limited curator-rule implementation.
docs/​anvil-data-issues.md	Describes resolved HPRC cases and remaining conflicts.
CLAUDE.md	Documents curator-rule behavior and attribution.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The override changes sample and donor identity, requiring validation of the claimed 1,310 answered files, retained exceptions, and donor reachability against the anvil15 corpus beyond source inspection alone.

No open issues 🎉

1 resolved since last review

• Prevent filename-only confirmation from assigning every cell member

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No active review issues are reported for the HPRC parent-read curator-rule change.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No active review issues remain for the curator-rule sample-reconciliation changes.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No active review issues remain for the curator-rule reconciliation changes.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

The HPRC override can be incorrectly authorized by another dataset’s sample.maternal_id observation because source identity is not validated.

1 open issue

• Restrict HPRC override to observations from the intended source · New

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No active review issues block the curator-rule changes for resolving HPRC parent-read sample conflicts while preserving their audit history.

No open issues 🎉

1 resolved since last review

• Restrict HPRC override to observations from the intended source

_{Review effort: Balanced}