Repository navigation
Give HPRC_R2's gaps and T2T chromosome tables a data type (#765) - #783
Merged
Merged
Conversation
AnVIL_HPRC_R2's gaps BEDs and t2t_chromosomes.tsv files (462 each) had no data_type: their value-map rows were seeded and declared nothing. Both now declare `annotations`, scoped to AnVIL_HPRC_R2 because `gaps` and `sequences` are generic words. HPRC files both under each assembly's annotation/chrom_assignment/ folder, and the HPRC Data Explorer's catalog lists them as "ChromAlias Gaps" and "ChromAlias T2T". The GenomeAnnotationActivity reason now names them among the kinds it covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VF7tbFa5ZTmYWZ6JWKiBBS Co-authored-by: cc-claude-tools <cc-claude-tools@clevercanary.com>
Classified and reconciled at 42c09af. Against run 20261104_183742, only AnVIL_HPRC_R2's data_type moved: annotations 2,772 -> 3,696 and not_classified 3,709 -> 2,785 (924 files). The review queue loses the gaps and sequences rows; the rules report shows each firing 462 times. The other reports change only their run id and commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VF7tbFa5ZTmYWZ6JWKiBBS Co-authored-by: cc-claude-tools <cc-claude-tools@clevercanary.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified for the dataset-scoped annotation mapping changes.
No open issues 🎉
What changed in this PR
This PR fills the missing data types for 924 HPRC_R2 gap and telomere-to-telomere chromosome files through the existing reconciliation translation table, aligning them with their genome annotation lineage steps (#765).
Changes:
- Maps
gapsandsequencestoannotations, scoped to AnVIL_HPRC_R2 and supported by documented evidence. - Expands the genome annotation rationale without changing activity behavior and updates review-queue test expectations.
- Refreshes reports to show 924 additional classified files and two fewer queued values; inference coverage remains unchanged.
| File | Description |
|---|---|
tests/test_value_map.py |
Removes the two authored mappings from seeded-row expectations. |
src/meta_disco/rules/value_map.yaml |
Defines dataset-scoped annotation mappings and their rationale. |
src/meta_disco/rules/activities.yaml |
Updates rationale wording to include gaps and T2T tables. |
docs/unprocessable-report.md |
Refreshes the referenced run. |
docs/rules-report.md |
Documents both mappings and 462 applications each. |
docs/review-queue.html |
Removes resolved values and displays their authored mappings. |
docs/review-queue-report.md |
Updates queue totals and mapping documentation. |
docs/reconcile-report.md |
Records the 924-file improvement and updated provenance. |
docs/coverage-dashboard.html |
Refreshes the run timestamp; inference counts stay unchanged. |
docs/consistency-report.md |
Refreshes the run reference, retaining zero reported violations. |
docs/anvil-coverage-report.md |
Refreshes the classification timestamp. |
Review effort: Balanced
The genome annotation step's reason said every file it covers is written in the sequence's own coordinates; a .t2t_chromosomes.tsv holds one row per sequence instead. Found by the no-context review. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VF7tbFa5ZTmYWZ6JWKiBBS Co-authored-by: cc-claude-tools <cc-claude-tools@clevercanary.com>
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
src/meta_disco/rules/value_map.yaml: the rowsdata_type.gapsanddata_type.sequenceswere seeded and declared nothing. Both now declaredata_type: annotations, apply to AnVIL_HPRC_R2 only, and give a reason citing HPRC's evidence.src/meta_disco/rules/activities.yaml: the genome annotation step's reason now names the gaps and telomere-to-telomere sequence files among the kinds it covers. Wording only.tests/test_value_map.py: the two rows are no longer in the list of rows left unfilled, and the HPRC fixture's review queue drops from 7 values to 5.docs/: regenerated from run20261109_191616, classified and reconciled at42c09af.Why
AnVIL_HPRC_R2 has two submitter tables,
gapsandt2t_sequences, that hold 924 files with no data type: 462*.gaps.bedand 462*.t2t_chromosomes.tsv. They were the only assembly annotation tables without one. Each already has a genome annotation step from its assembly (#682). That step expects anannotationsfile, but a file with no data type is never checked against it.The evidence for
annotations:annotation/chrom_assignment/folder.catalog/build/intermediate/annotations.csv) lists 462 files each under the annotation types "ChromAlias Gaps" and "ChromAlias T2T".HG03453_pat_hprc_r2_v1.0.1.gaps.bedhas 19 intervals on contigs such asHG03453#1#CM088011.1..t2t_chromosomes.tsvis a table of the assembly's telomere-to-telomere sequences. Its columns aresequence_id,chr_name,level,sequence_length,num_gapsandtotal_gap_length; I read one file from AnVIL's public mirror.Assumptions I made
annotations, not a new term. Dave decided this. A T2T list could be read as a quality report on the assembly, but it describes the assembly's own sequences the waycensatandsegdupsdo.annotationsis also the kind the genome annotation step expects, so the step needs no change.sequencestable could hold the sequences themselves. The IGVF rows are limited to their dataset for the same reason. Neither value appears in any other dataset's evidence today.t2t_sequencesstayssequences. Renaming it tot2t_sequenceswould make the raw value say what the table holds, but it would mean re-importing the evidence. Limiting the row to HPRC_R2 was the smaller change.How to verify
Definition of done, from #765:
data_type, and HPRC_R2'sdata_typegap falls by 924 (steps 1 and 2).output kindmisfit forGenomeAnnotationActivity(step 3).make reconcile RUN_DIR=<a run>, or use run20261109_191616. Among AnVIL_HPRC_R2's records, the 462*.gaps.bedfiles (inbed_classifications) and the 462*.t2t_chromosomes.tsvfiles (inremaining_classifications) are allclassifiedasannotations.docs/reconcile-report.mdand find its comparison with run20261104_183742. Itsdata_typerow shows+924filled and-924not classified. The per-dataset table lists only AnVIL_HPRC_R2's data type:annotationsgoes from 2,772 to 3,696, andnot_classifiedfrom 3,709 to 2,785. No other dataset's count of any value moved.output/anvil/20261109_191616/reconciled/reconcile_report.json,lineage.misfitshas no entry whose activity isGenomeAnnotationActivity.docs/review-queue-report.md: the 462'gaps'and 462'sequences'rows are gone. Indocs/rules-report.md,data_type.gapsanddata_type.sequencesshow their reasons and fire 462 times each.docs/anvil-coverage-report.mdcounts inference, which this change doesn't touch.make test-allpasses (3,732 root tests and the schema's 98), as domake lint-all(which includes the typecheck) andmake format-check.make classifyrunsvalidate-metadatafirst, and it passed.Closes #765
🤖 Generated with Claude Code
https://claude.ai/code/session_01VF7tbFa5ZTmYWZ6JWKiBBS
🐦 Built with cc-claude-tools