Give HPRC_R2's gaps and T2T chromosome tables a data type (#765)

docs/anvil-coverage-report.md

@@ -1,6 +1,6 @@
 # AnVIL Classification Coverage Report
 
-Classification run: **2026-11-04 18:37:42**
+Classification run: **2026-11-09 19:16:16**
 
 Source: **708,088** files across **12** open-access datasets on [explore.anvilproject.org](https://explore.anvilproject.org/).
 Processed **708,088** files.

docs/consistency-report.md

@@ -1,6 +1,6 @@
 # Self-Consistency Report
 
-Run: **output/anvil/20261104_183742** — 708,088 records, 10 rules  
+Run: **output/anvil/20261109_191616** — 708,088 records, 10 rules  
 **Total violations: 0**
 
 Cross-field invariants over classified records (#314). *Active* is how many records a rule tested; a rule with 0 active is **vacuous** (no matching data in this run), not verified-clean.

docs/reconcile-report.md

@@ -1,12 +1,12 @@
 # Reconciliation report
 
-Generated by `make reconcile-report` from `output/anvil/20261104_183742/reconciled/reconcile_report.json` (#395).
+Generated by `make reconcile-report` from `output/anvil/20261109_191616/reconciled/reconcile_report.json` (#395).
 
-- **Run:** `20261104_183742`, 708,088 files
+- **Run:** `20261109_191616`, 708,088 files
 - **Input:** `data/anvil/prod/anvil_files_metadata.json` (repository `anvil`, catalog `anvil15`)
-- **Classified by:** commit 49cc8bf7550c, version 0.1.0
-- **Reconciled by:** commit c08d748d67c2, version 0.1.0
-- **Translation table sha256:** `ab1743df143cf8584d5cde5e10539ca2d97cb641d96cb65402b841014f1a5c59`
+- **Classified by:** commit 42c09af7e94b, version 0.1.0
+- **Reconciled by:** commit 42c09af7e94b, version 0.1.0
+- **Translation table sha256:** `263f712c3e06c75ceef46f678b2ae8e1191885369dfc28e5f51940a7eb640f79`
 
 ## How each slot settled
 
@@ -22,27 +22,27 @@ The last two columns are extra counts laid over those, not part of the sum:
 | dimension | published | published harmonized | submitter | submitter harmonized | external | external harmonized | inference | inherited | conflict (inference) | conflict (sources) | conflict (published) | published unreviewed | not applicable | not classified | added over published | filled over inference |
 | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
 | data_modality | 0 | 6,743 | 0 | 36,747 | 0 | 6,962 | 341,811 | 270,005 | 0 | 0 | 0 | 0 | 23,822 | 21,998 | 655,513 | 36,558 |
-| data_type | 0 | 0 | 464 | 9,796 | 0 | 0 | 678,577 | 0 | 0 | 2 | 0 | 0 | 0 | 19,249 | 688,837 | 3,404 |
+| data_type | 0 | 0 | 464 | 10,720 | 0 | 0 | 678,577 | 0 | 0 | 2 | 0 | 0 | 0 | 18,325 | 689,761 | 4,328 |
 | platform | 0 | 0 | 9,830 | 16,305 | 6,962 | 0 | 27,309 | 584,978 | 0 | 0 | 0 | 0 | 31,631 | 31,073 | 645,384 | 12,571 |
 | reference_assembly | 0 | 4,476 | 52,506 | 263,942 | 0 | 0 | 124,417 | 176,697 | 8 | 0 | 0 | 220 | 68,099 | 17,723 | 617,562 | 218,508 |
 | assay_type | 0 | 6,753 | 33,547 | 2,726 | 6,962 | 0 | 5,441 | 589,286 | 0 | 6 | 2 | 0 | 31,625 | 31,740 | 637,962 | 49,574 |
 | instrument_model | 0 | 0 | 19,102 | 520 | 6,962 | 0 | 12,623 | 587,881 | 0 | 0 | 0 | 0 | 31,631 | 49,369 | 627,088 | 25,901 |
 | variant_kind | 0 | 0 | 0 | 0 | 0 | 0 | 293,383 | 231,247 | 0 | 0 | 0 | 0 | 150,689 | 32,769 | 524,630 | 0 |
 | variant_origin | 0 | 0 | 0 | 0 | 0 | 0 | 293,395 | 234,823 | 0 | 0 | 0 | 0 | 150,689 | 29,181 | 528,218 | 0 |
-| **all dimensions** | **0** | **17,972** | **115,449** | **330,036** | **20,886** | **6,962** | **1,776,956** | **2,674,917** | **8** | **8** | **2** | **220** | **488,186** | **233,102** | **4,925,194** | **346,516** |
-| **% of slots** | **0.0%** | **0.3%** | **2.0%** | **5.8%** | **0.3%** | **0.1%** | **31.3%** | **47.2%** | **<0.1%** | **<0.1%** | **<0.1%** | **<0.1%** | **8.6%** | **4.1%** | **86.9%** | **6.1%** |
+| **all dimensions** | **0** | **17,972** | **115,449** | **330,960** | **20,886** | **6,962** | **1,776,956** | **2,674,917** | **8** | **8** | **2** | **220** | **488,186** | **232,178** | **4,926,118** | **347,440** |
+| **% of slots** | **0.0%** | **0.3%** | **2.0%** | **5.8%** | **0.3%** | **0.1%** | **31.3%** | **47.2%** | **<0.1%** | **<0.1%** | **<0.1%** | **<0.1%** | **8.6%** | **4.0%** | **86.9%** | **6.1%** |
 
 **How complete the metadata is.** A slot is one file in one dimension. It is *filled* when it settled with a value or as not applicable, which is an answer (the dimension does not apply to the file), unlike not classified. A filled slot is counted once, under the input its answer is credited to: *original*, the catalog's published values; *submitter tables*; *external*, ENA's run records (each source's row adds its own column above and its *harmonized* one); *inherited*, where no source declared the value, the file's parents across its `generated_by` did, and inference declared another or none; *inferred*, every other value no source declared, which is credited to inference; *not applicable*, which reconcile does not yet credit to an input (#634). A slot not classified, published unreviewed or in conflict is not filled. Each % is its row's count over the *slots* row of its column.
 
 |  | meta-disco (8 dimensions) | % | catalog today (2 dimensions) | % |
 | --- | ---: | ---: | ---: | ---: |
 | original (catalog's published values) | 17,972 | 0.3% | 11,451 | 0.8% |
-| submitter tables | 445,485 | 7.8% | — | — |
+| submitter tables | 446,409 | 7.8% | — | — |
 | external (ENA) | 27,848 | 0.4% | — | — |
 | inferred | 1,776,956 | 31.3% | — | — |
 | inherited | 2,674,917 | 47.2% | — | — |
 | not applicable | 488,186 | 8.6% | — | — |
-| **filled** | **5,431,364** | **95.8%** | **11,451** | **0.8%** |
+| **filled** | **5,432,288** | **95.8%** | **11,451** | **0.8%** |
 | **slots (files × dimensions)** | **5,664,704** | **100%** | **1,416,176** | **100%** |
 
 *Catalog today* is the metadata the catalog publishes itself, before ours: only the dimensions it has a column for (`data_modality`, `reference_assembly`), so fewer slots. A slot is filled where it publishes a value for the file, whether or not a translation row reads it, and all of it is original. Our *original* row can be larger: one of the catalog's columns can fill more than one of our dimensions (its `data_modality` value "single-nucleus RNA sequencing assay" also gives `assay_type`), though it counts only the values a reviewed translation row reads. It is shown only where this run read the catalog's published values.
@@ -113,10 +113,10 @@ What each dataset holds, per dimension: every file counted once, under its recon
 | `ANVIL_nhp_dGTEx_V1` | 2 | 756 | · | · | · | 754 | · | · | · | · | · | · | · | · | · | · | · | · | · | · | 2,079 | 3,591 | 42.1% | 42.1% |
 | `AnVIL_ENCORE_293T` | · | · | · | 448 | 896 | 224 | · | · | · | · | · | · | · | 224 | · | · | · | · | · | · | 200 | 1,992 | 89.9% | 89.9% |
 | `AnVIL_ENCORE_RS293` | · | · | · | 820 | 1,640 | 410 | · | · | · | · | · | · | · | 410 | · | · | · | · | · | · | 472 | 3,752 | 87.4% | 87.4% |
-| `AnVIL_HPRC_R2` | 2 | 1,858 | · | 5,868 | 2,772 | 1,124 | 466 | · | · | · | · | · | 464 | · | · | 6 | · | · | · | 2 | 3,709 | 16,271 | 77.1% | 77.1% |
+| `AnVIL_HPRC_R2` | 2 | 1,858 | · | 5,868 | 3,696 | 1,124 | 466 | · | · | · | · | · | 464 | · | · | 6 | · | · | · | 2 | 2,785 | 16,271 | 82.8% | 82.8% |
 | `AnVIL_IGVF_Mouse_R1` | · | 4 | · | 3,959 | · | 2 | · | 11 | · | · | · | · | · | 8 | 580 | · | · | · | · | · | 2,222 | 6,786 | 67.2% | 67.2% |
 | `AnVIL_MAGE` | 3 | 831 | · | 1,558 | 12 | 779 | · | · | · | · | · | · | · | · | 5 | · | · | · | · | · | 97 | 3,285 | 97.0% | 97.0% |
-| **(every dataset)** | 328,484 | 224,726 | 37,548 | 24,383 | 14,518 | 14,284 | 14,233 | 12,522 | 5,708 | 3,637 | 3,074 | 2,854 | 960 | 642 | 585 | 477 | 160 | 24 | 18 | 2 | 19,249 | 708,088 | 97.2% | 97.2% |
+| **(every dataset)** | 328,484 | 224,726 | 37,548 | 24,383 | 15,442 | 14,284 | 14,233 | 12,522 | 5,708 | 3,637 | 3,074 | 2,854 | 960 | 642 | 585 | 477 | 160 | 24 | 18 | 2 | 18,325 | 708,088 | 97.4% | 97.4% |
 
 ### platform
 
@@ -228,12 +228,12 @@ What each dataset holds, per dimension: every file counted once, under its recon
 
 ## Change since the previous reconciled run
 
-Against `20261103_153217` (708,088 files; the same translation table). Each cell is this run's count minus that run's.
+Against `20261104_183742` (708,088 files; a different translation table). Each cell is this run's count minus that run's.
 
 | dimension | published | published harmonized | submitter | submitter harmonized | external | external harmonized | inference | inherited | conflict (inference) | conflict (sources) | conflict (published) | published unreviewed | not applicable | not classified | added over published | filled over inference |
 | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
 | data_modality | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
-| data_type | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+| data_type | 0 | 0 | 0 | +924 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -924 | +924 | +924 |
 | platform | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
 | reference_assembly | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
 | assay_type | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
@@ -243,7 +243,10 @@ Against `20261103_153217` (708,088 files; the same translation table). Each cell
 
 ### Values that moved, by dataset
 
-No dataset's count of any value moved.
+| dataset | dimension | value | before | after | change |
+| --- | --- | --- | --- | --- | --- |
+| `AnVIL_HPRC_R2` | data_type | `annotations` | 2,772 | 3,696 | +924 |
+| `AnVIL_HPRC_R2` | data_type | `not_classified` | 3,709 | 2,785 | -924 |
 
 ## The join, per evidence file
 
@@ -1002,25 +1005,25 @@ The files whose sources named different samples (the *conflict* and *incomparabl
 | dimension | published | published harmonized | submitter | submitter harmonized | external | external harmonized | inference | inherited | conflict (inference) | conflict (sources) | conflict (published) | published unreviewed | not applicable | not classified | added over published | filled over inference |
 | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
 | data_modality | 0 | 0 | 0 | 6,511 | 0 | 0 | 490 | 6,474 | 0 | 0 | 0 | 0 | 466 | 2,330 | 13,475 | 5,152 |
-| data_type | 0 | 0 | 464 | 8,391 | 0 | 0 | 3,705 | 0 | 0 | 2 | 0 | 0 | 0 | 3,709 | 12,560 | 2,772 |
+| data_type | 0 | 0 | 464 | 9,315 | 0 | 0 | 3,705 | 0 | 0 | 2 | 0 | 0 | 0 | 2,785 | 13,484 | 3,696 |
 | platform | 0 | 0 | 3,432 | 3,079 | 0 | 0 | 19 | 0 | 0 | 0 | 0 | 0 | 7,414 | 2,327 | 6,530 | 462 |
 | reference_assembly | 0 | 0 | 0 | 934 | 0 | 0 | 1,589 | 467 | 0 | 0 | 0 | 0 | 11,422 | 1,859 | 2,990 | 463 |
 | assay_type | 0 | 0 | 5,819 | 230 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 7,414 | 2,808 | 6,049 | 6,049 |
 | instrument_model | 0 | 0 | 5,819 | 230 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 7,414 | 2,807 | 6,050 | 5,366 |
 | variant_kind | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 13,476 | 2,795 | 0 | 0 |
 | variant_origin | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 13,476 | 2,795 | 0 | 0 |
-| **all dimensions** | **0** | **0** | **15,534** | **19,375** | **0** | **0** | **5,804** | **6,941** | **0** | **2** | **0** | **0** | **61,082** | **21,430** | **47,654** | **20,264** |
-| **% of slots** | **0.0%** | **0.0%** | **11.9%** | **14.8%** | **0.0%** | **0.0%** | **4.4%** | **5.3%** | **0.0%** | **<0.1%** | **0.0%** | **0.0%** | **46.9%** | **16.4%** | **36.6%** | **15.5%** |
+| **all dimensions** | **0** | **0** | **15,534** | **20,299** | **0** | **0** | **5,804** | **6,941** | **0** | **2** | **0** | **0** | **61,082** | **20,506** | **48,578** | **21,188** |
+| **% of slots** | **0.0%** | **0.0%** | **11.9%** | **15.5%** | **0.0%** | **0.0%** | **4.4%** | **5.3%** | **0.0%** | **<0.1%** | **0.0%** | **0.0%** | **46.9%** | **15.7%** | **37.3%** | **16.2%** |
 
 |  | meta-disco (8 dimensions) | % | catalog today (2 dimensions) | % |
 | --- | ---: | ---: | ---: | ---: |
 | original (catalog's published values) | 0 | 0.0% | 0 | 0.0% |
-| submitter tables | 34,909 | 26.8% | — | — |
+| submitter tables | 35,833 | 27.5% | — | — |
 | external (ENA) | 0 | 0.0% | — | — |
 | inferred | 5,804 | 4.4% | — | — |
 | inherited | 6,941 | 5.3% | — | — |
 | not applicable | 61,082 | 46.9% | — | — |
-| **filled** | **108,736** | **83.5%** | **0** | **0.0%** |
+| **filled** | **109,660** | **84.2%** | **0** | **0.0%** |
 | **slots (files × dimensions)** | **130,168** | **100%** | **32,542** | **100%** |
 
 | dimension | kind | files | competing values |

docs/review-queue-report.md

@@ -1,6 +1,6 @@
 # Review queue
 
-Source values the slot map or the entity map routed to a slot but that no authored translation rule reads yet, so they make no claim (contract 5.2), by kind of source and then by slot. A value is listed once per source, dataset, table and column it was found in; "rule it would use" names the seeded rule that matches it, or — where none does. For a donor's slot (phenotypic_sex, genetic_ancestry, organism_type, reported_ethnicity), files counts donors, and for a sample's slot (biosample_type), samples, each dataset's identifiers apart. 155 listed, from evidence under data/source_evidence and data/entity_evidence.
+Source values the slot map or the entity map routed to a slot but that no authored translation rule reads yet, so they make no claim (contract 5.2), by kind of source and then by slot. A value is listed once per source, dataset, table and column it was found in; "rule it would use" names the seeded rule that matches it, or — where none does. For a donor's slot (phenotypic_sex, genetic_ancestry, organism_type, reported_ethnicity), files counts donors, and for a sample's slot (biosample_type), samples, each dataset's identifiers apart. 153 listed, from evidence under data/source_evidence and data/entity_evidence.
 
 ## Published: the catalog's published columns
 
@@ -34,8 +34,6 @@ Source values the slot map or the entity map routed to a slot but that no author
 | 462 | `'alignments'` | `anvil` | `AnVIL_HPRC_R2` | `alignments_winnowmap` |  | data_type.alignments |
 | 462 | `'chains'` | `anvil` | `AnVIL_HPRC_R2` | `chains_to_chm13_mc` |  | data_type.chains |
 | 462 | `'chains'` | `anvil` | `AnVIL_HPRC_R2` | `chains_to_grch38_mc` |  | data_type.chains |
-| 462 | `'gaps'` | `anvil` | `AnVIL_HPRC_R2` | `gaps` |  | data_type.gaps |
-| 462 | `'sequences'` | `anvil` | `AnVIL_HPRC_R2` | `t2t_sequences` |  | data_type.sequences |
 | 25 | `'barcode to sample mapping'` | `anvil` | `AnVIL_IGVF_Mouse_R1` | `file` | `content_type` | — |
 | 12 | `'alignments'` | `anvil` | `AnVIL_HPRC_R2` | `alignments_v2` |  | data_type.alignments |
 | 11 | `'Nanopore reads'` | `anvil` | `AnVIL_IGVF_Mouse_R1` | `file` | `content_type` | — |
@@ -236,8 +234,10 @@ Every authored translation rule, by slot and most files first: the source values
 | 466 | 0 | 466 | 0 | data_type.assembly | `any` | `'assembly'` | data_type: assembly | The `assembly` table-name span; its files are the de-novo assemblies (.fa.gz). |
 | 462 | 0 | 462 | 0 | data_type.censat | `any` | `'censat'` | data_type: annotations | Centromeric satellite annotation tracks (BED) of an assembly. |
 | 462 | 0 | 462 | 0 | data_type.censat_centromeres | `any` | `'censat_centromeres'` | data_type: annotations | Centromere annotation tracks (BED) of an assembly. |
+| 462 | 0 | 462 | 0 | data_type.gaps | `anvil / AnVIL_HPRC_R2` | `'gaps'` | data_type: annotations | The `gaps` table-name span (#765): each file is a BED of one assembly's gaps, on the assembly's own sequences. HPRC files it under the assembly's `annotation/chrom_assignment/` folder, and the HPRC Data Explorer's annotations catalog lists it as "ChromAlias Gaps". Scoped to this dataset: elsewhere the word need not name an assembly's annotation. |
 | 462 | 0 | 462 | 0 | data_type.liftoff | `any` | `'liftoff'` | data_type: annotations | Gene annotations lifted onto an assembly with Liftoff (GFF3). |
 | 462 | 0 | 462 | 0 | data_type.segdups | `any` | `'segdups'` | data_type: annotations | Segmental duplication tracks (BED) of an assembly. |
+| 462 | 0 | 462 | 0 | data_type.sequences | `anvil / AnVIL_HPRC_R2` | `'sequences'` | data_type: annotations | The `t2t_sequences` table's span (#765): each file (`.t2t_chromosomes.tsv`) is a table of one assembly's telomere-to-telomere sequences, each with its chromosome and its gaps. HPRC files it under the assembly's `annotation/chrom_assignment/` folder, and the HPRC Data Explorer's annotations catalog lists it as "ChromAlias T2T". Scoped to this dataset: elsewhere a `sequences` table could hold the sequences themselves. |
 | 304 | 0 | 304 | 0 | data_type.kallisto_single_cell_rnaseq_output | `anvil / AnVIL_IGVF_Mouse_R1` | `'kallisto single cell RNAseq output'` | data_modality: transcriptomic | IGVF's `file.content_type`. kallisto's single-cell RNA-seq output, as IGVF names it, so RNA. Most of these files are titled snRNA-seq alone, which already says transcriptomic; on the two of the SHARE-seq pair it is what says which half the file holds (#570). Scoped to this dataset, the one whose vocabulary it is. No data_type: the tar is a bundle of kallisto's outputs — the cached tar heads hold, variously, BUS files, `matrix.ec`, `kb_info.json` and cells × genes `.mtx` matrices — not one data type, though IGVF files it as a cell-by-gene `MatrixFile`. |
 | 288 | 0 | 288 | 0 | data_type.unaligned_reads_with_5mcg_5hmcg_mods | `any` | `'unaligned reads with 5mCG_5hmCG mods'` | data_type: reads | Reads carrying base-modification calls; the modification tags do not change the data type. |
 | 230 | 0 | 230 | 0 | data_type.bam | `any` | `'bam'` | nothing (reviewed, no claim) | A file format, not a data type: the kinnex table's data_type column holds what its filetype column should. Ruled to mean nothing here rather than left queued. |