feat: explicit retrieval-sources override with {edge_id} record-URL templating

[SkyeAv]

Summary

ManualProvenance gains an optional explicit sources template that replaces the derived primary/upstream sources[] emission entirely, plus an {edge_id} placeholder for per-edge record URLs.

provenance:
  override:
    sources:
      - resource_id: infores:multiomics-drugapprovals
        resource_role: aggregator_knowledge_source
        upstream_resource_ids: [infores:dailymed, infores:faers]
        source_record_urls:
          - "https://db.systemsbiology.net/gestalt/cgi-pub/KGinfo.pl?id={edge_id}"
      - resource_id: infores:faers
        resource_role: primary_knowledge_source
      - resource_id: infores:dailymed
        resource_role: supporting_data_source

Motivation

DAKP needs to reproduce its legacy KG retrieval provenance — the transforming resource as aggregator_knowledge_source carrying a per-edge gestalt viewer URL (KGinfo.pl?id=<edge id>), an upstream as the primary_knowledge_source — declaratively from the table config, instead of post-processing the built edges file.

Design

• SourceOverride model (models.py): resource_id / resource_role / optional upstream_resource_ids / optional source_record_urls. Roles validate against the three Biolink ResourceRoleEnum members (literals, no heavy biolink import); all CURIEs must be infores:-prefixed; URLs must be http(s) after stripping {edge_id} occurrences.
• Coherence validation (override-bad-sources): sources is mutually exclusive with upstream_resource_ids / upstream_source_record_urls (it subsumes both), must be non-empty, resource_ids unique, and at least one entry must be a primary or aggregator knowledge source (keeps the RIG audit's non-empty-primary check satisfiable).
• retrieval_sources(explicit=...) (lib.py): emits exactly the templated entries, in order; id still mirrors resource_id and absent list fields stay typed nulls for the Rust null-stripper, so output formatting is unchanged.
• {edge_id} resolution: the edge id is a deterministic content hash assigned by the Rust deduper after subgraphs are written, so the literal placeholder is what gets hashed and a post-dedup streaming sweep (_resolve_edge_id_placeholders) substitutes each record's own id into sources[].source_record_urls in the final edges NDJSON only. Files without the marker are left byte-identical (per-line substring precheck, no full parse).

Tests

• tests/test_models.py: 6 new validation tests (valid template, bad role, non-infores ids, non-URL, both mutual-exclusion combos, empty/duplicate/all-supporting).
• tests/test_lib.py: op-arg + executed-op test for the explicit template (exact order/roles/upstreams, literal placeholder unresolved at build stage, typed nulls on bare entries) and a direct test of the placeholder sweep.
• Full suite: 1005 passed, 39 skipped; ruff check/format clean; pyright 0 errors.

Docs: new "Explicit sources template" subsection in docs/configuration/table.md.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 89ced56d-c60c-4035-9331-4dcf55a459bc

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}