policy(v2.10): certify source/data rights, redistribution scope, and derivative-publication policy for the terminal ML handoff

[dmidlo]

Parent handoff milestone: #721
Related: #526 #530 #535 #623 #642 #643

Editorial audit gap

The repository has strong scientific provenance, source identity, restricted-feed privacy, broker data-rights, official-source registry, retention and publication contracts, but the terminal ML handoff lacks one cross-program rights policy binding every source and derived artifact family, including HistData.com inputs and synthetic derivatives.

The Python package's MIT license governs repository software; it does not by itself establish rights to redistribute third-party source data, normalized source rows, retained captures, calendar documents, or derived datasets. Likewise, a product being synthetic does not automatically prove unrestricted redistribution rights.

This issue is a policy/engineering gate, not legal advice. It requires evidence-backed operator decisions and fail-closed publication behavior where rights are unknown or restricted.

Outcome

Define a versioned DataRightsPolicyV1 (or equivalent) and rights registry covering every source/product admitted to #721's HandoffBundleV1.

For each source/artifact family retain at least:

• source/provider/institution identity;
• governing terms/license/policy evidence locator and review timestamp;
• exact source/terms artifact hash where retention is permitted;
• acquisition/download permission state;
• local cache/retention permission;
• transformation/derivation permission;
• internal research/model-training permission;
• raw-row redistribution permission;
• normalized-data redistribution permission;
• synthetic/derived-product redistribution permission;
• aggregate/statistical-output publication permission;
• manifest/hash/lineage metadata publication permission;
• required attribution/notices;
• account/geographic/use-class restrictions where applicable;
• retention/deletion obligations;
• review expiry/change-detection policy;
• stable rights-policy ID and status.

Unknown rights are not treated as permission.

Publication modes

Every durable dataset/product receives one explicit publication class such as:

• public_redistributable;
• operator_local;
• restricted_private;
• derived_only;
• metadata_only;
• unknown_blocked.

These states govern bytes, not merely documentation wording.

HistData source boundary

Perform an explicit current review of HistData.com terms and provider statements relevant to downloading, retaining, transforming and redistributing source and derivative data. Retain the evidence and decision rather than inferring rights from historical project practice.

If raw/normalized or synthetic-derivative redistribution is not clearly authorized, the scientific campaign may still complete and #721 may support an operator-local/private data handoff when permitted. Package code, schemas, hashes, manifests and methodology may be public only to the extent their own rights policy permits. Do not make public dataset publication a scientific requirement when source rights require a narrower distribution mode.

No source rows are relicensed by the repository's software license.

Other source families

Apply the same registry to:

• official sovereign/statistical/calendar artifacts;
• CFTC/positioning/context products;
• optional commercial/restricted context;
• broker/plugin capture evidence;
• independent overlapping feeds;
• externally supplied reference/validation datasets;
• generated trader/orderflow products whose ancestors carry restrictions;
• model/feature products whose distributability is constrained by ancestor policy.

An official/public institution is not automatically assumed public-domain in every jurisdiction or for every artifact; record the actual reviewed policy where material.

Rights propagation

Define deterministic propagation from ancestors to derived products. A derived artifact cannot silently receive a more permissive publication state than its governing policy allows.

Where transformation genuinely changes redistribution status under reviewed terms, bind that conclusion to the transformation/product class and rights-policy identity rather than applying it globally.

#693 lineage must make ancestor rights discoverable, and #643 provenance-aware garbage collection must respect retention/deletion obligations.

Enforcement

Publication/export APIs must fail closed when the requested distribution action conflicts with the effective rights policy. At minimum test:

• public export of operator_local data is refused;
• restricted raw source bytes cannot be embedded in a public handoff bundle;
• metadata-only publication omits prohibited payloads;
• local clean-room consumer can resolve an authorized local artifact without making it publicly distributable;
• changed source terms create a successor rights review/policy identity and do not rewrite historical publication receipts;
• restricted ancestor lineage is not lost during feature projection/materialization;
• credentials/private account identifiers never enter rights evidence or public manifests.

Handoff interaction

HandoffBundleV1 must bind the effective rights-policy registry/root and the allowed handoff/distribution mode. A downstream private repository must be able to determine what it may retain, copy, export or publish without reading informal project notes.

The clean-room certification under #721 tests both:

1. an authorized consumer path; and
2. at least one prohibited export path that must refuse deterministically.

Documentation/nonclaims

Clearly separate:

• open-source software licensing;
• source-data licensing/terms;
• derivative/synthetic product distribution policy;
• private research use;
• public publication.

Do not describe a scientifically verified dataset as open data unless the rights registry independently supports that claim.

Acceptance

Every source and retained/derived product reachable from the terminal handoff has one reviewed rights state; HistData.com source/derivative handling has explicit evidence-backed policy; prohibited publication/export paths fail closed; operator-local/private handoff remains possible where authorized even if public redistribution is not; #526/#721 publication wording matches the actual allowed mode; #691 retains the requirement and #525 tracks implementation/execution/evidence independently.