build: isolate Pixi integration environments

[leej3]

AI-generated draft — not reviewed by John

Summary

• make the root Pixi environment installable and testable before any submodule
  checkout;
• retain checkout-submodules as the explicit exact recursive setup for broad
  cross-component work;
• add locked static and service-backed upstream deployments in isolated inline
  Pixi environments;
• provide recorded modes that restore parent gitlinks and reject dirty
  components, plus worktree modes that preserve candidate commits and edits;
• restore the 14 legacy service assertions and their scoped integration-contract
  successors;
• add a read-only semantic diff between the prepared pool cache and the live
  public pool; and
• document how Milestones 1–3 map into the Milestone 4 product and engineering
  surface.

Why

The previous root environment declared Python packages from uninitialized
submodule paths. Pixi therefore needed those submodules before it could create
the environment containing the checkout helper—a circular bootstrap. It also
mixed engine development, the historical CON migration stack, Hugo, Node, and
git-annex into one lock and task namespace.

Submodules remain the reviewed source-level dependency pins. They are not root
environment prerequisites. Each upstream deployment task now initializes the
exact repositories it needs, including nested submodules, before resolving its
own locked dependencies.

Two policies make upstream development explicit:

• recorded tasks restore the commits in the parent tree and refuse modified
  component worktrees; and
• *-worktree tasks initialize only missing repositories and preserve current
  commits, tracked edits, and untracked files.

This supports coordinated work across upstream repositories without letting a
validation command silently discard the work being tested. Once component
changes are committed and their gitlinks are recorded, the clean recorded tasks
and hosted workflow establish the next known-good parent stack.

Capability audit

Milestone 4 carried the complete accepted content and supported product
behavior forward: 186 canonical and 13 reference records, the 199-record and
185-page projection, 186-node/467-edge graph, assets, project-path behavior,
credential-free editor review bundles, Zotero evidence, offline operation,
deterministic builds, and rollback. Its traceability baseline maps all 106
parent Python methods, five Playwright definitions/nine executions, 42 Zotero
methods, and eight editor methods with no unmapped source assertion.

The audit did find one execution gap: 19 assertions for the required
service-backed engineering stack were described as retained but no longer ran
from main. This PR restores the 14 detailed service tests and the five former
integration concerns through a scoped full-stack contract. The service stack is
a first-class engineering integration capability; only the downstream static
consumer is designed not to require it.

The new capability map distinguishes carried product behavior,
engineering-only functionality, and intentionally superseded milestone
mechanisms.

Task surface

Scope	Recorded pins	Candidate worktrees
Static site	build-upstream-static, serve-upstream-static	build-upstream-static-worktree, serve-upstream-static-worktree
Full stack	check-upstream, serve-upstream	check-upstream-worktree, serve-upstream-worktree

test-upstream-full runs the isolated service and orchestration contracts.
diff-upstream-pool verifies the prepared cache manifest, fetches the live
public collection without replacing the cache, and reports added, removed, and
field-level record changes by PID. It writes the exact ignored JSON report to
build/upstream-stack/pool/live-diff.json; differences are informational unless
the script's explicit --check mode is selected.

The manual Engineering environment workflow runs the live recorded full-stack
check on a selected parent ref after this workflow is available on main.

Known limitation

The static recorded deployment is reproducible from Git, annex, and tool pins.
The full deployment pins its software stack, but a fresh checkout still fetches
the current public pool snapshot. The historical capture contained 4,978
records; this review found 4,979. The ignored cache is digest-checked and reused,
and the new diff makes subsequent live drift explicit. True bit-for-bit
full-data reconstruction would still require an explicitly licensed immutable
snapshot artifact. This PR records that boundary rather than silently
republishing the live pool.

Verification

• root suite: 44 engine tests and 39 development tests passed;
• isolated full-stack contracts: 21 passed, including the 14 legacy service
  assertions;
• recorded static build: 1,973 pages, artifact audit clean, component worktrees
  clean afterward;
• dirty-worktree proof: recorded mode rejected the test edit, worktree mode
  built successfully and preserved it exactly;
• live recorded full check: 4,979 records matched in each collection, anonymous
  writes were rejected, editor writes were confined to protected incoming, UI
  and schema assets passed, and all child services stopped cleanly;
• live cache comparison: 4,979 cache records and 4,979 live records were
  semantically identical at digest ca43704f89afb75c29ecd9b338c018177e5e696a3bc821e0ded661ce8b269c6c;
• root and both inline locks pass --check; and
• Snapper/pre-commit, workflow parsing, and git diff --check passed.

The pull-request matrix runs the root suite, recorded and worktree static builds,
and isolated full contracts on Ubuntu 24.04 and macOS 14 without making ordinary
PR checks depend on mutable public-pool availability.

[leej3]

AI-generated draft — not reviewed by John

The post-milestone metadata source-adapter goal is now part of this branch in docs/metadata-source-adapters.md.

The first two concrete proofs are:

• downstream draft con/test-orinoco-downstream-website#7: experimental Zotero plugin host, semantic live diff, and fail-closed evidence refresh;
• stacked draft #8: one-off DataLad provenance run over dump-research-info:data/con_site, retained as inactive review evidence.

The live Zotero review caught a collection-policy rename that would otherwise appear to remove 146 candidates. The DataLad run found 19 source-only candidates and enrichment evidence for 60 matched records, and reran to the identical output tree. Neither proof promotes canonical metadata.