Skip to content

Verifier: bind absent and relocated scoped base inputs without inventing an empty agent surface #580

Description

@pengfei-threemoonslab

Selected execution — 2026-09-24

Days 1–5 design is in PR #869, docs/research/application-days1-5/comparison-design.md. Bind independent base/head scopes and exact compared trees; distinguish Git-proven path absence, unconfigured existing app, relocation and unread input. A rename alone is not agent identity. Preserve reviewed-subject requirements for verifier/qualification; generated advisory extraction cannot satisfy them. Implementation and the fixed #1975/#1977 acceptance remain open for Weeks 2–3. Selected milestone 13.


The earlier advisory-release deferral below is historical. This issue is now selected for the application proof milestone; its implementation is not yet complete.

First advisory release scope — 2026-09-11

Owner decision: adoption > completeness; make v1.0 ready with repository-independent work, then expand adoption. Historical scoped tool-source inputs are deferred while the host workflow is made ready. The partial red test is not an implementation or a passing result.

Removed from the selected v1.0 milestone; kept open and deferred, not resolved. #643/#644/#645/#646 define current product acceptance; #572 preserves the historical qualified no-go and #648 owns the actual publication contract. The prior evidence and acceptance below remain a record of the deferred work, not newly satisfied conditions.


Problem

Found while implementing #564. The miner's missing-base-directory return hides an engine input-contract gap: a committed verifier comparison uses one repository-relative config_relative for both trees. A real PR-added manifest can already produce a terminal result with base_status=missing_manifest, but that does not prove an evaluated empty agent surface. A renamed scoped project has no separately bound base-manifest path.

Evidence

At main ded4be37f, cli/verify/orchestrator.py materializes the base tree and reads base_tree_dir / config_relative; absence returns missing_manifest. --diff-from can consume a separately produced old-scope report, but marks diff_from_provided rather than an independently scanned base and lacks the normal base subject/config join. The qualification scorer requires a successful, tree-bound base run. Neither workaround may be represented as that evidence.

The unchanged fixed public history supplies both concrete cases:

Those observations establish paths and file identities, not reviewed business authority or whole-agent identity. #564 will preserve them in a typed unsupported-input obligation instead of copying head sources into base or fabricating a verifier catch.

Required design and implementation

Define how the existing verifier plan/receipt binds independent base and head scoped inputs. A renamed scope needs the real base source path, head path and their evidence; an absent path needs an explicit, reviewed comparison subject with a precise absence claim. Absence of a manifest must never by itself mean absence of tools or runtime authority. Keep genuine PR-added manifests and all trust-root changes visible.

Prefer a bounded extension of existing manifest/diff/verification identities rather than a parallel verdict engine. State how this affects base status, capability coverage, qualification eligibility and report compatibility before implementing. Preserve fail-closed behavior until the input can actually be supported.

Acceptance

  • Real PR-added manifests continue through the existing terminal verifier route with their trust-root delta intact.
  • Renamed scopes bind original base/head trees and their actual input locations; partial/ambiguous renames cannot silently choose an agent identity.
  • A new scope distinguishes a Git-proven absent path from an unconfigured existing surface and an unreadable tree; no head inventory is manufactured at base.
  • Plan, receipt, report and qualification consumers agree on the new subject/status semantics without weakening successful-base or policy requirements.
  • Fixed #1975/#1977 refs and new/rename/unrelated-deletion fixtures demonstrate truthful evidence or precise refusal.
  • Cold-start and reviewed-scope measurements remain separate; no relabeling, invented effect/authority/binding declaration or scan fallback is used as a successful PR catch.

Sequencing

Split from #564 as its existing implementation direction requires; defer this newly discovered contract work under the user's instruction. Coordinate #563's reviewed scope, #559's exact-byte scorer, #569's compatibility freeze and #572's release decision. A typed miner obligation does not satisfy the historical catch bars or qualify v1.0. No milestone or release eligibility is granted by filing this issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

P1Next after P0; blocks other work or ships a misleading resultarea:benchmarkLabeled corpus, accuracy measurementarea:identityVerification identity, receipts, reproducibility

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions