[Proposal] Mechanize the configuration contract: five-way sync check, provenance script, and a discipline ledger

[liwen-aaa]

Summary

AGENTS.md already states the repo's core stance: "Do not rely on prompts for runtime invariants" and "The runtime owns enforceable facts." Yet the Package configuration contract — the repo's own load-bearing promise — is enforced entirely by prompts. This proposal closes that gap by mechanizing three specific promises, in the same style as the existing drift guard (which proves the approach works): a five-way sync check for user-selectable config, a runtime-provenance script, and a lightweight append-only discipline ledger.

Scope guardrail up front: this is not a workflow engine, an orchestration layer, or a meta-framework. It is 2–3 small scripts that turn already-written prose promises into runnable checks.

---

Background: the contract is already written

AGENTS.md:75 (Package configuration contract):

A user-selectable model, feature toggle, permission, concurrency limit, theme/UI preference, or other package-owned choice must update extensions/setup/, extensions/shared/setup-config.ts, the no-argument /openpi-setup status output, SETUP.md, and the canonical defaults and prose in README.md.

AGENTS.md:79 (runtime provenance):

Before diagnosing installed behavior, provider compatibility, a manual Pi smoke, or any UI result, … Prove both the checkout revision and the single OpenPI source reported by pi list before reasoning from source code.

Both are unenforced prose today. A contributor adding a config option must remember five touch points by hand; a maintainer must review all five by eye.

The gap: contract vs enforcement

Promise (AGENTS.md) | Enforced today? | Evidence -- | -- | -- Tool classification (child-safe / excluded), fail-closed | ✅ Yes — drift guard | tests/extensions/shared/child-session.test.ts (source-scan, reaches factory registrations) Pi host packages stay peers, not bundled | ✅ Yes | tests/extensions/setup/package-contract.test.ts bun run check / test exist and run every round | ✅ Yes (CI) | .github/workflows/ci.yml Both test suites must be non-empty | ✅ Yes | scripts/run-tests.mjs Five-way sync for config options (AGENTS.md:75) | ❌ No — prose only | no test or script references SETUP.md / README.md as config-consumer contracts Runtime provenance (AGENTS.md:79) | ❌ No — prose only | no script exists as any last resort (AGENTS.md) | ⚠️ partial — lint disabled (noExplicitAny: off); 11 occurrences in tests/ today | biome.json

---

Impact

Before

• Adding a config option: contributor must remember 5 touch points; maintainer must eyeball all 5 in review. Nothing catches a miss except a user discovering the knob is undocumented.

• Diagnosing a runtime issue: "read the README section and prove provenance" is a prose step that gets skipped under time pressure — exactly the D-02 failure shape (a requirement written in text is accepted to be skipped).

• New contributors cannot tell which promises are actually enforced: the repo looks fully mechanized (drift guard, package-contract) while the soft contracts are unguarded.

After

• A config field missing from docs fails bun run check — the miss is caught in the same round it is introduced, not in a user support thread weeks later.

• Provenance is one command; the diagnostic path starts from verified facts.

• The ledger makes enforcement status visible: green rows are real mechanisms, prose rows are explicitly marked as trusted-to-skip.

Cost

• ~2 small Node scripts (no new deps), ~1 appendix doc. Adds a few seconds to bun run check (well within the 20-min CI budget).

• Review overhead of the proposal itself is the main cost; the ongoing cost is near zero because the checks are append-only additions to the existing check script.

---

Acceptance criteria

• scripts/check-config-contract.mjs exists, runs in bun run check, and:

• fails when a field in setup-config.ts is absent from all of SETUP.md / README.md / extensions/setup/ (fixture-based test included),

• fails when a doc references a config knob absent from the field table,

• passes on the current tree (no existing field is orphaned — if any is, the fix is to document it, not to weaken the check).

• scripts/provenance.mjs exists and prints checkout revision + pi list source + match verdict.

• (Phase 3) docs/disciplines.md exists with the enforcement-status table and an append-only check wired into check.

• All existing tests and bun run check stay green.

---

Open questions

1. Doc-coverage semantics: presence-based (grep) vs semantic (does the doc actually explain the knob)? This proposal starts presence-based (machine-checkable) and leaves semantic quality to human review — confirm that's acceptable.

2. Where should the ledger live — docs/disciplines.md or root DISCIPLINES.md? (The repo currently has no such file; docs/ is the natural home.)

3. Should the provenance script also be exposed as a pi skill (pull-based, so agents invoke it before diagnosing), or is a plain script enough for the first cut?

---

Related

• (Windows test-suite issue): shares the same root cause (CI only runs Linux, prose promises unguarded) and the same fix pattern (small scripts + explicit status).

• .gitattributes proposal: another first-day contributor-experience fix, independent of this one.

[liwen-aaa]

Feasibility proof — this proposal is already implemented and green against the current tree.

Since opening this, I built the three pieces as a proof of concept and ran them against the current checkout. They work:

1. Five-way sync check — check-config-contract.mjs (249 lines, zero deps)
Three layers:

• Layer 1 (hard, zero false positives): every leaf field path in extensions/shared/setup-config.ts must appear in extensions/setup/index.ts (the typed write tool). A field that is defined but never consumed = fail.
• Layer 2 (mapping): each field is registered with doc keywords from the actual wording in SETUP.md / README.md; at least one keyword must hit. This avoids grepping field paths (e.g. ui.showHeader) that never appear literally in docs, which would immediately red on the current tree — a false positive, not a check.
• Layer 3 (insurance): README must still mention /openpi-setup.

Current-tree result (pure upstream checkout, no local edits):

✓ config contract
  layer 1: all 13 leaf fields consumed by setup/index.ts
  layer 2: all 13 fields hit at least one doc keyword (SETUP.md ∪ README.md)
  layer 3: README still mentions /openpi-setup

6 fixture tests, all passing: leaf-field extraction across single-line / multi-line / empty-object shapes, layer-1 missing-wiring red, layer-2 zero-doc-hit red, layer-2 unregistered-field red, layer-3 entry-removed red.

2. Provenance script — straightforward: print git rev-parse --short HEAD + the single OpenPI source from pi list + whether they match. No blockers.

3. Discipline ledger — 12 OP-* criteria in an append-only table with an enforcement column; two mechanisms (append-only-ledger, wired-check) keep it append-only and verify the claimed commands actually exist and are called by CI. Mirrors the drift-guard style.

Two points I want to pre-empt, since they are the natural review objections:

"This is preventive work with no observed failure mode." True — and that is precisely the D-02 shape this repo already accepts as a cost elsewhere ("a requirement written in text is accepted to be skipped"). The proposal never claims the tree is currently broken; it claims the check makes a future miss fail bun run check in the round it is introduced. Whether that is worth ~2 small scripts is a fair maintainer call — the proof above makes the cost concrete (zero new deps, a few seconds of CI).

"The keyword map is itself a new source of truth to maintain." Also true. The map is the judgment body: adding a field requires registering it, and the layer-2 unregistered-field test catches a forgotten registration. That is a real ongoing cost, and it is the price of making doc coverage machine-checkable while docs use human wording instead of field paths. If the maintainers prefer dropping layer 2 and keeping only the hard layer 1 + layer 3, the script supports that — the layers are independent.

Relationship to #73: no overlap. #73 is about cross-extension imports, test discovery, and peer-dep drift. This proposal is scoped to the config contract (AGENTS.md:75) and runtime provenance (AGENTS.md:79) only.

Happy to turn this into a PR if the direction is accepted — or to adjust the scope (e.g. drop layer 2, move the ledger location) first.

[liwen-aaa]

Relationship to #198: complementary, not overlapping. #198 organizes where validated knowledge lives (document states, index, templates). This proposal validates whether prose promisesin AGENTS.md are actually enforced (config contract, provenance). #198's "static documentation contract check" (user story 29) is the same check style — this proposal is the execution-side counterpart for the config contract specifically.

[627150795]

Hi! I would like to work on this issue. I will implement the three proposed contract checks against the current repository conventions: configuration synchronization, runtime provenance, and an append-only discipline ledger. I will preserve existing runtime semantics, keep the checks reviewable, add focused validation, and update this thread before opening a PR.

[627150795]

Implementation is ready in PR #242: https://github.com/tt-a1i/openpi/pull/242\n\nCommit: 1aca868\nBranch: 627150795:feat/issue-213-config-contract\n\nValidation passed for the config contract (14 persisted fields plus 8 fixture checks), discipline ledger, runtime provenance, targeted Node test, lint, typecheck, and diff whitespace checks. The full Windows check/test baseline still has existing CRLF, path, symlink, and process-lifecycle failures unrelated to this change.\n\nPlease review when convenient; the branch is ready for maintainer CI approval and review.

[627150795]

代码审计更新（对应 PR #242）：

• 配置契约检查器现在会发现 MyPiSetupConfig 中新增但未登记的可选字段。
• discipline ledger 检查器现在会拒绝删除、重编号或改写历史 OP 行。
• 新增删除并重编号的回归测试；配置检查、ledger 检查、lint、typecheck 均通过。

这是对原 PR 检查器实际漏检行为的修复。

[627150795]

代码审计补丁已追加到本 PR：

• 复现并修复了配置检查器的误报：仅在真正的 buildConfig 写入区验证字段，参数 schema 中残留 params.xxx 不再被当成已落盘。
• 新增反例：保留 schema 参数、删除实际 writer 赋值时必须失败。
• 远程提交：daa33f3。
• 验证：配置自测 13 项通过，真实配置 14 项通过，Node 定向测试 1/1、lint、typecheck 均通过；当前远程 Node 22、Node 24、Windows CI 全部通过。

这只修正契约检查的有效性，不改变 OpenPI 运行时行为。