Skip to content

docs: 建立知识资产与公开证据管理契约,并写入 AGENTS.md #198

Description

@tt-a1i

Problem Statement

OpenPI currently uses GitHub Issues as the public entry point for bugs, feature work, research, architecture comparisons, product decisions, Benchmark publication, and implementation tracking. This supports rapid iteration, but it also makes one mutable timeline carry several incompatible evidence states.

As a maintainer, I cannot reliably tell which Issue comment is exploratory, which conclusion is validated, which decision still governs the project, which result belongs to a frozen source revision, or whether a public summary has a durable and reproducible evidence source. Readers and Agents face the same ambiguity. Important work is discoverable only if they already know the right Issue number, and local ignored Benchmark assets can appear complete on one machine while remaining absent from a clean repository checkout.

OpenPI needs a lightweight knowledge-asset contract that preserves the speed and visibility of Issues while moving reusable conclusions, formal Benchmark reports, and durable decisions into versioned repository documentation. The contract must also become part of the canonical cross-Agent guidance so future work is classified consistently without turning the always-loaded Agent instructions into a documentation manual.

Solution

Keep GitHub Issues as the public discussion and work-tracking surface. Establish the repository documentation tree as the canonical, reviewable source for validated research, design history, durable decisions, architecture explanations, and formal Benchmark protocols and results.

Add one repository-level documentation index that explains the information architecture, document states, source-of-truth boundaries, linking rules, and evidence handling policy. Preserve the existing design and research archives, and add minimal decision and Benchmark areas with reusable templates. Promote a small, reviewed slice of existing work—starting with the latest Luna high Pi/OpenPI Benchmark and one high-value research record—to prove the structure without migrating the entire Issue history.

Add a compact Knowledge and evidence rule to the canonical Agent contract. The always-loaded rule will state the classification invariant and point to the repository documentation index for branch-specific detail. Claude-specific guidance will continue to reference the canonical Agent contract rather than duplicate shared policy.

Formal Benchmark publication will use three linked projections: a concise public Issue summary, a versioned repository report, and a content-addressed raw-evidence archive. The report will state what is validated, what is inferred, what remains unknown, and which source/model/task/verifier revisions bound the result.

User Stories

  1. As an OpenPI maintainer, I want Issues to remain the fastest place to publish and discuss work, so that documentation governance does not slow product iteration.
  2. As an OpenPI maintainer, I want validated conclusions to live in versioned repository documents, so that they can be reviewed and changed with the codebase.
  3. As an OpenPI maintainer, I want one documentation index, so that I do not need to remember Issue numbers or directory conventions to find project knowledge.
  4. As an OpenPI maintainer, I want each knowledge category to have a clear canonical owner, so that the same conclusion does not drift across Issues, README prose, and reports.
  5. As an OpenPI maintainer, I want documents to declare their status and applicability, so that outdated material is not mistaken for current project policy.
  6. As an OpenPI maintainer, I want superseded documents to preserve history and point to their replacement, so that architectural reasoning is not silently rewritten.
  7. As a contributor, I want to know whether my finding belongs in an Issue, research report, design archive, Decision, Benchmark report, or release note, so that I publish it in the correct place the first time.
  8. As a contributor, I want existing document categories and templates, so that I can add evidence without inventing a new structure.
  9. As a contributor, I want Issue summaries and canonical documents to link both ways, so that discussion and durable knowledge remain connected.
  10. As a contributor, I want only high-value historical material migrated initially, so that this work stays reviewable and does not become a repository-wide rewrite.
  11. As a reader, I want to begin from a clean checkout and navigate to the current research, decisions, architecture, and Benchmark results, so that local machine state is not required to understand OpenPI.
  12. As a reader, I want to distinguish a promotional Benchmark summary from its full evidence, so that I can evaluate the claim at the appropriate depth.
  13. As a reader, I want every formal result to identify its model, thinking level, source revision, task set, verifier, sample size, and limitations, so that I know exactly what the result supports.
  14. As a reader, I want exploratory and infrastructure-failed runs labelled accurately, so that incomplete experiments are not presented as product outcomes.
  15. As a researcher, I want facts, inferences, recommendations, and unknowns separated, so that later work can challenge the right part of an argument.
  16. As a researcher, I want sources and last-verification dates retained, so that time-sensitive comparisons can be refreshed instead of copied forward blindly.
  17. As a Benchmark operator, I want a stable protocol and result template, so that repeated campaigns report the same evidence categories.
  18. As a Benchmark operator, I want small summaries and manifests stored with the repository while large artifacts remain in a verified archive, so that evidence remains auditable without bloating Git history.
  19. As a Benchmark operator, I want raw artifact archives to have stable identities, sizes, and SHA-256 manifests, so that a report can prove which evidence it references.
  20. As a Benchmark operator, I want local ignored runs preserved until they are explicitly classified and reviewed, so that unique evidence is not lost during documentation work.
  21. As a security-conscious maintainer, I want credentials, private Session content, user data, caches, and temporary workspaces excluded from publication by default, so that openness does not create a data leak.
  22. As a release manager, I want release notes to describe only shipped user-visible behavior, so that merged code, validated research, and public releases remain separate evidence states.
  23. As a release manager, I want releases to link relevant Decisions and Benchmarks without copying their full contents, so that each source of truth remains singular.
  24. As an Agent working in the repository, I want a short always-loaded classification rule with a precise pointer to detailed policy, so that I follow the contract without paying its full context cost on unrelated tasks.
  25. As an Agent publishing research, I want a completion criterion that requires a canonical document and bidirectional Issue link, so that publication does not stop at an Issue comment.
  26. As an Agent publishing a Benchmark, I want the contract to require frozen identities, failure classification, usage evidence, and an archive reference, so that I cannot promote an incomplete run into a result.
  27. As an Agent updating older conclusions, I want to mark them superseded rather than overwrite their historical meaning, so that the project retains a causal design record.
  28. As a Claude Code user, I want Claude-specific guidance to keep pointing at the shared Agent contract, so that repository policy is not duplicated in multiple always-loaded files.
  29. As a project maintainer, I want static validation of navigation, document status, and required Benchmark fields, so that documentation drift fails visibly in review.
  30. As a project maintainer, I want this first implementation to fit one focused change set, so that the contract can be adopted before broader historical curation begins.

Implementation Decisions

  • GitHub Issues remain the canonical location for discussion, questions, proposed work, and execution status. They are not the sole canonical copy of reusable conclusions.
  • Repository documentation becomes the canonical location for validated research, design history, durable Decisions, architecture explanations, and formal Benchmark protocols and reports.
  • Preserve the existing design and research categories. Add a repository-level documentation index plus minimal Decision and Benchmark categories; add an architecture category only when content needs a stable architecture projection distinct from design history.
  • The repository-level index defines the information hierarchy, category ownership, status model, linking contract, and raw-evidence policy. Category indexes remain concise projections of that single contract rather than redefining it.
  • Use three document states: draft, validated, and superseded. Every governed document records creation date, last-verification date, applicable OpenPI revision or version, related Issues and PRs, and any superseding relationship.
  • Research documents explicitly distinguish verified facts, inferences, recommendations, and unresolved questions. Design documents preserve alternatives and reasoning. Decisions record the selected constraint and its consequences. Release notes describe shipped user-facing behavior only.
  • Formal Benchmark publication has three linked layers: an Issue summary for communication and discussion, a repository report for protocol and interpretation, and a versioned raw-evidence archive for complete machine evidence.
  • A formal Benchmark report records the model/provider identity, thinking level, source revision, task and verifier identity, sample size, scheduling protocol, isolation boundary, usage accounting, failure classification, runner or manifest identity, limitations, and rerun entry point.
  • Small, reviewed, credential-free summaries, manifests, protocols, and rerun configurations may be committed. Large JSONL, logs, Sessions, candidate workspaces, caches, and result roots remain outside Git in a read-only, content-addressed archive with size and SHA-256 receipts.
  • Existing locally ignored Benchmark material is user-owned evidence. Implementation may selectively promote reviewed files, but it must not move, delete, overwrite, bulk-add, or assume the public safety of the ignored tree.
  • The first migration slice promotes the latest Luna high Pi/OpenPI result into a canonical Benchmark report and connects it bidirectionally with Issue [Benchmark] OpenPI 15/15 vs Bare Pi 10/15:少 45.2% tokens、33.6% 成本、27.7% wall-time #197. It also promotes or indexes one representative research record to validate the research path.
  • Older Issues remain intact. Initial curation may link a small set of high-value Benchmark and research Issues, but it does not rewrite their history or require immediate closure.
  • The canonical Agent contract receives only the always-needed invariants: classify reusable knowledge into canonical docs, preserve evidence states, and follow the detailed documentation contract when publishing research, decisions, or Benchmarks. Branch-specific metadata and archive rules stay behind a precise pointer to the documentation index.
  • Claude-specific guidance remains an adapter to the canonical Agent contract and does not duplicate the shared knowledge policy.
  • The implementation does not introduce a documentation framework or hosted site. A future site may render repository documents, but it cannot become a second source of truth.
  • The implementation must preserve unrelated untracked reproduction files and all ignored Benchmark assets already present in the working copy.
  • Updating Issue [Benchmark] OpenPI 15/15 vs Bare Pi 10/15:少 45.2% tokens、33.6% 成本、27.7% wall-time #197 with the canonical report backlink is a separate tracker mutation from the repository change and requires explicit external authority during execution.

Testing Decisions

  • Use one highest behavior seam: from a clean repository checkout, a reader or Agent can enter through the documentation index, identify each content category and evidence state, navigate to the first canonical research and Benchmark records, and follow valid links to their related project records and evidence references.
  • Add one repository-level static documentation contract check if no existing check covers this seam. It should validate governed document metadata, internal links, index reachability, and the required fields of the promoted Benchmark report without coupling tests to prose wording or directory implementation details.
  • A good test observes public repository behavior: whether knowledge is reachable, classified, and auditable. It does not snapshot whole Markdown files or assert sentence-level implementation details.
  • Use the existing design archive index and Benchmark result template as prior art for navigation and evidence structure; consolidate their useful contracts rather than create parallel templates.
  • Validate the promoted Benchmark report against the frozen local summary and receipts before publication. Aggregate figures, task counts, source identity, and limitations must match the existing formal run.
  • Validate the change from a tracked-files or clean-checkout view so that locally ignored files cannot make a missing public document appear present.
  • Run the repository's standard format, lint, typecheck, and test commands after the change. If Markdown link or metadata validation is newly introduced, include it in the normal validation path rather than leaving it as an undocumented manual command.
  • Audit the proposed tracked set for credentials, private Session content, oversized raw artifacts, caches, temporary directories, and accidental inclusion of unrelated user files.
  • Read back Issue [Benchmark] OpenPI 15/15 vs Bare Pi 10/15:少 45.2% tokens、33.6% 成本、27.7% wall-time #197 after any authorized backlink update. Tracker mutation success and repository validation are separate receipts.
  • Confirm that Claude-specific guidance still delegates shared policy to the canonical Agent contract and that the Agent contract points precisely to detailed policy without duplicating it.

Out of Scope

  • Building or deploying a documentation website or GitHub Pages site.
  • Migrating, rewriting, or closing the entire historical Issue backlog.
  • Requiring every Bug or Feature Issue to produce a formal document.
  • Publishing all local ignored Benchmark files or the complete result archive into Git.
  • Changing Benchmark scores, rerunning models, or creating new product-performance claims.
  • Redefining Runtime behavior, package configuration, provider selection, or OpenPI capability semantics.
  • Replacing README, source code, Runtime receipts, or release notes as their existing sources of truth.
  • Creating a database, custom CMS, or second knowledge control plane.
  • Duplicating the full documentation contract in every Agent-specific guidance file.
  • Deleting, moving, cleaning, or otherwise reorganizing user-owned local evidence.

Further Notes

  • The repository already has tracked design and research archives and a canonical cross-Agent contract. This work deepens those existing seams rather than introducing a separate documentation system.
  • Benchmark documentation and result assets currently exist locally under an ignored tree. Their presence is useful source material, not proof that they are already public or safe to commit.
  • The source baseline for implementation is main at 2a69d3f32994da4123f1312b7fa84ef3d6119be1. Two unrelated untracked reproduction files are present and must remain untouched.
  • The approved execution route is one focused fork using /spec-executor; the documentation contract, first promotion slice, Agent pointer, and static validation fit one coherent implementation context.
  • Commit, push, pull-request creation, Issue backlink edits, merge, release, deployment, raw-data publication, and real-service operations are not granted by this specification alone.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationready-for-agentSpecification is ready for implementation by an agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions