Skip to content

docs(benchmark): 落地 OpenPI 54-cell ARM64 诊断报告 - #271

Open
627150795 wants to merge 2 commits into
openpi-dev:mainfrom
627150795:docs/issue-46-benchmark-report-upstream
Open

docs(benchmark): 落地 OpenPI 54-cell ARM64 诊断报告#271
627150795 wants to merge 2 commits into
openpi-dev:mainfrom
627150795:docs/issue-46-benchmark-report-upstream

Conversation

@627150795

@627150795 627150795 commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Problem

Issue #46 已经记录了 54-cell ARM64 source-build-derived 诊断结果,但原 PR 将报告放在 docs/research/,缺少 Benchmark 类别所需的治理元数据和可检索的证据边界。

Value

把公开结果整理为可版本化、可引用的 Benchmark 记录,明确冻结条件、结果、限制和证据状态;同时明确这不是官方 Terminal-Bench 成绩,也不把小样本观察写成总体排名。

Approach

  • 将报告移至 docs/benchmarks/OPENPI_PI_OMP_54_CELL_DIAGNOSTIC.md
  • 增加 draft 状态、创建/复核日期、适用范围、模型与 thinking level、任务、样本量、验证器、隔离条件、失败分类、核算方式和复跑边界。
  • 新增 docs/benchmarks/receipts/openpi-issue-46-arm64-54-cell-v1.md,记录 18 条三次重复结果明细及六组可重算汇总。
  • 将 Issue 和 release 链接统一为 openpi-dev/openpi 的官方地址。
  • 不提交原始 JSONL、Sessions、日志、candidate workspace、缓存或凭据;收据保持 draft,直到有独立复核入口。
  • 本 PR 只修改 Benchmark 文档和无凭据收据,不修改 OpenPI 运行时代码。

Validation

  • 已基于当前 maina15a0d88837f580187cbb02d7fcab91bff8f0d34)完成 rebase。
  • git diff --check 通过。
  • 收据一致性校验通过:18 条明细、六组 pass/fail/indeterminate 汇总全部一致。
  • 已核对 Issue [Benchmark] OpenPI 0.3.0 / Pi / OMP:GLM-5.3 与 GPT-5.6 Luna 54-cell ARM64 派生诊断 #46 的公开结果、冻结版本、任务 revision、调度 hash、lock fingerprint 和安全边界。
  • Windows 本地全量格式检查仍受仓库既有 CRLF 基线问题影响;本次没有修改运行时代码。

Impact

  • 用户可见行为:新增一份位于 docs/benchmarks/ 的 Benchmark 报告及边界收据。
  • 模型可见工具与运行时:无变化。
  • 配置、artifact 和兼容性:无变化。
  • 该记录仍是 draft,不宣称独立复算或官方 Benchmark 接受;原始证据包仍需由维护者提供可审阅入口。

Fixes #46

@tt-a1i

tt-a1i commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

This Benchmark report remains open under #277. Its merge gate is source-evidence review against Issue #46: frozen revisions, task/model/verifier identities, accounting, failure classification, limitations, and evidence references must all read back correctly. Documentation alone is not runtime or official Benchmark acceptance.

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking evidence finding on exact head 8841a68d81bef7efe481aa57833af35201921808:

The prose matches Issue #46, and the underlying GLM/Luna evidence directories still exist locally, but this PR provides no retrievable evidence reference from which another reviewer can inspect or recompute the tables. Issue text, hashes, and a local-only archive are provenance pointers, not independently reviewable evidence. Under #278's Benchmark contract, keep this report unmerged until it is moved to docs/benchmarks/, given the governed metadata/status, and linked to a stable credential-free archive identity or bounded receipt that reviewers can retrieve. Do not bulk-publish raw Sessions, logs, credentials, or candidate workspaces.

Also replace the historical tt-a1i/openpi Issue/release links with the canonical openpi-dev/openpi identity (or explicitly label them as historical redirects).

No runtime finding; this is a publication/evidence-state gate.

@627150795
627150795 force-pushed the docs/issue-46-benchmark-report-upstream branch from 8841a68 to 14d0f19 Compare August 29, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Benchmark] OpenPI 0.3.0 / Pi / OMP:GLM-5.3 与 GPT-5.6 Luna 54-cell ARM64 派生诊断

2 participants