docs(benchmark): 落地 OpenPI 54-cell ARM64 诊断报告 - #271
Conversation
|
This Benchmark report remains open under #277. Its merge gate is source-evidence review against Issue #46: frozen revisions, task/model/verifier identities, accounting, failure classification, limitations, and evidence references must all read back correctly. Documentation alone is not runtime or official Benchmark acceptance. |
tt-a1i
left a comment
There was a problem hiding this comment.
Blocking evidence finding on exact head 8841a68d81bef7efe481aa57833af35201921808:
The prose matches Issue #46, and the underlying GLM/Luna evidence directories still exist locally, but this PR provides no retrievable evidence reference from which another reviewer can inspect or recompute the tables. Issue text, hashes, and a local-only archive are provenance pointers, not independently reviewable evidence. Under #278's Benchmark contract, keep this report unmerged until it is moved to docs/benchmarks/, given the governed metadata/status, and linked to a stable credential-free archive identity or bounded receipt that reviewers can retrieve. Do not bulk-publish raw Sessions, logs, credentials, or candidate workspaces.
Also replace the historical tt-a1i/openpi Issue/release links with the canonical openpi-dev/openpi identity (or explicitly label them as historical redirects).
No runtime finding; this is a publication/evidence-state gate.
8841a68 to
14d0f19
Compare
Problem
Issue #46 已经记录了 54-cell ARM64 source-build-derived 诊断结果,但原 PR 将报告放在
docs/research/,缺少 Benchmark 类别所需的治理元数据和可检索的证据边界。Value
把公开结果整理为可版本化、可引用的 Benchmark 记录,明确冻结条件、结果、限制和证据状态;同时明确这不是官方 Terminal-Bench 成绩,也不把小样本观察写成总体排名。
Approach
docs/benchmarks/OPENPI_PI_OMP_54_CELL_DIAGNOSTIC.md。draft状态、创建/复核日期、适用范围、模型与 thinking level、任务、样本量、验证器、隔离条件、失败分类、核算方式和复跑边界。docs/benchmarks/receipts/openpi-issue-46-arm64-54-cell-v1.md,记录 18 条三次重复结果明细及六组可重算汇总。openpi-dev/openpi的官方地址。draft,直到有独立复核入口。Validation
main(a15a0d88837f580187cbb02d7fcab91bff8f0d34)完成 rebase。git diff --check通过。Impact
docs/benchmarks/的 Benchmark 报告及边界收据。draft,不宣称独立复算或官方 Benchmark 接受;原始证据包仍需由维护者提供可审阅入口。Fixes #46