docs(benchmark): 落地 DeepSeek 24-cell 诊断报告 - #272
Conversation
|
This Benchmark report remains open under #277. Its merge gate is source-evidence review against Issue #22: frozen revisions, task/model/verifier identities, accounting, failure classification, limitations, and evidence references must all read back correctly. Existing CI is repository validation, not independent validation of the Benchmark conclusion. |
tt-a1i
left a comment
There was a problem hiding this comment.
Blocking evidence finding on exact head a6112a0d58e83e6439cbc38c66de31425af2aed5:
The prose matches Issue #22, and the 24-cell local evidence directory still exists, but openpiContentSha256 alone is not a retrievable source revision and the PR intentionally omits the artifacts needed to inspect or recompute the 10/12 vs 6/12 tables. Under #278's Benchmark contract, keep this report unmerged until it is moved to docs/benchmarks/, given the governed metadata/status, linked to a retrievable credential-free evidence receipt/archive identity, and tied to a source commit or immutable source package. Do not bulk-publish raw Sessions, traces, credentials, or candidate workspaces.
Also replace the historical tt-a1i/openpi Issue links with the canonical openpi-dev/openpi identity (or explicitly label them as historical redirects).
No runtime finding; this is a publication/evidence-state gate.
Problem
Issue #22 已经记录了 24-cell DeepSeek 0731 对照与收敛消融,但结果、冻结条件、机制命中情况和统计限制尚未进入仓库内的正式报告,后续引用和复核不方便。
Value
把已有公开收据版本化,保留结果边界和失败/超时信息,避免把本轮小样本观察误读成官方成绩、稳定提速或总体排名。
Approach
docs/research/OPENPI_BARE_PI_DEEPSEEK_24_CELL_DIAGNOSTIC.md。Validation
git diff --check通过。Impact
Fixes #22