Repository navigation
[PR-1] M1 显著性数值修复 + EvidenceLedger 最小结构 #24
Description
本任务属于 Epic #23。共享上下文(整体目标、EvidenceLedger 完整 schema、Agent Execution Contract、Non-goals、回归基线)见 #23,不在此重复。
依赖:无(第一个 PR)。
代码定位按当前仓库版本核对;若行号漂移,以函数名 / 字段名 / 挂载点语义为准,不得因行号变化改变任务目标。
完成本 PR 需遵守 Epic #23 的 Agent Execution Contract 与 Test Execution Protocol:先写失败测试与反例夹具 → 最小实现 → 跑里程碑测试 → 跑回归基线(pytest tests/test_pipeline_contracts.py、pytest tests/test_vnext_manuscript.py)。PR 描述需列出:changed files / tests added / tests run / before-after / non-goals。
目标
两件事:① 修掉显著性的子串误判;② 建立 EvidenceLedger 的最小结构 + builder(不要求 full traceability,那是 PR-2)。
Allowed changes
agents/paper_completeness.py- significance alpha 配置读取相关文件
- EvidenceLedger builder 的最小对象(
build_evidence_ledger,见下) - PR-1 fixtures
- PR-1 tests
Forbidden changes
- 不修改
contracts/pipeline.py - 不改变
:699的存在性检查语义 - 不通过修改测试来绕过
p=1.0000反例 - 不新增从
manifest.statistical_tests文本解析显著性的路径 - 不在本 PR 实现
assert_traceable/ full traceability(留 PR-2)
M1 现状(bug,已定位三处)
paper_completeness.py:241-242:summary["statistical_tests"] = f"p={packet['p_value']}"—— 把数值塞进字符串。paper_completeness.py:589,591:p_text = _lower(manifest.get("statistical_tests"))→has_significance = "p=" in p_text or "bootstrap" in p_text or "permutation" in p_text。p=1.0000含"p="子串即判显著。:699(存在性检查)、:753(显著性问答)也读statistical_tests文本。
M1 目标行为
has_significance改为读数值:p_value is not None and p_value < alpha,数值来源 ledger 的p_value/alpha。- verdict 绑定:
verdict ∈ {confirmed, reproduced}才允许显著性支撑 Abstract 级断言;verdict ∈ {refuted, inconclusive}一律不显著、不得进 Abstract。 - α 默认
0.05,新增 envDEEPGRAPH_SIGNIFICANCE_ALPHA可覆盖(沿用现有DEEPGRAPH_*风格)。 - 699 与 753 分别处理:
:753是显著性判定,改为按数值判;:699是"是否记录过检验"的存在性检查,语义不同,本 PR 不动(改成 p<α 会把"记录了但不显著"误报成"缺失")。
取数路径
build_claim_evidence_matrix 内 packet 已在 scope(paper_completeness.py:586 经 _packet_parts(state)),p_value 就在 packet(packet.get("p_value"))。直接读即可,无需改签名。PR 描述说明数值取自 packet。
EvidenceLedger 最小 builder(本 PR 范围)
按 Epic #23 的 schema,实现:
def build_evidence_ledger(packet: dict, benchmark_summary: dict, *, alpha: float,
provenance: dict | None = None) -> dict:
"""Build the single source of truth for presentation evidence."""最小子集需含:alpha / verdict / p_value / effect_size / confidence / per_method / seed_variance / seeds(字段语义与来源见 #23 schema)。validate_evidence_ledger / assert_traceable 留 PR-2。
验收用例(单元,纯 CPU)
| # | 输入 | 期望输出 |
|---|---|---|
| M1-1 | p_value=1.0000, verdict=inconclusive |
has_significance=False;"Improves utility" 行 can_appear_in_abstract=False |
| M1-2 | p_value=0.0123, verdict=confirmed,manifest.seeds ≥3 项、per_method ≥2 键 |
has_significance=True;该行 can_appear_in_abstract=True(591 行还要 len(manifest["seeds"])>=3,夹具须同时满足) |
| M1-3 | p_value=0.0123, verdict=refuted |
can_appear_in_abstract=False(verdict 否定) |
| M1-4 | p_value=0.049,DEEPGRAPH_SIGNIFICANCE_ALPHA=0.01 |
has_significance=False(0.049 ≥ 0.01) |
| M1-5 | p_value=None |
has_significance=False,不报错 |
| M1-6(回归) | manifest.statistical_tests="p=1.0000" 文本仍在,但数值 p_value=1.0 |
以数值为准判不显著(证明不再走子串) |
| M1-7 | 753 显著性问答:p_value=1.0 |
答 "No";699 存在性语义保持不变 |
反例夹具: fixtures/m1_p_eq_one.json、fixtures/m1_significant.json(含 seeds:[0,1,2]、per_method 二键)、fixtures/m1_refuted.json。
DoD
M1-1~7 全绿;591 与 753 改数值;DEEPGRAPH_SIGNIFICANCE_ALPHA 落地;最小 build_evidence_ledger 可用;回归基线全绿;收敛分支、未确认不强推。
Activity
Protocol-zero-0 commentedon Jun 4, 2026
Completed in PR #29 on branch
epic-23-evidence-ledger.Changed files:
agents/paper_completeness.pytests/test_paper_completeness_m1.pytests/fixtures/m1_p_eq_one.jsontests/fixtures/m1_significant.jsontests/fixtures/m1_refuted.jsonTests added:
DEEPGRAPH_SIGNIFICANCE_ALPHA, missing p-value, textp=1.0000regression, reviewer significance answer, and minimalbuild_evidence_ledger.Commands actually run:
pytest tests/test_pipeline_contracts.py->12 passedpytest tests/test_vnext_manuscript.py->5 passedpytest tests/test_paper_completeness_m1.py->7 passedpytest tests/test_pipeline_contracts.py->12 passedpytest tests/test_vnext_manuscript.py->5 passedNon-goals respected:
contracts/pipeline.py.assert_traceablein this commit; that is in [PR-2] M4 EvidenceLedger full traceability checker #27.Complete Epic #23 presentation evidence gates (#29)
Protocol-zero-0 commentedon Jun 6, 2026
Done via #29 (squash-merged into master 2026-06-06). 验收用例与回归基线均在 PR #29 全绿。