Repository navigation
fix(discovery): state the document case in the discovery routing prompt - #566
tietoumiao wants to merge 1 commit into
Conversation
|
Compared this patch with current main The disclosed +51-token cost and lack of model-in-loop evidence should remain visible for maintainer review. #408's directory resolution does not establish that this additional guidance has no value. Leaving this PR open for that scope decision; no overlapping implementation or fork-CI approval was performed in this review. |
Nicholas023
left a comment
There was a problem hiding this comment.
审查结论:REQUEST_CHANGES — 但这是 9 个 PR 中唯一不与既定工作重复的增量提案,修完可以合并
已核实为真的部分
- token 数字精确:用 js-tiktoken cl100k_base 独立核算,117→319 字符(+202)、22→73 tokens(+51),与 PR 声称完全一致
policy14/14、docs-facts6/6 通过- revert-red(撤
src/policy.ts)→ 1/14 红 - 0.5.0 基线的路由句确实仍是「Discover a capability when needed」,缺口真实存在
需要修改的 4 点
1. 触发条件比 docs 弱,CSV/TSV 半边可能不生效(阻断)
prompt 写的是 "a task needs one this table does not show, e.g. reading a .pdf …";但常驻的 ops_read_files 就在工具表里且能读 CSV/TSV 文本,模型按字面可以判定「表里已有 reader」→ 不 discover。docs/discovery.md 的写法("whose reader is absent from the tool table")更准确,请让 prompt 与 docs 使用同一句。
2. 漏掉 issue 明确要求的回退(阻断)
#408 最后一段要求提示「读失败就退回 office 技能」。本 PR 只让出了创建/编辑/渲染/导出,没有覆盖读取失败这条最需要的路径。
3. Closes #408 已过期
#408 已于 2026-11-01T17:30Z 以 COMPLETED 关闭(结论是 #536 已覆盖)。请改为 Related to #408。
4. test/docs-facts.test.mjs:69 行尾空格
git diff --check 直接报错,属仓库 PR 的常规检查项。
成本视角
51 token 里约 28 个与同一提示词下方的目录段(已列 pdf/docx/csv/tsv 及「按任务关键词查询」)重复,真正新增的是 23 token 的 Office 授权边界。方向合理,但证据只到「提示词文本正确」,没有模型在环数据(PR 已如实披露)。
处置建议
保留。 修完上述 4 点即可合并;这是本批次里唯一有独立价值的 PR。
f05855c to
8ffbe37
Compare
|
All four points from the review are addressed in
Re-measured with the encoder the suite pins (my earlier
On the duplication question: this is the only PR in the batch whose subject is a prompt-text |
8ffbe37 to
7ab74ac
Compare
Nicholas023
left a comment
There was a problem hiding this comment.
APPROVE — 技术实现已达标;token 成本的实际情况需后续验证
上一轮的 4 点意见已全部落实
- 触发条件 ✅ 改为「a
.pdf/.docx/.csv/.tsvfile whose reader is absent from the tool table」,与docs/discovery.md同句。CSV/TSV 半边不再可能被「表里已有 reader」误判。 - 读失败回退 ✅ 新增「a document read that fails falls back to it rather than retrying the reader」。
- issue 关联 ✅
Closes #408→Related to #408。 - 行尾空格 ✅
git diff --check现在 clean。
数字核对(独立复算)
新措辞实测 +279 字符 / +64 cl100k_base tokens(js-tiktoken 独立计算,与 PR body 第 60 行一致);body 第 83 行主动更正了旧数字(+202/+51),这点值得肯定。
policy 14/14 通过,中英文档成对,CHANGELOG 成对。
保留意见:token 成本需要真实场景验证
实现本身没有问题,但 +64 token / step 的收益目前只有提示词文本证据,没有模型在环数据:
- 这 64 token 里约 28 与同一提示词下方的目录段(已列 pdf/docx/csv/tsv 与「按任务关键词查询」)语义重叠;
- 真正新增的是 Office 授权边界与「读失败回退」这约 36 token;
- 它到底能省掉多少次 discovery 往返,需要真实会话数据,而不是文本推理。
技术达标,可以合并。但成本收益请维护者在合并后的真实使用中验证,再决定是否长期保留这段路由提示。
7ab74ac to
420b1f4
Compare
Related to #408
What was wrong
The on-demand
read_pdf/read_docx/read_tablereaders are only reachable throughdiscovery, yet the only routing sentence a model sees before selecting them said "when needed" and
named no format — while the host office skill advertises reading Office files directly. A model
facing a
.pdftherefore had no stated reason to discover the reader.Fixed in this PR after review (see
## Review response).The change
src/policy.ts— the discovery routing line now uses the condition the shipped docs alreadystate: a
.pdf/.docx/.csv/.tsvfile whose reader is absent from the tool table isa reason to select that reader. The weaker "when a task needs one this table does not show" wording
is gone, because the resident
ops_read_filesis in that table and reads CSV/TSV text, so aliteral reading of the old sentence could conclude no discovery was needed. A second line concedes
Office authoring to the host office skill and now also covers the failed-read fallback the
issue asked for: a document read that fails falls back to that skill instead of retrying the
reader.
test/policy.test.mjs— pins the two sentences, and keeps asserting that no unpublished readername (
ops_read_pdf/ops_read_docx/ops_read_table) is ever advertised.test/docs-facts.test.mjs— derives the always-loaded trio cost (810 tokens / 3619 bytes) fromthe catalogue so the documented figure cannot drift; a stray whitespace-only line is removed.
docs/discovery.md+docs/工具发现.md— both carry the same condition and the fallback sentence.docs/CHANGELOG.md—## [Unreleased]entry (EN + ZH).Deliberately not done: no reader schema change, no change to what is published on demand, and the
office skill boundary is stated rather than implemented.
Validation
Rebased onto
32daf5b(main after #557) and re-ran everything above on that base; the CHANGELOGconflict was resolved by keeping this entry on top of the existing
## [Unreleased]lines.policy的discovery routing names the document case and the Office-authoring boundary (#408)docs/CHANGELOG.md## [Unreleased]:已加(EN + ZH 成对)+279 字符 / +64 cl100k_base tokens,用套件同一个编码器实测;工具 schema 未变,
file/discovery的 261 tokens / 1196 bytes 预算与 114-token 目录条目 pin 仍然成立pull_request事件触发 run(fork 首跑停在action_required,需维护者批准)。下面
## CI段贴的是该 run 的 url/event/conclusion;如果之后 rebase 或补推,run 会换成新 head 的那一次,届时同步更新这一段。
Environment
32daf5bv0.5.0-alpha4(e3edb27)的发布基线用于后续开发;本 PR 的目标分支是main,因此没有夹带发布提交(
git diff --stat origin/main..HEAD= 6 文件)npm ci --ignore-scriptsReview response
针对 2026-11-02 的 reviews(4 点全部处理):
table")。如实说明:这句只是把判据写得更明确,并没有消除 CSV/TSV 的歧义——常驻的
ops_read_files描述的仍是 "Read UTF-8 ranges… Binary needs a binary reader",文本 CSV 仍可能被读成「表里已有 reader」。若维护者希望彻底闭合,可加一个 format-aware 限定词(措辞与新增 token
我按实测补,并同步 docs/CHANGELOG/两个 test)。
文档同步。
Closes #408已过期 — 改为Related to #408(issue 已于 2026-11-01T17:30Z 以 COMPLETED 关闭)。test/docs-facts.test.mjs:69行尾空格 — 已改为空行;git diff --check现在干净。数字更正:之前 PR body 写的 +202 字符 / +51 tokens 是本轮之前的措辞;本轮实测 +279 / +64,
CHANGELOG 与本文均已按实测更新。
未验证 / 风险披露
真正新增的是 Office 授权边界与失败回退。
提交前交叉循环检验
CI
该 run 由本 PR 自己的
pull_request事件触发,停在action_required(fork 首跑需维护者批准);批准后把结论补在这一行。若后续 rebase 或补推,换成新 head 的那次 run。