Skip to content

refactor(ptc): ship TypeScript sources instead of tsc output - #443

Open
yanfei0725 wants to merge 12 commits into
T-Auto:mainfrom
yanfei0725:main
Open

yanfei0725 wants to merge 12 commits into
T-Auto:mainfrom
yanfei0725:main

Conversation

@yanfei0725

@yanfei0725 yanfei0725 commented Oct 27, 2026 •

Copy link
Copy Markdown

这次只改一件事:把包从「压缩包 / 编译产物」变回可 review 的源码

上一版提交进来的 src/ptc/*.js 是 tsc 编译结果,外加 11 份手写 .d.ts。diff 里只能看到 exports.x = x 和 (0, mod.fn)(),reviewer 无法判断改了什么。这个 PR 把 13 个模块还原成 TypeScript 源,带原来的注释和类型。

目录

packages/ops-ptc/
├── src/ptc/            13 个 .ts(唯一实现源)+ agentbox.worker.cjs + schema.json
├── test/               12 个 .mjs 用例
├── scripts/build.mjs   tsc → dist,再拷两个非 TS 资源
├── tsconfig.json
├── vendor/README.md    说明 es.exe 由发布产物携带,不入库
├── bench/ docs/
└── package.json

没有 .tgz、没有 .tar.gz、没有 .exe、没有 dist/、没有 node_modules/。.gitignore 只拦这三类产物,src/ 下任何 .ts / .js 都不受影响。

构建变成真编译

build.mjs 原来是 cpSync(src, dist) —— 因为源就是产物。现在跑 tsc -p tsconfig.json,然后只拷两样不进 tsc 的东西:手写的 agentbox.worker.cjs(proc.ts 按 __dirname 直接拉起)和 schema.json(宿主当纯数据读)。

相对 import 去掉了 .js 后缀('./engine-load.js' → './engine-load'),这样同一份源既能被 tsc 编译,也能被 node --experimental-strip-types 直接跑 —— 测试套件里有几支就是直引 src/ptc/*.ts 的。

单文件 1000 行

原 inproc.ts 1023 行,超了。按枚举接缝拆开:

文件 行数 内容
inproc.ts 884 AST 定位、改写、审计编排
enumerate.ts 269 walk / Everything 分级寻址、30s 缓存、子串预筛
inproc-error.ts 21 共享错误类

enumerate / clearEnumCache / EnumerateResult / LocateStrategy 仍从 ./inproc 导出,包对外 import 路径不变。当前最大文件 884 行,全部提交都在 1000 行以内。

验证

npm run build     tsc → dist(worker + schema 单独拷)
npm run verify    全绿

verify 展开后:22/22 pipeline regression、11 项 audit root cache、28 项 MCP 契约、safety / worker / protocol / mode-audit / locate-parity 各套件全过。导出面与旧 dist 逐模块比对一致(dist-shape 只对基线里存在的模块比对,新增的 engine-load / enumerate 不算导出变更)。

一个提醒

src/ptc/ 的手写 .d.ts 不在了 —— 声明文件现在由 tsc 的 declaration: true 生成,跟着类型走。如果维护者希望保留手工声明(比如为了压注释体积),那是另一个决定,这里按「源码即唯一真相」处理。

@yanfei0725 yanfei0725 changed the title 引入基于 napi 与原生 STDIN 的高性能 PTC 引擎 (Fixes #442) feat(ptc): ops-ptc 2.0.6 — trimmed resident tool schemas Oct 29, 2026
@yanfei0725 yanfei0725 changed the title feat(ptc): ops-ptc 2.0.6 — trimmed resident tool schemas feat(ptc): ops-ptc 2.0.8 — honest audit line counts, real sandbox mode, non-retryable fences Oct 30, 2026
…azy engine

把「调用次数」当主要成本:每次工具往返都要重发整个上下文,省一次往返胜过省几百字节回执。

- locate 附赠命中正文:body 缺省 true(≤8 KiB 随 hits[].body 返回)、数字(上限 32 KiB)、
  false(只给行号)。不附时给 bodyOmitted 与 readHint(约 70 B),把第二轮从「自己算
  offset 试错」变成「照抄参数」。40 行/1583 B 的符号 437 B + 一轮 -> 1911 B 一次到位;
  182 行/12.7 KB 的符号 -> 473 B + readHint。
- 回执 24 KiB 闸门:先退正文并补 readHint -> 再按稠密前缀削 hits/files/edits 并报
  ${key}Omitted 与 nextOffset -> 最后只留计数。27 KB 输入 -> 943 B。
- 命中字段瘦身:删 chars/bodyBytes/fileBytes/savedRatio 与顶层 contextBytes,
  find 的 savedTokensRatio 同步删除。每命中 -82 B 约 -21 token。
- 引擎懒加载:inputSchema 改由 dist/ptc/schema.json 承载,引擎引用挪到首调用。
  会话启动导入 19.01 ms -> 4.56 ms,启动期 require 缓存不再有 @ast-grep/napi。

README 与 AUDIT 同步到 2.1.0。
@yanfei0725 yanfei0725 changed the title feat(ptc): ops-ptc 2.0.8 — honest audit line counts, real sandbox mode, non-retryable fences feat(ptc): ops-ptc 2.1.0 — locate body carry, receipt size gate, field trim, lazy engine Oct 30, 2026
@yanfei0725 yanfei0725 changed the title feat(ptc): ops-ptc 2.1.0 — locate body carry, receipt size gate, field trim, lazy engine feat(ptc): ops-ptc 2.1.1 — placeholder write gate, dual-anchor engine load, stale-index walk retry Oct 30, 2026
…tead of a tarball

The PR carried ops-ptc-2.1.1.tgz, so the engine could not be read or reviewed in
the diff at all.

- packages/ops-ptc/src/ptc/    the CommonJS implementation plus its hand-written .d.ts
- packages/ops-ptc/test/       AST, worker, protocol, pipeline, mode-audit regressions
- packages/ops-ptc/scripts/    build.mjs copies src/ to dist/ (same bytes, no compiler step)
- packages/ops-ptc/vendor/     placement note for es.exe; the binary itself stays out
- drop the tarball and the duplicated root README and AUDIT files
- test fixes: Everything checks accept E_BIN_MISSING, locate asserts the 8 KiB body gate
- .gitignore: release archives, vendored .exe and test scratch trees
@yanfei0725 yanfei0725 changed the title feat(ptc): ops-ptc 2.1.1 — placeholder write gate, dual-anchor engine load, stale-index walk retry feat(ptc) Oct 30, 2026
@yanfei0725 yanfei0725 changed the title feat(ptc) feat(ptc): ops-ptc 2.1.1 as source under packages/ Oct 30, 2026
… output

packages/ops-ptc/src/ptc/ used to hold tsc output (.js + hand-written
.d.ts), so the PR diff showed only exports.x = x and (0, mod.fn)().
The 13 modules are back as TypeScript with their types and comments
intact; dist/ is generated by tsc and stays out of the tree.

Build is now real compilation instead of a copy:
scripts/build.mjs runs tsc, then copies the two non-TS resources
(agentbox.worker.cjs, schema.json). Relative imports drop the .js
suffix so the same sources run under both the tsc build and the
node --experimental-strip-types test harness.

inproc.ts is split at the enumeration seam: enumerate.ts carries the
walk/Everything strategy, the 30s cache, and the substring prefilter;
inproc-error.ts carries the shared error class. Largest file is now
884 lines, down from 1023.

npm run verify passes: 22/22 pipeline, 11 audit-root-cache, 28 MCP
checks, plus the smoke and safety suites.
@yanfei0725 yanfei0725 changed the title feat(ptc): ops-ptc 2.1.1 as source under packages/ refactor(ptc): ship TypeScript sources instead of tsc output Oct 30, 2026
@OldLigant OldLigant assigned OldLigant and unassigned OldLigant Oct 30, 2026
@OldLigant
OldLigant self-requested a review October 30, 2026 15:46
@OldLigant OldLigant self-assigned this Oct 30, 2026
…t_* and drop the MCP exit

- Move the prototype out of packages/ops-ptc into src/ast as reviewed
  repository source: dsh-ops/ast* package exports, the ops-ast bin,
  bench/ast benchmarks and bilingual docs. The standalone-package shape
  could not support tool registration.
- Rename the five tools ops_ptc_* -> ops_ast_*: the PTC naming collides
  with the existing dsh-ops-ptc row; titles drop the PTC branding and the
  .ops-ptc.lock / .ptc- artifact names follow.
- Fix the locate body parameter: the host assertSupportedJsonSchema
  rejects type arrays, so the boolean/number dual shape becomes an
  exact-one oneOf; all five inputSchemas now pass the real host
  validator.
- Remove the MCP stdio exit and its two suites: an independent endpoint
  serves handlers with no sandbox policy, credential fence or host
  authority -- the bypass dsh-ops itself removed with FastCtx in 0.3.2.
  createHostTools/registerHostTools with a trusted bridge stay the only
  model-facing wiring point.
- The engine remains unregistered with the running host;
  progressive-discovery wiring (discovery row pool plus the trusted
  bridge) is the follow-up.
@OldLigant

OldLigant commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

按 review 结论对这个 PR 做了一轮重整(提交 941a9e9,已直接推送到本 PR 分支),说明一下动机和取舍。

1. 文件整理

最初以压缩包形式提交、后来落到 packages/ops-ptc 独立包——这两种形态都撑不起工具注册,也不符合本仓库结构(原生能力都在 src/ 单包内)。现在整理为:

  • 引擎源码 src/ast/(.cts 编译为 lib/ast/*.cjs),基准移至 bench/ast/,11 套测试改名 test/ast-*.test.mjs 随常规门禁运行
  • 包导出 dsh-ops/ast、/ast/inproc、/ast/tools,ops-ast CLI 入口
  • 双语文档 docs/ast-engine.md / ast-engine.zh.md

2. 工具重命名 ops_ptc_* → ops_ast_*

原名与现有 PTC 行(dsh-ops-ptc)撞名,"PTC 进程内 AST 改写"这类标题也名不副实。顺带修复一个硬伤:locate 的 body 参数用了 type: ['boolean','number'],而宿主 assertSupportedJsonSchema 直接拒绝 type 数组——已改为 exact-one oneOf,五个工具的 inputSchema 现已全部通过真实宿主校验器验证。

3. 披露方式:按渐进式披露设计(声明就绪,接线随后续 PR)

这五个工具是细分场景工具(大仓库符号级定位 / 结构化改写 / 语法审计),不是通用编辑:schema 大、使用频率低。若设为常驻,每轮请求都要在 tools 里带全五份声明,与本项目 token 效率的立身之本相悖;README 对 discovery 平台的定位(pdf / docx / 超多文件批量修改)恰好就是这类能力的归处。因此:

  • tools.cts 保留 tools_addition 声明(load: 'on-demand'、能力关键词、minRepoFiles: 800)
  • adapter.cts 的宿主 Outcome 信封(completed / partial / rejected + effectState / retry)是唯一模型侧接线点
  • 引擎当前未注册进运行中的宿主,不给任何会话增加工具

4. 移除 MCP stdio 出口

跨 harness 的 AST / lint 类工具普遍以 MCP 形态存在是合理的:面向多客户端、协议通用、由服务器自管文件系统——本 PR 的原型也确实由此而来。但在 DSH 插件里这条路走不通:宿主权限模型(per-call sandboxPolicy、凭据防护、审计、取消)必须罩住每一次副作用,而独立 stdio 端点调用 handlers 时会全部绕过——原型文档自己也写了"使用 MCP 也不会自动继承原包的权限实现"。

这里有现成前例:dsh-ops 0.2.0 就是以 vendor FastCtx(一个 MCP 运行时)起家的,0.3.2 refactor! 把 FastCtx 连同 MCP 客户端一并移除,全面转向宿主原生的 per-call 策略。因此本次将 MCP server、其两个测试套件和 ./ast/mcp 导出全部删除;ops-ast CLI 保留为开发工具,模型可见的工具面只保留 createHostTools / registerHostTools + 可信桥这一条路。

5. 后续接线待办(把工具真正注册成渐进式披露)

  • 实现可信 HostBridge:会话 sandboxPolicy 解析(read-only / workspace-write / full)、read-only 拒绝写模式(mutates)、AbortSignal 传递、凭据栅栏复核
  • 经 registerHostTools 把五个定义挂进 discovery 行的 provider 模型:作用域所有权、presentation 判定、readiness 重校验,复用现有 ops_discover_tools 入口
  • DISCOVERY_TERMS / allowedTools 增项:关键词复用 tools.cts 的 discover.keywords;确定默认池边界(locate / find / audit 只读可进,inproc / refactor 写路径按沙箱策略拒绝)
  • dsh-plugin.json 声明(x-components 的 tools 清单)
  • 文档同步:配置参考、组件参考、docs/工具发现.md
  • 新增测试:read-only 拒写、跨作用域隔离与不继承、discovery 增量装配(新 agent 零重复探测)、宿主兼容冒烟回归

验证

node test/run.mjs ast- 11/11 通过,package-source 通过。本机全量门禁另有 42 个失败,均为预存环境问题(worker ACCESS_DENIED、linux-* 套件平台不匹配、时序抖动),与本改动无关,建议以 CI 为准。

OldLigant and others added 4 commits October 31, 2026 06:42
reparseEscape walked to the volume root, so on macOS — where tmpdir sits
behind /var -> /private/var — every candidate under a tmpdir workspace was
named as an escape and silently skipped: inproc locate/refactor always
returned E_INPROC_NO_MATCH with no firstError, while win32/linux passed.
The lstat walk now stops at the workspace boundary (the enumerated root,
or cwd for a caller-named file); links at or above the boundary are the
caller's authority decision, links inside the workspace are still refused.
Windows junctions in the smoke suite pin both sides of that line.
The pinned 0.2.0-rc.2 host types declare `agent/created` as
`(payload) => Promise<undefined> | undefined`, so a void block body fails
`tsc` and `npm run build` aborts before it copies assets or writes the PTC
declarations. Three call sites had a void body; each now returns undefined
explicitly, which is what the runtime already did.
Registers the AST engine that landed under `src/ast` as host tools instead of
leaving it unregistered: `src/execution/ast-inproc.ts` is dispatched from both
execution bridges (`src/execution/files.ts`, `src/execution/jobs.ts`) before any
worker spawn or policy resolve, so a locate/rewrite/audit never starts a
subprocess. The five rows (`ast_locate`, `ast_find`, `ast_refactor`,
`ast_inproc`, `ast_audit`) enter the catalogue with their schemas spread from
`src/ast/schema.json`, so `required` cannot drift from the engine's own.

Availability follows the file rows: locate and inproc stay resident, find,
refactor and audit arrive through discovery. Only `ast_refactor` mutates, so
the other four are concurrency-safe.

The adapter is the trusted bridge and carries the native worker's four fences in
the same order: session identity plus absolute cwd, one authoritative policy
resolve with a mode whitelist, the credential fence before anchoring, and write
intent requiring workspace-write. Outside danger-full-access every named file
and enumeration root must stay inside the workspace or session cwd by real path,
so a junction that points outside is refused rather than trusted by name.
Receipts are canonicalised to the host vocabulary and degraded by item (bodies
first, then a dense prefix with nextOffset) past 24 KiB.

`test/ast-inproc.test.mjs` (18 checks) covers the dispatch, the fences, the
receipt shape, enumerate caps and the unresolved-placeholder gate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants