feat(host): workspace index infrastructure behind an opt-in switch

[muzimu217]

PR 1 of the split agreed in #502 — Index infrastructure only, nothing else. Based on the latest main, one clean branch.

What's in

• FTS5 store under <data>/index/index.db (trigram tokenizer, fixed budgets: 50k files / 1MB per file / 2GB total), deliberately separate from pi.sqlite; integrity or schema failure quarantines the old file and recreates an empty store
• Lifecycle RPCs index.status / index.rebuild / index.clear, workspace-root-scoped (INDEX_ROOT_OUTSIDE_WORKSPACE otherwise), metadata only — never file contents; rebuild/clear write an audit entry
• Index page in the existing Workspace settings group: one health card (status / file count / size / unreadable count / last update), rebuild & clear actions, a build-progress line while building, empty and error states
• One switch, indexGrepBoost, default off. Turned on, it indexes newly opened workspaces in the background (workspace.set stays fast; the health card polls while building). Turning it on arms the current workspace immediately
• Opening the store never blocks host startup — a failed open degrades to a disabled store and the RPCs report unavailable
• i18n ×8, spec updates (en/zh) and E2E coverage for exactly this surface

What's deliberately out (per the split in #502)

• Grep is untouched. tools/mod.rs has zero changes; tools execute exactly as before whether or not an index exists. The fast path + freshness changes are PR 2, to be reviewed on "does it ever change Grep's answer".
• No workspace-watch watcher, no mark_stale wiring, no stats surface. The three stats leftovers named in feat(settings): fold the workspace index into the Workspace group #502 (stats.rs, stats-real-rpc-reconcile.mjs, idx_turns_started) are not present — this branch never carried them.

The two blockers named in #502

• Conflicts against main (SettingsPage.tsx, 06-settings-ia.md): resolved by starting from the current main instead of stacking.
• workspace_set_auto_indexes… wall-clock polling: the background build's join handle is now registered on the app state (index_builds), and the test awaits that handle directly, then asserts the settled status. No wall-clock polling. (For what it's worth: the polling version was also hiding a real platform race — watcher events landing mid-build flip the root to stale, which the test then never sees become fresh. That race belongs to PR 2's surface; PR 1 has no watcher at all.)

Verification

• cargo test -p host-core --locked: 549 passed / 0 failed (incl. new RPC tests: switch defaults + type validation, auto-index gated on the switch via the awaited handle, rebuild/clear audit, store self-healing, budget, multi-root scoping)
• cargo fmt --check clean; cargo clippy -p host-core -- -D warnings clean for every file this PR touches
• desktop suite 2465 passed / 0 failed; i18n catalogs 25 passed (key sets equal ×8)
• pnpm docs:check: 487 pages en/zh paired; architecture budget passed
• E2E: pnpm test:e2e:index → PASS E2E-INDEX-status-rebuild-clear
  • E2E: NOT RUN (full desktop UI suites)
  • Reason: this PR's E2E surface is the RPC lifecycle, which is automated above; UI-side scenarios are documented in the e2e plan
  • Alternative validation: desktop unit tests assert the Index page wiring, switch gating and locale keys; E2E-INDEX-settings-health-card documented as pending UI automation
  • Remaining risk: none known beyond the documented UI-automation gap

Real-workspace measurement (details in the comments)

Driven the shipped binary (cc726590) over stdio RPC against a real checkout of this repository (64k files incl. node_modules; isolated PI_DESKTOP_DATA_DIR):

• workspace.set with the switch off: 0.6 ms, zero index rows — no cost while off
• switch on: 2.6 ms to return; background build lands fresh in 3.0 s — 1,644 source text files / 19.2 MB, 0 errors (node_modules, target/, build outputs correctly excluded)
• foreground index.rebuild: 5.7 s; steady-state index.status latency 0.5–0.7 ms
• index.db: 115.5 MB (~6× content: FTS5 trigram postings + stored bodies; within the 2 GB budget — a contentless-FTS follow-up can shrink this substantially)

Note on pre-existing main breakage (not from this PR)

Typecheck and clippy each currently fail on untouched files from today's main, reproducible on pristine upstream/main (7a59a25) with the same commands:

• pnpm --filter @pi-desktop/desktop typecheck → electron/main/user-login-path.ts(28,74): TS2345 (from 574c78a; the JS job on the fix(desktop): classify crash dumps and keep them in the data directory #689 merge commit already shows failure)
• cargo clippy -p host-core -- -D warnings → user_skills.rs(921): useless_format

I left both alone to keep this PR single-purpose — happy to send a one-line fix separately if useful.

Closes nothing; part of the #502 split (PR 2: Grep fast path + freshness, PR 3: stats RPCs).

[muzimu217]

为方便 review，补一份实测数据和一份diff 导航。数据全部来自本 PR 的交付二进制（cc726590）+ 一份本仓库的真实检出（含 node_modules 约 4.9 万文件、构建产物等共 6.4 万文件，隔离的 PI_DESKTOP_DATA_DIR，不触碰真实数据）。

实测（真实工作区，stdio RPC 直驱）

操作	结果
workspace.set（开关关，默认态）	0.6 ms 返回；index.status 为空——关时零索引开销
settings.set 打开开关	2.6 ms 返回，后台构建启动
后台构建 → fresh	3.0 s（轮询粒度 200ms）：1,644 个源码文本文件 / 19.2 MB 内容，0 错误；node_modules、target/、构建产物被 ignore 口径正确排除
index.rebuild 前台全量	5.7 s（同规模）
index.status 稳态延迟	0.5–0.7 ms（×5 采样）
index.clear	2.3 s，清后状态为空
index.db 体积	115.5 MB

两点如实说明：

• 体积比：19.2 MB 内容 → 115.5 MB 存储（约 6×）。这是 FTS5 trigram 倒排 + 正文全存的已知特征，在 2 GB 总预算内；后续把 FTS 换成 contentless 表（只留倒排、不存正文）可以显著压下来，属于独立的后续改动，不混进本 PR。
• 可见集口径：健康卡的「已索引文件」是 ingest 成功的文本文件数；可见但未 ingest 的行（二进制、超限）只参与完整性、不进显示，所以 1,644 会小于 find 出来的源码文件数。

diff 导航（44 文件）

核心逻辑集中在两个新文件，其余多为薄接线：

• 存储层（值得细读）：crates/host-core/src/index.rs（状态机、爬取、预算、quarantine 自愈、ensure 幂等）+ index/fts.rs（FTS5 DDL 与读写）
• RPC 层：rpc/mod.rs 的改动只有三块——3 个 index case、workspace.set/settings.set 的开关接线（含 join 句柄注册）、3 条测试；state.rs（store 持有 + 开库失败降级 + index_builds 句柄表）；main.rs 仅一行 mod index;
• 前端：IndexPage.tsx + MetricTile.tsx 为新增；SettingsPage.tsx/settings-search.ts/api.ts/session-ipc.ts 各约 10–20 行接线
• 契约：workspace-index.ts/protocol.ts/errors.ts/types/settings.ts
• 机械面：i18n ×8（键集全等）、docs en/zh、e2e-index.mjs

测试 ↔ 不变量

测试	守护
workspace_set_auto_indexes_only_while_the_switch_is_on	开关关绝不建索引；开后后台构建落 fresh（等 join 句柄，非墙钟轮询）
index_grep_boost_defaults_off_and_rejects_non_booleans	默认关；非布尔补丁被拒
index_rebuild_and_clear_are_audited	两个破坏性操作都写审计
index.rs 内嵌 9 条	路径规范化、预算落 skipped_over_limit、损坏库隔离重建、多 root 命名空间隔离、building 残留自愈、进度只在 building 期出现

对照 #502 裁决清单：FTS5 store ✓ / 三个 lifecycle RPC ✓ / Workspace 组 Index 页 ✓ / 单开关默认关 ✓ / 不接 Grep（tools/mod.rs 零改动）✓ / 开库失败不挡 Host ✓ / stats 三残留不存在 ✓ / 等句柄不碰墙钟 ✓。

---

更新 1 — 与最新 main 的兼容：main 前进到 aad46adb（含 #692/#695/#702/#703/#704）后本 PR 一度 CONFLICTING，已合并解决（三处：settings.css、e2e 汇总表、package.json——均为双方各自新增的样式段/行首条目，两边保留；顺带清掉了本 PR 自己一个无消费者的 CSS 类）。合并后门禁重跑：host 576/0、desktop 2474/0、typecheck 0 错（#692 修好了 user-login-path.ts 的 TS2345，基线比上一版更干净）、i18n 25、docs 490 页、架构预算、cargo fmt --check、E2E-INDEX-status-rebuild-clear 全部通过；PR 现为 MERGEABLE。

更新 2 — 多轮稳定性与查询测试（同一真实工作区，隔离 data dir，合并后二进制 0e7ce522）：

维度	结果
重建确定性	5 轮 index.rebuild，fresh 行逐字节一致（fresh / 1644 文件 / 19.2 MB / 0 错），单轮 5.4–6.1 s
查询稳定性	50 次 index.status：50 次返回 payload 完全相同；延迟 min/P50/P95/max = 0.39 / 0.48 / 0.59 / 0.68 ms
build→clear→build ×3	每轮 cleared=1、重建后 fileCount / indexedBytes 零漂移；clear 后 DB 文件不收缩（SQLite freelist 页复用语义），行数据无残留
构建期间并发查询	rebuild 进行中 20 次 index.status 全部即时应答、0 错误，可观察到 building 态，构建最终落 fresh——查询与构建互不阻塞

[muzimu217]

你好，我们注意到 09-26 生效的贡献范围调整（外部 PR 暂只收 fix/perf，CONTRIBUTING R6.1）。#697 是按您 09-20 在 #502 给出的拆分方案开出的 PR 1（Index 基础设施），分支一直在持续 rebase 最新 main（当前 head ebed5a4c，四项 checks 绿）。

窗口期内我们不想让它占用您的审阅精力，您方便时给一个处置意向即可：

• a) 窗口解除后纳入 review 队列（我们继续把分支保持可合状态）；
• b) 按新政策转成 feature issue、由您排期实现（fork 分支与实测数据可作为参考实现保留）；
• c) 依政策关闭（分支我们自行保留，完全理解）。

另想顺带确认：被拆分取代的 #502 是否可随本裁定一并关闭？其内容已由 #697（PR1）取代，其余部分本就留待后续窗口再议，挂着一个陈旧分支只会给 triage 添噪音。

无论哪种处置我们都配合，一条回复即可，不占您时间。谢谢！