diff --git a/.alfonso/plans/codegraph-benchmark-replication.md b/.alfonso/plans/codegraph-benchmark-replication.md new file mode 100644 index 00000000..602eb384 --- /dev/null +++ b/.alfonso/plans/codegraph-benchmark-replication.md @@ -0,0 +1,87 @@ +# CodeGraph benchmark replication plan for AFT + +## 1. Metrics to replicate + +Replicate the deterministic, no-LLM retrieval-quality eval from `codegraph/__tests__/evaluation/`: + +- **Recall** over expected symbols, using the same pass rule as CodeGraph (`recall >= 0.5`). +- **MRR** from the first ranked result that matches an expected symbol or expected file. +- **Precision@k** for k = 1, 5, 10. CodeGraph's current scorer does not expose P@k, but the user asked for it and it is compatible with the same ranked result list. +- **Found/missed symbols** per case. +- **Real wall-clock latency** around the actual tool dispatch. The report will include per-case latency samples plus median and p95 latency at driver summary level. With `--runs > 1`, each query gets per-query median/p95; with the default single run those values equal the single dispatch time. + +Keep CodeGraph's `nodeCount`, `edgeCount`, and `edgeDensity` fields optional. AFT's retrieval tools do not expose graph edge counts for `aft_search`, `grep`, or ripgrep, so those fields will remain absent instead of fabricated. + +## 2. Corpus choice + +Use three corpus sources: + +1. **`codegraph` (default for apples-to-apples AFT runs):** an AFT-side translation of CodeGraph's 12 test-case shapes. It preserves CodeGraph's split between exact symbol lookup (`searchNodes`) and broader context exploration (`findRelevantContext`), but rewrites Elasticsearch-specific symbols (`TransportService`, `RestController`, etc.) to equivalent symbols in this repository (`BinaryBridge`, `BridgeOptions`, `handle_semantic_search`, etc.). Each rewritten case records its `sourceCaseId` and a note explaining the substitution. +2. **`codegraph-original`:** a JSON copy of the exact CodeGraph structured corpus. This is useful when someone points the harness at Elasticsearch or another checkout containing those symbols. It is expected to fail or be skipped on `opencode-aft`, so it is not the default run for this repo. +3. **`aft`:** small AFT-native supplemental cases for tool-surface coverage that CodeGraph does not have one-to-one (outline/zoom/navigate-oriented cases). Custom corpus files can also be loaded by path with the same schema. + +This keeps the publishable comparison honest: `codegraph-original` is the literal upstream corpus; `codegraph` is the translated corpus used to run the same methodology against AFT itself. + +## 3. Tool mapping + +| CodeGraph eval API/tool | AFT equivalent in this harness | Notes | +| --- | --- | --- | +| `searchNodes(query, { limit, kinds })` | `aft_search` (`semantic_search` bridge command with `top_k`) | Use symbol/file/kind metadata from AFT hybrid results. `kinds` is retained as corpus metadata and reported, but AFT does not currently filter semantic search by kind. | +| `findRelevantContext(query, { searchLimit, traversalDepth, maxNodes })` | `aft_search` by default; optional corpus cases may request `aft_outline`, `aft_zoom`, or `aft_navigate` | AFT has separate focused tools instead of one subgraph-returning context API. For apples-to-apples scoring, the ranked retrieval result is still normalized into the same item list. | +| CodeGraph `node`/source inspection | `aft_zoom` | Only for cases with explicit `file` + `symbol`; not used for broad search scoring by default. | +| CodeGraph `context`/file overview | `aft_outline` | Useful for AFT-specific supplemental cases. Outline text is normalized into file/symbol-ish result items when possible. | +| CodeGraph `trace`/call graph | `aft_navigate` commands (`callers`, `call_tree`, `trace_to_symbol`, etc.) | Only measured for explicit navigate cases; graph edge density is not scored. | +| Plain lexical baseline | AFT bridge `grep` and external `rg -F` | Both use real wall-clock dispatch and fixed-string lexical matching. | +| Sanity baseline | List files only | Ranks file paths without looking at query text; proves the scorer is not trivially passing. | + +## 4. What will not be replicated + +- **Agent A/B matrix** (`scripts/agent-eval/`, tmux/Claude runs, token/cost/tool-call behavior): explicitly out of scope for this task and depends on harness machinery AFT does not have here. +- **Graph edge metrics** (`edgeCount`, `edgeDensity`) for non-graph AFT drivers: AFT does not expose a CodeGraph-style returned subgraph for `aft_search`, AFT grep, ripgrep, or list-files. Reporting zero would be misleading, so those fields stay omitted. +- **Kind-filtered semantic retrieval:** CodeGraph can pass `kinds` into `searchNodes`; AFT's semantic search does not accept a kind filter today. Kinds are used only for metadata/diagnostics. +- **AFT `aft_search` vs CodeGraph on Elasticsearch in this commit:** the harness supports `codegraph-original`, but the verification run for this task is against `opencode-aft` because that is the indexed local target. + +## 5. Output format + +Emit JSON close to CodeGraph's `EvalReport`: + +```ts +{ + timestamp: string, + codebasePath: string, + codegraphSha: string, + aftSha?: string, + benchmark: "codegraph-replication", + corpus: string, + driver: string, + summary: { + total: number, + passed: number, + failed: number, + skipped: number, + meanRecall: number, + meanMRR: number, + meanPrecisionAt1: number, + meanPrecisionAt5: number, + meanPrecisionAt10: number, + latencyMsMedian: number, + latencyMsP95: number + }, + results: EvalResult[] +} +``` + +`EvalResult` keeps CodeGraph-compatible fields (`caseId`, `pass`, `recall`, `mrr`, `foundSymbols`, `missedSymbols`, `latencyMs`) and adds ranked `results`, `precisionAtK`, `driver`, `api`, and optional `skipReason`. A markdown summary with the same aggregate table and per-case rows will be written beside the JSON so results can be pasted into docs/README. + +## 6. code-review-graph patterns borrowed + +I also read `/Users/ufukaltinok/Work/OSS/code-review-graph/code_review_graph/eval/` for methodology inspiration. This benchmark will still replicate CodeGraph first, but borrows these low-cost patterns where they improve reproducibility without adding dependencies on that project: + +- **Pinned repo metadata shape:** corpus entries can carry repo name, URL, language, size category, and pinned commit fields, matching code-review-graph's `configs/*.yaml` discipline. v1 runs against `opencode-aft`, but this schema lets us add the reusable `fastapi`, `flask`, `gin`, `express`, `httpx`, and `code-review-graph` repos later without redesign. +- **Separated task axes:** keep CodeGraph's `searchNodes` vs `findRelevantContext` API labels, but also tag cases with categories analogous to code-review-graph's `search_queries` and `multi_hop_tasks` so later reports can split symbol lookup, context exploration, and navigation/multi-hop retrieval. +- **Deterministic reporting:** include corpus path, codebase SHA, AFT binary path, driver, top-k, and runs in every report. This mirrors code-review-graph's pinned-SHA/config-driven reproducibility while keeping the AFT harness simple. +- **Real wall-clock timing per dispatch:** code-review-graph times build/search stages directly; AFT will time the actual bridge or process dispatch around each query and aggregate median/p95. +- **Token accounting is deferred:** code-review-graph's tiktoken-calibrated token-efficiency axis is useful, but it belongs to a broader agent/context benchmark, not this no-LLM CodeGraph retrieval replication. v1 may record result payload sizes later, but will not mix token-efficiency scores into retrieval quality. + +Patterns intentionally not borrowed for v1: the six-axis suite (`impact_accuracy`, `multi_hop_retrieval`, `search_quality`, `token_efficiency`, `flow_completeness`, `build_performance`) and repository cloning/build orchestration. Those are valuable follow-on axes, but this deliverable stays focused on deterministic retrieval scoring against AFT's actual tool surface. + diff --git a/.alfonso/release-notes/v0.30.0.md b/.alfonso/release-notes/v0.30.0.md new file mode 100644 index 00000000..5e64a675 --- /dev/null +++ b/.alfonso/release-notes/v0.30.0.md @@ -0,0 +1,52 @@ +# PTY support — agents can now drive real terminals + +The headline of this release. `bash` now accepts `pty: true` (with `background: true`) to spawn commands inside a real PTY — every interactive program that needed a terminal is now reachable from an agent loop. Python and Node REPLs, `vim`, `htop`, `top`, `less`, `fzf`, build TUIs, even a nested `opencode` session — all work end-to-end. + +![Yo dawg, I heard you like OpenCode so I put an OpenCode inside your OpenCode](assets/ocinoc.png) + +Yes, really — `opencode` inside `opencode` works. PTY support means the agent can drive any TUI, including a full nested AFT-equipped OpenCode session, complete with sidebar, MCP servers, LSP status, and another agent answering prompts. Recursion all the way down. + +### How it works + +- **`bash({pty: true, background: true, ptyRows?, ptyCols?})`** — spawn a PTY-backed task. Defaults are 24×80; caps are 60×140 to keep `bash_status` snapshots bounded. +- **`bash_status({taskId, outputMode})`** — read the terminal state. + - `"screen"` — vt100-rendered visible terminal (rows × cols characters) + - `"raw"` — uncompressed bytes including ANSI escape sequences + - `"both"` — separate fields for each +- **`bash_write({taskId, input})`** — send keystrokes. Input is either a verbatim string or an array mixing strings and `{key: "..."}` objects for atomic text + control key sequences: + + ``` + bash_write({taskId, input: [ + "iHello", + {key: "esc"}, + ":wq", + {key: "enter"}, + ]}) + ``` + + Named keys cover `enter`/`return` (CR), `tab`, `space`, `backspace`, `esc`/`escape`, arrow keys, navigation keys, `delete`, `insert`, `f1`–`f12`, and `ctrl-a` through `ctrl-z`. + +PTY tasks run on Unix via `portable-pty` and on Windows via ConPTY. + +## bash_watch unifies pattern notifications and sync waits + +New `bash_watch` tool replaces ad-hoc wait flags on `bash_status`. Two modes: + +**Sync** — `bash_watch({taskId, pattern?, timeoutMs?})` blocks until the pattern matches, the task exits, or timeout. Without a pattern it waits for task exit. Returns the snapshot inline so the agent gets the result without a separate completion reminder. + +**Async** — `bash_watch({taskId, pattern, background: true})` registers a pattern watcher and returns immediately. When the pattern matches mid-stream or the task exits, a single `[BG BASH NOTIFY]` reminder fires with the matched line. The default `[BACKGROUND BASH COMPLETED]` reminder is suppressed for that task. + +`bash_status` is now a pure snapshot tool — wait/watch semantics live in `bash_watch`. + +## URL fetches no longer hang on slow servers + +`aft_outline` and `aft_zoom` URL targets now abort with a clear stall error after 15 seconds without a chunk. Previously a slow or stalled server could hang the bridge indefinitely while waiting on `reader.read()`. + +## Other + +- `bash` schema rejects `pty: true` without `background: true` and `ptyRows`/`ptyCols` without `pty: true`. +- OpenCode subagent sessions silently convert `background: true` to foreground bash unless `bash.subagent_background = true` in config. +- `bash_status` and `bash_kill` are always registered when `bash` is registered (no longer gated on `experimental.bash.background`). +- Background bash completion delivery now persists `completion_delivered` across plugin restarts, so previously-delivered tasks no longer replay as fresh reminders after restart. +- Async `bash_watch` exit notifications render as `task X exited` instead of the prior `matched "exited (exit 0)"` framing. +- The release script blocks minor-version releases when the in-plugin `ANNOUNCEMENT_VERSION` is stale relative to the release tag. diff --git a/.alfonso/release-notes/v0.30.1.md b/.alfonso/release-notes/v0.30.1.md new file mode 100644 index 00000000..55c3de63 --- /dev/null +++ b/.alfonso/release-notes/v0.30.1.md @@ -0,0 +1,38 @@ +# v0.30.1 + +Patch release. Three classes of user-facing fixes: bash PTY parameter handling, LSP failure diagnostics, and Windows plugin auto-update. + +## Bash — PTY parameter handling + +Agents that defensively included `ptyRows` or `ptyCols` on regular (non-PTY) bash calls were hitting a strict validation error. Some models tried to "fix" it by adding `pty: true` to non-interactive commands, which auto-promoted them to background and broke inline output. + +- `ptyRows` and `ptyCols` are now soft-ignored when `pty` is unset or false. The dimensions are only applied when a PTY is actually requested. +- `pty: true` now implies `background: true`. The two flags no longer have to be set together. +- Out-of-range or non-integer values return a clean error naming the allowed bounds (e.g. `ptyRows must be an integer between 1 and 60`). +- Tool descriptions for `ptyRows`/`ptyCols` clarify they apply only when `pty: true`. + +## Plugin tool schemas + +All optional numeric parameters across the OpenCode plugin (bash, read, aft_search, aft_navigate, aft_zoom, aft_outline, refactor, lsp_diagnostics) now use a JSON-Schema-representable bounded integer schema. Empty sentinels (null, empty string, zero) are rejected at validation with a clear message instead of silently being coerced or — as in an earlier internal build — causing the plugin to fail to load. + +A schema-conversion regression test now covers every registered tool, so any future change that introduces an unrepresentable shape will fail before release. + +## LSP — failure visibility + +When an LSP server fails to start, AFT's response now surfaces stderr output captured from the child process. Previously, broken language-server shims (such as a `typescript-language-server` whose `cli.mjs` was missing) returned opaque `spawn_failed` errors without context. + +- Stderr from LSP children is captured in a bounded ring buffer and included in failure responses. +- When stderr contains `MODULE_NOT_FOUND`, the response adds a hint pointing at the likely fix (reinstall the package-manager binary, or check the `lsp.servers..binary` path). +- Clients that crash after a successful initialize are now marked as failed so subsequent file requests stop re-issuing pulls against the dead pipe. + +## Auto-update on Windows + +Plugin self-update used `spawn("npm")` directly, which fails on Windows because the binary is `npm.cmd`. The auto-update path now resolves `npm.cmd` on Windows (same fix shape as the v0.28.2 LSP install correction). + +- `npm install` stderr is captured on failure for diagnostic visibility. +- `--ignore-scripts` is now passed to the install (matches the LSP install hardening). + +## Other + +- `aft_outline`/`aft_zoom` URL fetch keeps the 15-second body-stall safety net that landed in v0.30.0. +- Subagent sessions continue to silently convert `background: true` bash to foreground (introduced in v0.30.0), because subagents have no completion-reminder mechanism. diff --git a/.alfonso/release-notes/v0.30.2.md b/.alfonso/release-notes/v0.30.2.md new file mode 100644 index 00000000..092910c2 --- /dev/null +++ b/.alfonso/release-notes/v0.30.2.md @@ -0,0 +1,107 @@ +# v0.30.2 + +Patch release. Large correctness pass across LSP diagnostics, navigation, ast-grep, format/checker, search, bash, plugin notifications, and the CLI. + +## LSP diagnostics + +- Pull-diagnostics `unchanged` responses are no longer accepted on the first pull. Previously a server returning `kind: "unchanged"` with no prior cache caused AFT to report a clean file even when real errors existed. +- When a server advertises pull diagnostics but rejects `textDocument/diagnostic` mid-session (`MethodNotFound`/invalid params), AFT now falls back to push instead of marking the file `pull_failed` indefinitely. +- Diagnostic cache is cleared when a file is closed. Reopening a file no longer surfaces stale diagnostics from a prior session. +- Disk-drift detection for open documents now uses a content hash when `(mtime, size)` are unchanged but the filesystem timestamp can't be trusted (coarse-mtime FS, same-size rewrites). +- Servers advertising `workspace.didChangeWatchedFiles` via initialize options (without dynamic registration) now receive file-watch notifications. +- LSP JSON-RPC framing rejects unsupported content types and non-UTF-8 charsets instead of accepting any payload that happens to decode as UTF-8. + +## Format and checker + +- Biome, Ruff (check), and staticcheck now report real errors. Previously these checkers were selected but had no parser, so AFT reported a clean check even when they exited non-zero with real diagnostics. +- Checker non-zero exit without parseable diagnostics (e.g. `Cargo.toml` malformed before cargo emits JSON) now returns `skip_reason: "error"` with stderr context, not silent success. +- Absolute paths for `cargo`/`go` resolved via PATH or well-known locations are now executed directly. Previously the resolved path was discarded and the bare name re-resolved, which failed in GUI-launched shells. +- Local Windows `node_modules/.bin` lookups probe `.cmd`/`.bat`/`.exe` variants. Previously only the bare command was checked, so local npm tools were invisible on Windows. +- Pyright column offsets are now 1-based to match other checkers. +- Go vet output with Windows drive-letter paths (`C:\...`) is parsed correctly. +- Ruff checker no longer gated on `ruff format` availability; older Ruff versions that support `ruff check` are usable as checkers. +- Formatter/checker stdout/stderr capture is bounded; noisy tools can no longer OOM AFT. +- Windows formatter/checker timeouts now kill the full process tree, not just the immediate child. + +## ast-grep + +- `ast_replace` validates rewritten file syntax before writing. Invalid rewrites are rejected with `invalid_rewrite` and the operation rolls back instead of persisting broken code. +- `ast_search`/`ast_replace` no longer prune `node_modules`/`target`/`dist`/`build` when the user explicitly passes those as the search root. +- PHP files are now parsed with the full PHP grammar instead of the snippet-only grammar. Real `.php` files with `` work correctly. + +## Search + +- Indexed grep now correctly handles case-insensitive searches for non-ASCII patterns. Previously a file containing `äbc` was excluded from results when searching `Äbc` case-insensitively. +- Incremental refresh detects same-size edits when `mtime` is preserved (delete/recreate, coarse-mtime filesystems). Previously stale postings could survive a content change. + +## Navigation + +- `aft_navigate call_tree` now rejects paths outside the project root, matching the other navigation operations. +- Namespace imports (`import * as lib from './index'`) follow barrel reexports correctly. False positives for private symbols and missed callers through index reexports are fixed. +- Workspace package resolution covers `.mts`/`.cts`/`.mjs`/`.cjs` extensions, so monorepos with modern module formats route cross-package callers through source instead of falling back to built `dist/`. +- Unresolved member calls (`db.connect()`) are no longer reported as false callers of any same-file `connect` symbol. +- Watcher invalidation now covers `.mts`/`.cts`/`.mjs`/`.cjs` edits. +- Same-named symbols in one file are tracked by scoped identity. `class A { run() }` and `class B { run() }` no longer overwrite each other in the navigation index. +- `trace_to.entry_points_found` dedupes on `(file, symbol)` so two `main` functions in different packages count as two paths. +- `call_tree`/`callers`/`impact` now report `depth_limited`/`truncated` when results hit the depth cap. Pi rendering surfaces the truncation flags. + +## Bash + +- Watch patterns spanning scan boundaries no longer go missed. +- Stderr is scanned by watches alongside stdout. +- POSIX shell resolution corrected for Windows bash invocations. +- Redirect targets are canonicalized before permission checks. + +## Plugin — wake delivery + +- The live-server wake-transport probe now requires a successful response. Plain TUI's internal listener (which returns 404 for `/session`) is correctly classified as unreachable, so wake delivery uses the in-process fallback instead of failing silently. +- If the live server becomes unreachable mid-session, wake delivery now falls back to in-process delivery and demotes the cached transport decision instead of hard-stopping after 5 failed retries. +- Live-server availability is keyed by `serverUrl`. Multiple windows or projects with different server URLs no longer race and overwrite each other's transport choice. +- Synthetic prompts for background-bash wakes now resolve from the newest effective context. Mid-conversation model switches no longer cause wakes to use the prior assistant's model/agent. + +## TUI + +- Sidebar status no longer imports from `@cortexkit/aft-bridge`, which pulled `undici` into the Bun TUI runtime. The known Bun/undici load failure path is removed. +- Sidebar attaches the project directory to each snapshot and clears stale state on project transitions. Project A's status no longer briefly shows up in Project B. +- Sidebar and `/aft-status` polling cancel in-flight requests on unmount, so late completions cannot mutate state after a dialog closes or a project switches. + +## RPC + +- RPC port discovery always considers the legacy `port` file as a final fallback candidate. In mixed upgrade scenarios where new per-instance port files exist but are stale, AFT no longer hides live legacy servers behind dead per-instance entries. + +## CLI + +- `aft doctor` binary probe rejects mismatched-version binaries. Stale or unrelated `aft` binaries on PATH no longer report as healthy; `aft doctor --fix` correctly proceeds to download the matching version. +- `aft doctor lsp ` now sends `harness` in its configure payload. The previous omission caused `configure payload missing required field 'harness'` for users on the v0.30.0/0.30.1 CLI binary. +- `aft setup` and `aft doctor --fix` preserve comments in `opencode.jsonc` when adding the plugin entry. +- `aft doctor lsp ` resolves the project root from the target file path, so inspecting a file from outside its repo loads the right config. +- `aft doctor --fix` shows planned mutations and prompts before editing host config or running `pi install`. + +## URL fetch + +- `aft_outline`/`aft_zoom` now accept `application/json` (npm registry, unpkg, OpenAPI specs, JSON-LD). The top-level JSON keys are surfaced as outline symbols. +- URL fetching no longer stalls on body reads under OpenCode's Bun plugin runtime. The bundled undici inside the plugin bundle stalled on common hosts (`github.com`, `registry.npmjs.org`, `api.github.com`); AFT now uses Bun's native fetch in that runtime. SSRF validation still runs. + +## Import management + +- `aft_import add` merges new names into an existing same-module same-kind named import instead of inserting a duplicate `import { ... } from "lib"` line. Linter complaints about duplicate imports no longer appear after adding a second symbol from the same module. + +## Refactoring + +- `aft_refactor inline` now works on exported TypeScript functions. After the parser change that included the `export` keyword in symbol ranges, the inline path couldn't reach the wrapped function declaration and returned `symbol_not_found`. + +## Bridge transport + +- UTF-8 multibyte characters split across NDJSON chunks are decoded correctly. +- Timeout on one request aborts sibling in-flight requests immediately instead of leaving them queued against a dead pipe. +- Caller's `transportTimeoutMs` now applies to implicit configure and version RPCs. +- Bridge crashes mid-configure no longer leave the bridge marked as configured. +- Stderr tail is buffered per logical line so split chunks don't corrupt the diagnostic output. + +## CI and release infrastructure + +- `wait-release.sh` exits on the first terminal job failure instead of waiting for every job to drain. +- `scripts/release.sh` recovery path no longer terminates after the first publishing fallback. +- E2E harness cleanup uses `trap`-based teardown; failed runs no longer leak mock server processes or temp directories. +- Linux Docker E2E `Dockerfile` fails the image build when plugin preinstall or local artifact placement fails, instead of silently testing against stale npm-published artifacts. +- External benchmark harness exits non-zero when any repo fails to evaluate, unless `--allow-partial` is passed. diff --git a/.alfonso/release-notes/v0.30.3.md b/.alfonso/release-notes/v0.30.3.md new file mode 100644 index 00000000..5dbb04cb --- /dev/null +++ b/.alfonso/release-notes/v0.30.3.md @@ -0,0 +1,34 @@ +# v0.30.3 — URL fetching moves to Rust, plus quality-of-life fixes + +This release moves `aft_outline` / `aft_zoom` URL fetching out of the TypeScript plugin layer and into the Rust binary, fixing a class of body-stall failures under OpenCode's Bun runtime. It also tightens a handful of long-standing UX rough edges across both plugins. + +## URL fetching now happens in Rust + +Fetching for `aft_outline({ target: "https://..." })` and `aft_zoom({ url: "..." })` now uses `reqwest` in the Rust binary instead of `undici` in the TypeScript plugin. This eliminates the "body read stalled (no data for 15000ms)" failures that were specific to OpenCode's Bun-bundled runtime — most visibly for `api.github.com`, `registry.npmjs.org`, and large HTML pages. + +Network behavior under the hood: +- TLS via `rustls` with a 30-second connect timeout. +- 15-second per-chunk body stall timeout. +- SSRF guard (private/loopback/link-local rejection) preserved. +- New: up to 2 silent retries on transient connect/transport failures. TCP connect blips and momentary TLS hiccups no longer surface as errors to the agent; HTTP 4xx/5xx and SSRF rejections still pass through immediately. +- JSON responses are now accepted, so npm registry endpoints and unpkg package metadata can be outlined directly. Top-level keys appear in the outline. + +## Fresh-install announcements no longer spam + +The version-announcement dialog (the "What's new in v..." panel) used to fire on every restart in ephemeral environments — Docker containers, CI sandboxes, disposable dev containers — because they had no record of any previous version. First-time users on a brand-new install saw the same dialog with changelog bullets they had no context for. + +Fresh installs now silently record the current version and suppress the dialog. Real upgrades from an older recorded version continue to fire as expected. The shared helper lives in `@cortexkit/aft-bridge` so OpenCode and Pi stay in lockstep. + +(Same fix pattern as `cortexkit/magic-context#99`.) + +## Notifications carry the current agent (#62) + +Ignored messages (configure warnings, auto-update notices, startup announcements, status messages) now carry the active agent identity. Previously, when the user had switched to a non-default agent through `oh-my-openagent` or a similar extension, these one-way messages still appeared under the default agent label in the UI. Background-bash completion reminders were already correct; this brings the rest of the notification surface to parity. + +## Bun-test failure markers survive `bun run` wrappers + +When a `package.json` `test` script calls `bun test` and the agent invokes it through `bun run --cwd packages/foo test`, the bun output compressor used to drop the `(fail) ...` markers — agents would see only a summary like `2 fail / Ran 940 tests across 82 files` and have to re-run with `| grep fail` to see what actually failed. The compressor now detects bun-test output by its private summary line and preserves failure context regardless of how `bun test` was invoked. + +## Acknowledgements + +`@cortexkit/aft-bridge`, `@cortexkit/aft-opencode`, `@cortexkit/aft-pi`, `@cortexkit/aft`, `agent-file-tools`, `aft-tokenizer`, and the platform binary packages all ship at `v0.30.3`. diff --git a/.alfonso/release-notes/v0.31.0.md b/.alfonso/release-notes/v0.31.0.md new file mode 100644 index 00000000..96301ed2 --- /dev/null +++ b/.alfonso/release-notes/v0.31.0.md @@ -0,0 +1,106 @@ +# v0.31.0 — Cross-file navigation, indexed file trees, and Pi grep that doesn't hang + +Three new agent capabilities plus a long-overdue Pi UX fix: external-path searches no longer freeze waiting on a permission prompt that has no policy behind it. + +## Trace the call path between two symbols + +`aft_navigate` has a new `trace_to_symbol` op for "how does function A reach function B" queries — the single most expensive question to answer by hand. One call returns the shortest call path through every intermediate hop, with file and line for each node. + +``` +aft_navigate({ + op: "trace_to_symbol", + filePath: "src/bridge.ts", + symbol: "send", + toSymbol: "spawn_child", +}) +``` + +Returns either the shortest path (each hop annotated with file + line), or a structured error if the target is missing, ambiguous, or unreachable: +- `target_symbol_not_found` — name doesn't exist anywhere in the indexed graph +- `ambiguous_target` — multiple symbols share the name; rerun with `toFile` from the candidates list +- `target_symbol_not_in_file` — `toFile` provided but no matching symbol in it; candidate list returned +- `to_file_not_found` — the file you named doesn't exist +- `no_path_found` — the graph genuinely has no path + +The default depth cap is 10; pass `depth` to raise it. + +## `aft_outline files: true` — indexed file tree with per-file metadata + +`aft_outline target: "", files: true` now returns a flat indexed file tree with language, symbol count, and byte size per file — no symbol bodies, no signatures, just the structural metadata an agent needs to pick which files to actually open next. + +``` +aft_outline({ target: "packages/aft-bridge/src", files: true }) +``` + +Reuses AFT's existing symbol cache, so the call is fast even on cold bridges. The output is honest about truncation: when a directory exceeds the 200-file walk cap, the response sets `complete: false` and surfaces both `walk_truncated` and `unchecked_files` so agents don't mistake a partial tree for a complete one. Multi-target calls render every entry as a project-root-relative path, so two files named `lib.rs` from different crates can't collide in the output. + +Also accepts an array of directories: `target: ["crates/aft/src", "packages/aft-bridge/src"]`. + +## `aft_zoom` — cross-file batches and a polymorphic schema + +`aft_zoom` is the read-the-source-of-this-symbol tool. Two changes this release: + +**New: `targets` for cross-file batching.** Previous `symbols: [...]` array could only zoom into multiple symbols within the same file. The new `targets` array lets agents pull bodies from different files in one call: + +``` +aft_zoom({ targets: [ + { filePath: "src/a.ts", symbol: "callBridge" }, + { filePath: "src/b.ts", symbol: "spawn_child" }, +]}) +``` + +**Schema consolidation (breaking).** `symbol` and `symbols` collapse into a single polymorphic `symbols` parameter that accepts either a string or an array. Same for `targets` (single object or array). URL mode follows the same shape, so an agent can pull multiple sections from a single URL fetch: + +``` +aft_zoom({ url: "https://docs.example.com/api", symbols: ["Authentication", "Errors", "Examples"] }) +``` + +The four shapes (`filePath + symbols`, `targets`, `url + symbols`, and combinations) are mutually exclusive with a clear error when mixed. Old callers using `symbol: "name"` need to migrate to `symbols: "name"`; the surface change is small but it is a break. + +## Output-shape compression — fewer "what failed?" reruns + +Bun, npm, and pnpm test compressors now match on the shape of the captured output, not just on the head token of the command. Wrapper invocations like `bun run --cwd packages/foo test`, `npm test`, `pnpm test`, or even `bun test && echo done` now go through the test-aware compressor instead of falling through to the generic line-dedup path. Failing test bodies and assertion diffs are preserved on the first run; the agent doesn't need to follow up with `| grep fail` to see what broke. + +## Pi: external-path tool calls no longer hang + +The headline Pi fix. Pi `grep`/`write`/`edit` against a path outside the project root would block the bridge indefinitely on a `ui.confirm` "Allow external directory access?" prompt — even when the user had `restrict_to_project_root: false` (the Pi default) which explicitly opts into "no path restriction." + +Three causes, all addressed: + +1. **No tilde expansion.** `~/Work/...` arrived in the plugin as a literal, `path.resolve(cwd, "~/...")` resolved to `/~/...`, stat() failed, and Rust returned `path_not_found`. Both `assertExternalDirectoryPermission` and `resolvePathArg` now expand `~` / `~/foo` before any check. + +2. **No `ui.confirm` timeout.** When Pi ran the call from a context that couldn't surface the prompt, the confirm promise simply never resolved. Now bounded at 30 seconds with a deterministic "Permission denied: prompt timed out" so the agent unblocks. + +3. **No policy-aware skip.** When `restrict_to_project_root: false` — the Pi default and what the user explicitly opted into — the plugin used to nag anyway. Pi has no host-level `external_directory` allow-list to consult (unlike OpenCode), so the prompt had no policy behind it. The plugin now defers to Rust without prompting when the user opted into "no restriction." + +Behavior matrix: + +| `restrict_to_project_root` | Pi behavior | +|---|---| +| `false` (default) | Plugin defers to Rust; no prompt | +| `true` + interactive UI | `ui.confirm` with 30s timeout | +| `true` + no UI | Immediate deny with a clear error | + +OpenCode's grep/glob path also gained tilde expansion, for parity. OpenCode external-directory checks already routed through the host `context.ask({permission: "external_directory"})` which the host resolves against configured rules without blocking on a UI, so the hang did not reproduce there. + +## Tool descriptions: ~1.2K tokens trimmed + +Dropped redundant `Returns:` blocks from `aft_transform`, `aft_import`, `aft_refactor`, `aft_safety`, `ast_grep_search`, and `ast_grep_replace` — agents see the actual response shape at runtime, no need to also restate it in the prompt. Collapsed `lsp_diagnostics` from a 700-token inline JSON schema + verbose honesty playbook to a 250-token version that keeps the load-bearing "don't claim 'no errors' when nothing was checked" rule. Standardized path-resolution wording across `filePath` / `path` / `directory` params so the surface is consistent. + +Combined: 5,546 → 4,498 tokens (-18.9%) on OpenCode, 4,418 → 4,315 (-2.3%) on Pi. Pi gained less because it had no `Returns:` blocks to strip. + +Per-tool, the biggest cuts: +- `aft_transform`: 788 → 543 (-31%) +- `lsp_diagnostics`: 704 → 255 (-64%) +- `aft_import`: 435 → 281 (-35%) +- `ast_grep_search`: 484 → 384 (-21%) + +A separate audit-driven pass also fixed three release-blocking description bugs and seven smaller polish items: an `apply_patch` claim about atomic rollback that no longer matches the actual per-file-commit behavior, a `aft_outline({ url })` example that named the old parameter shape, and ambiguous mutual-exclusion wording in `aft_zoom`. + +## Other + +- `trace_to_symbol`: ambiguity recovery error now renders the full candidate list as plain text instead of swallowing it inside `data:`. Agents can re-issue the call with `toFile` immediately. +- `aft_outline files: true`: now asks OpenCode for the `external_directory` permission when the directory is outside the project, matching how other file-touching tools behave. +- `bun` output-shape compressor was claiming output from arbitrary text that happened to include a `Ran N tests across M files` summary. It now requires a structurally valid bun-test marker (`(pass)`/`(fail)` followed by name and duration) before claiming the output. +- PTY watchdog test budget tightened below the watchdog poll interval so a passing test now actually proves the wake channel beat the periodic poll, instead of just measuring overall wall-clock. +- Pi added e2e and Pi-RPC coverage for `trace_to_symbol` and `aft_outline files: true`. diff --git a/.alfonso/release-notes/v0.31.1.md b/.alfonso/release-notes/v0.31.1.md new file mode 100644 index 00000000..2d521f67 --- /dev/null +++ b/.alfonso/release-notes/v0.31.1.md @@ -0,0 +1,29 @@ +# v0.31.1 — Strict-LSP diagnostics, Windows doctor UX, Pi grep that doesn't hang + +A patch release with two focused fixes: `lsp_diagnostics` now works correctly against strict LSP servers like tsgo, and `aft doctor` handles Windows setup gaps that used to produce silent dead ends. + +## `lsp_diagnostics` against tsgo and other strict servers (#63) + +AFT was sending `identifier: null` and `previousResultId: null` in pull-diagnostics requests when those fields had no value, because the upstream `lsp-types` crate at 0.97 omits the `skip_serializing_if = "Option::is_none"` annotation on them. The LSP 3.17 spec defines those fields as string-or-absent, not string-or-null. Permissive servers like `typescript-language-server` accept the null and return diagnostics anyway; strict servers like `tsgo` reject the request with `InvalidParams (-32602)`, and AFT then waited for push diagnostics that never came from a pull-only server. The user-visible symptom was `lsp_diagnostics` silently returning empty results for files that genuinely had type errors. + +Fixed by introducing AFT-local `AftDocumentDiagnosticParams` and `AftWorkspaceDiagnosticParams` types with the missing serde annotations, sent through `AftDocumentDiagnosticRequest` / `AftWorkspaceDiagnosticRequest` using the same `textDocument/diagnostic` and `workspace/diagnostic` method strings as upstream. No behavior change for servers that accept either shape; tsgo now returns diagnostics correctly. Thanks to `@null-axiom` for the precise diagnosis. + +## `aft doctor` no longer hides Windows setup problems (#64) + +Five related Windows / setup-UX fixes for `aft doctor` and `aft doctor --fix`: + +- **Plugin/CLI version skew is now a visible issue.** Running `npx @cortexkit/aft@latest doctor --fix` against an installation with an older plugin (for example CLI v0.30.3 against `@cortexkit/aft-opencode@0.29.1`) used to silently download the newer binary into the cache, where the plugin would then ignore it because of strict protocol pinning. Doctor now detects the skew, surfaces a high-severity issue with remediation, and `--fix` prompts before downloading the binary instead of silently caching one that won't be used. `--yes` proceeds; `--ci` and non-TTY environments skip the download cleanly. + +- **Windows ONNX detection now scans `PATH`.** Users who install ONNX Runtime via Scoop or a manual zip on Windows typically put the `onnxruntime.dll` directory on `PATH`. The previous detector only looked in fixed locations and missed those installs. The new path adds `PATH` entries on Windows with conservative guards: absolute paths only, no current directory or null bytes, case-insensitive filename match. Mac and Linux detection is unchanged. + +- **Storage "not created" no longer reads as a failure.** When AFT hasn't yet spawned a bridge in a session, the storage directory doesn't exist on disk — that's expected lazy behavior, not a problem. Doctor now says so explicitly, and `aft doctor --fix` opportunistically creates the directory for registered plugins so the next session starts clean. + +- **Doctor output has an "Issues found" summary.** The previous output was a wall of green checkmarks with any real warnings buried inline. The markdown report now leads with an `Issues found` block — severity (HIGH/MEDIUM/LOW), scope, message, and remediation — for any non-zero findings. The full per-harness diagnostic stays below for context. Renders the same on TTY, CI, and `--issue` bug reports. + +- **`bg-notifications` log noise.** The plugin used to log `WARN [aft-plugin] Live OpenCode HTTP listener unreachable, falling back to in-process promptAsync` on every wake delivery in non-`--port 0` TUI sessions, which is the expected fallback path. The fallback transition is now DEBUG-level; the WARN level is reserved for cases where no wake transport actually delivers. Thanks to `@Zireael` for the detailed bug report. + +## Acknowledgements + +`@cortexkit/aft-bridge`, `@cortexkit/aft-opencode`, `@cortexkit/aft-pi`, `@cortexkit/aft`, `agent-file-tools`, `aft-tokenizer`, and the platform binary packages all ship at `v0.31.1`. + +Join us on Discord: https://discord.gg/F2uWxjGnU diff --git a/.alfonso/release-notes/v0.32.0.md b/.alfonso/release-notes/v0.32.0.md new file mode 100644 index 00000000..5da01e59 --- /dev/null +++ b/.alfonso/release-notes/v0.32.0.md @@ -0,0 +1,51 @@ +# v0.32.0 — Unified `aft_search` and queryable-during-refresh semantic + +The headline change: `aft_search` is now a single tool that handles every code-search shape — exact identifiers, anchored regex, error messages, natural-language descriptions, and file/URL paths. It auto-routes between regex, literal, semantic, and hybrid lanes based on query shape, with a `hint` parameter for explicit overrides. Output adapts per mode — grep-style lines for regex/literal, symbol-blocks with provenance for semantic/hybrid. The semantic index now also stays queryable through edits instead of falling back to lexical-only after every save. + +## Unified `aft_search` + +`aft_search` replaces the previous split between concept search and grep-style lookup. One `query` parameter, automatic mode detection, one consistent response shape per mode. + +- **Classification before status check.** Regex queries succeed even when the semantic backend is unavailable; the lexical lane is always available when grep is registered. +- **Pre-Tier path/URL exemption.** Queries shaped like file paths (`src/lib/main.rs`), Windows paths (`C:\new\test`), URLs (`https://api.github.com`), and filenames with metacharacters (`is_valid?.ts`, `Cargo.lock`) stay in hybrid mode instead of misrouting to regex. +- **Sequence-based regex detection.** Sequences like `.*`, `.+`, `\d+`, and `[A-Z]` correctly trigger regex routing while bare punctuation that commonly appears in code (`map.get()`, `foo()`, `bar?.baz`) stays hybrid. +- **`hint` override.** Pass `hint: "regex"`, `hint: "literal"`, or `hint: "semantic"` to force a specific lane. Short literals (under three bytes) honor `hint: "literal"` with a full scan instead of silently rerouting to semantic. +- **Adaptive output per query mode.** Regex and literal modes return grep-style `file:line: text` matches. Semantic and hybrid modes return symbol-blocks with `source: "semantic" | "lexical" | "hybrid"` provenance per result. The `interpreted_as` field tells callers which shape to expect. +- **Response flags reflect engine limits.** `more_available`, `engine_capped`, and `fully_degraded` replace the previous `total_matches` field, which conflicted with the engine's caps. `humanize_degraded_reasons` translates internal codes to user prose. +- **Tier D rejections.** Lookaround, backreferences, and other regex features the engine doesn't support return explicit errors with rewrite guidance instead of silent zero-match. + +The two plugin layers use the same query classification before mutual-exclusion permission checks, so OpenCode and Pi behave identically. + +## Semantic index stays queryable through edits + +Previously, `aft_search` fell back to lexical-only after every file save because the watcher invalidation set `SemanticIndexStatus` to `Building`. The in-memory index still held fresh embeddings for every unchanged file, but the query gate matched on `Building` regardless of stage and refused the semantic lane. + +`SemanticIndexStatus::Ready` now carries a `refreshing: Vec` list. Watcher invalidations append the changed file to that list without leaving `Ready`. The query path runs the normal semantic lane and adds a soft warning when files are mid-refresh. `Building` is now reserved for cold builds and fingerprint changes (model, embedding dimension, or base URL changed). + +User-visible effects: + +- `aft_search` returns real semantic results immediately after edits, with a warning like `"1 file(s) refreshing; results for those files may be temporarily missing"`. +- The TUI sidebar and `/aft-status` dialog show `Ready (N file(s) refreshing)` as a small dim line instead of `Rebuilding…`. Above 20 refreshing files it collapses to `Ready (many files refreshing)`. +- The status RPC adds `refreshing_count` to the semantic block. Existing fields are preserved. + +## Workflow hints promote `aft_search` + +The system prompt's code-exploration section now teaches `aft_search` as the primary code-search tool, with `grep` framed as the specialized fallback for exhaustive enumeration (every TODO, every import of X) or strict path-scoped search. Users running with `semantic_search: false` continue to see the grep-primary hint unchanged. + +## Bare escape sequences route to regex + +Bare `\n`, `\t`, and `\r` queries now correctly route to regex mode. They were missing from both `tier_a_regex_signal` and the path-exemption guards in the v0.32 classifier. Path-shaped queries containing those escapes (Windows `C:\new\test`) remain exempt and stay hybrid. + +## Empty params no longer mislead the agent + +GPT-family models often send empty strings, empty arrays, and empty objects (`""`, `[]`, `{}`) instead of omitting optional parameters. Previously, that triggered misleading mutual-exclusion errors like `'targets' is mutually exclusive with 'filePath', 'url', and 'symbols'` when the agent only meant to pass `filePath`. The plugin now normalizes empties to `undefined` before mutual-exclusion checks. + +Affected tools across both plugins: + +- `aft_zoom` — `targets: []` and `symbols: ""` no longer trigger spurious exclusion errors. +- `aft_refactor` — required-field validation rejects empty strings for `symbol`, `destination`, and `name` instead of accepting them and crashing downstream. +- `ast_grep_search` / `ast_grep_replace` — empty `paths` and `globs` arrays no longer round-trip to Rust as "scope present" when the agent meant whole project. + +OpenCode's tool-call header also now stringifies array and object args into the rendered metadata so users can see what the agent actually sent in the call. + +Join us on Discord: https://discord.gg/F2uWxjGnU diff --git a/.gitignore b/.gitignore index 325e2973..cfae1827 100644 --- a/.gitignore +++ b/.gitignore @@ -81,3 +81,21 @@ packages/npm/*/bin/aft.exe smoke-tests/ .aft-windows-vm benchmarks/aft-search/.bench/ + +# Beads / Dolt files (added by bd init) +.dolt/ +*.db +.beads-credential-key +.beads/proxieddb/ + +# Local agent tooling directories (not for distribution) +.beads/ +.qartez/ +.claude/ +omo/ +.kiro/ +.lean-ctx/ +agents.md +beads-data-*.jsonl +magic-context-*.md +biome.json_ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 98ea292e..076d7798 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -6,7 +6,7 @@ **Key Characteristics:** - Use `packages/opencode-plugin/src/index.ts` to register OpenCode tools and map them onto Rust commands. -- Use `packages/opencode-plugin/src/bridge.ts` and `packages/opencode-plugin/src/pool.ts` to isolate one `aft` process per session. +- Use `packages/aft-bridge/src/bridge.ts` and `packages/aft-bridge/src/pool.ts` to isolate one `aft` process per session. Both harness adapters (OpenCode, Pi) import these shared primitives from `@cortexkit/aft-bridge`. - Use `crates/aft/src/commands/` handlers to keep protocol dispatch thin and command logic modular. - Use `crates/aft/src/edit.rs`, `crates/aft/src/format.rs`, `crates/aft/src/callgraph.rs`, and `crates/aft/src/lsp/` as shared engines behind multiple commands. @@ -16,21 +16,21 @@ - Purpose: Register tools, load config, and attach post-execution metadata. - Location: `packages/opencode-plugin/src/index.ts` - Contains: Plugin bootstrap, tool-surface selection, hoisting logic, disabled-tool filtering -- Depends on: `packages/opencode-plugin/src/config.ts`, `packages/opencode-plugin/src/tools/*.ts`, `packages/opencode-plugin/src/pool.ts` +- Depends on: `packages/opencode-plugin/src/config.ts`, `packages/opencode-plugin/src/tools/*.ts`, `packages/aft-bridge/src/pool.ts` - Used by: OpenCode plugin loading through `@cortexkit/aft-opencode` **Plugin transport layer:** - Purpose: Resolve or download the binary, start worker processes, and forward requests. -- Location: `packages/opencode-plugin/src/bridge.ts`, `packages/opencode-plugin/src/pool.ts`, `packages/opencode-plugin/src/resolver.ts`, `packages/opencode-plugin/src/downloader.ts` -- Contains: Session bridge lifecycle, restart handling, version checks, binary discovery, binary download -- Depends on: Node child-process APIs, GitHub releases, `packages/opencode-plugin/src/logger.ts` -- Used by: `packages/opencode-plugin/src/tools/*.ts` and `packages/opencode-plugin/src/index.ts` +- Location: `packages/aft-bridge/src/bridge.ts`, `packages/aft-bridge/src/pool.ts`, `packages/aft-bridge/src/resolver.ts`, `packages/aft-bridge/src/downloader.ts` +- Contains: Session bridge lifecycle, restart handling, version checks, binary discovery and download, ONNX runtime helpers, URL fetch +- Depends on: Node child-process APIs, GitHub releases, per-host logger adapters (via `setActiveLogger`) +- Used by: `packages/opencode-plugin/src/index.ts` and `packages/pi-plugin/src/index.ts` (both import from `@cortexkit/aft-bridge`) **Tool definition layer:** - Purpose: Convert OpenCode tool arguments into protocol requests and permission checks. - Location: `packages/opencode-plugin/src/tools/` - Contains: Hoisted tools, reading tools, import tools, transform tools, navigation tools, refactoring tools, safety tools, conflict tools, permissions helpers -- Depends on: `packages/opencode-plugin/src/pool.ts`, `packages/opencode-plugin/src/metadata-store.ts`, `packages/opencode-plugin/src/lsp.ts` +- Depends on: `packages/aft-bridge/src/pool.ts`, `packages/opencode-plugin/src/metadata-store.ts`, `packages/opencode-plugin/src/lsp.ts` - Used by: `packages/opencode-plugin/src/index.ts` **Protocol and command layer:** @@ -38,11 +38,11 @@ - Location: `crates/aft/src/main.rs`, `crates/aft/src/protocol.rs`, `crates/aft/src/commands/` - Contains: Request dispatch, response encoding, command handlers for read/edit/refactor/LSP/conflicts - Depends on: `crates/aft/src/context.rs`, `crates/aft/src/parser.rs`, `crates/aft/src/callgraph.rs`, `crates/aft/src/edit.rs` -- Used by: `packages/opencode-plugin/src/bridge.ts` +- Used by: `packages/aft-bridge/src/bridge.ts` **Analysis and mutation engine layer:** - Purpose: Parse code, compute call graphs, apply edits, format files, and manage imports. -- Location: `crates/aft/src/parser.rs`, `crates/aft/src/callgraph.rs`, `crates/aft/src/edit.rs`, `crates/aft/src/format.rs`, `crates/aft/src/imports.rs`, `crates/aft/src/extract.rs` +- Location: `crates/aft/src/parser.rs`, `crates/aft/src/callgraph.rs`, `crates/aft/src/edit.rs`, `crates/aft/src/format.rs`, `crates/aft/src/imports.rs`, `crates/aft/src/extract.rs`, `crates/aft/src/vector_store.rs`, `crates/aft/src/semantic_index.rs` - Contains: Tree-sitter parsing, symbol extraction, diff generation, formatter detection, type-checker integration, refactor helpers - Depends on: tree-sitter grammars, ast-grep, external formatter and checker processes - Used by: `crates/aft/src/commands/*.rs` @@ -59,7 +59,7 @@ **Tool invocation flow:** 1. Register tool definitions and config-driven surface selection — `packages/opencode-plugin/src/index.ts` -2. Get a session bridge and send a command over NDJSON — `packages/opencode-plugin/src/pool.ts`, `packages/opencode-plugin/src/bridge.ts` +2. Get a session bridge and send a command over NDJSON — `packages/aft-bridge/src/pool.ts`, `packages/aft-bridge/src/bridge.ts` 3. Dispatch the request to a Rust handler and return structured JSON — `crates/aft/src/main.rs`, `crates/aft/src/commands/mod.rs` **Edit pipeline:** @@ -76,20 +76,20 @@ **Binary resolution flow:** -1. Check cache, npm platform package, PATH, and cargo install locations — `packages/opencode-plugin/src/resolver.ts` -2. Download and checksum-verify a release asset when local resolution fails — `packages/opencode-plugin/src/downloader.ts` -3. Start bridges against the resolved binary and hot-swap after version mismatch — `packages/opencode-plugin/src/bridge.ts`, `packages/opencode-plugin/src/pool.ts` +1. Check cache, npm platform package, PATH, and cargo install locations — `packages/aft-bridge/src/resolver.ts` +2. Download and checksum-verify a release asset when local resolution fails — `packages/aft-bridge/src/downloader.ts` +3. Start bridges against the resolved binary and hot-swap after version mismatch — `packages/aft-bridge/src/bridge.ts`, `packages/aft-bridge/src/pool.ts` ## Key Abstractions **BinaryBridge:** - Purpose: Keep one live `aft` subprocess available for request/response traffic. -- Location: `packages/opencode-plugin/src/bridge.ts` +- Location: `packages/aft-bridge/src/bridge.ts` - Pattern: Persistent child-process adapter with timeout-triggered restart **BridgePool:** - Purpose: Scope bridges per OpenCode session and preserve isolated undo history. -- Location: `packages/opencode-plugin/src/pool.ts` +- Location: `packages/aft-bridge/src/pool.ts` - Pattern: Session-keyed object pool with LRU eviction **Tool groups:** @@ -102,6 +102,12 @@ - Location: `crates/aft/src/context.rs` - Pattern: Interior-mutable service container for a single-threaded request loop +**VectorStore (trait):** +- Purpose: Decouple vector storage and similarity search from the semantic index lifecycle. +- Location: `crates/aft/src/vector_store.rs` +- Pattern: Trait with two built-in implementations — `FlatF32VectorStore` (f32 cosine similarity, same as original in-memory store) and `FlatBinaryHammingVectorStore` (packed binary Hamming search for quantized vectors). +- Used by: `crates/aft/src/semantic_index.rs` + **CallGraph:** - Purpose: Cache per-file call data and answer callers, call-tree, impact, and trace queries. - Location: `crates/aft/src/callgraph.rs` @@ -116,7 +122,7 @@ **Rust protocol entry point:** - Location: `crates/aft/src/main.rs` -- Triggers: `packages/opencode-plugin/src/bridge.ts` spawns the `aft` binary +- Triggers: `packages/aft-bridge/src/bridge.ts` spawns the `aft` binary - Responsibilities: Read NDJSON requests from stdin, dispatch handlers, drain watcher and LSP events, and write JSON responses **Release automation entry point:** @@ -126,7 +132,7 @@ ## Error Handling -**Strategy:** Return structured Rust `Response::error` payloads from command handlers, convert failed responses into plugin-side exceptions, and restart hung or crashed worker processes in `packages/opencode-plugin/src/bridge.ts`. +**Strategy:** Return structured Rust `Response::error` payloads from command handlers, convert failed responses into plugin-side exceptions, and restart hung or crashed worker processes in `packages/aft-bridge/src/bridge.ts`. ## Honest Reporting Convention @@ -159,14 +165,15 @@ **Goal:** reduce hoisted-bash output to fewer tokens while keeping the information the agent actually needs (errors, summaries, ref updates) and discarding the noise (progress bars, repeated headers, deep nested directory listings). -**Three-tier dispatch in `crates/aft/src/compress/mod.rs`:** +**Four-tier dispatch in `crates/aft/src/compress/mod.rs`:** -1. **Rust [`Compressor`] modules** — stateful, hand-written parsers for high-traffic tools where heuristics like JSON parsing or section detection are required. Always wins when matched. Each module lives in its own file under `crates/aft/src/compress/` (e.g. `git.rs`, `cargo.rs`, `eslint.rs`) and implements the `Compressor` trait (`fn matches(&str) -> bool` + `fn compress(&str, &str) -> String`). -2. **Declarative TOML filters** — strip + truncate + cap + shortcircuit rules for the long tail of CLI tools, loaded from three sources at startup with project > user > builtin priority by filename: - - **Builtin**: shipped via `include_str!()` from `crates/aft/src/compress/builtin_filters/*.toml`, registered in `crates/aft/src/compress/builtin_filters.rs::ALL` +1. **Specific Rust [`Compressor`] modules** — hand-written parsers for specific tools identified by tool token. Wins before broad package-manager modules. Each module lives in its own file under `crates/aft/src/compress/` and implements the `Compressor` trait (`fn matches(&str) -> bool` + `fn compress(&str, &str) -> String`). Current modules: `git.rs`, `cargo.rs`, `eslint.rs`, `biome.rs`, `tsc.rs`, `pytest.rs`, `vitest.rs`, `playwright.rs`, `mypy.rs`, `prettier.rs`, `ruff.rs`, `go.rs`, `next.rs`. +2. **Package-manager [`Compressor`] modules** — broad head-token matchers (`npm.rs`, `pnpm.rs`, `bun.rs`) that compress unclaimed package-manager output. +3. **Declarative TOML filters** — strip + truncate + cap + shortcircuit rules for the long tail of CLI tools, loaded from three sources at startup with project > user > builtin priority by filename: + - **Builtin**: 22 filters shipped via `include_str!()` from `crates/aft/src/compress/builtin_filters/*.toml`, registered in `crates/aft/src/compress/builtin_filters.rs::ALL` - **User**: `/filters/*.toml` (XDG-aware via the active `storage_dir`) - **Project**: `/.aft/filters/*.toml` — gated by [`crate::compress::trust`]; never loaded for an untrusted project -3. **Generic fallback** — ANSI strip + consecutive-line dedup + middle-truncate. Always applies when no Rust module or TOML filter matches. +4. **Generic fallback** — ANSI strip + consecutive-line dedup + middle-truncate. Always applies when no Rust module or TOML filter matches. **Pipeline for TOML filters** (in `crates/aft/src/compress/toml_filter.rs::apply_filter`): @@ -188,6 +195,6 @@ **Logging:** Write plugin logs through `packages/opencode-plugin/src/logger.ts` and Rust logs through `env_logger` in `crates/aft/src/main.rs`. -**Caching:** Cache resolved binaries in `~/.cache/aft/bin` through `packages/opencode-plugin/src/downloader.ts`, cache session bridges in `packages/opencode-plugin/src/pool.ts`, cache tool availability in `crates/aft/src/format.rs`, and cache call-graph state in `crates/aft/src/callgraph.rs`. +**Caching:** Cache resolved binaries in `~/.cache/aft/bin` through `packages/aft-bridge/src/downloader.ts`, cache session bridges in `packages/aft-bridge/src/pool.ts`, cache tool availability in `crates/aft/src/format.rs`, and cache call-graph state in `crates/aft/src/callgraph.rs`. -**Storage:** Store undo snapshots in `crates/aft/src/backup.rs`, named checkpoints in `crates/aft/src/checkpoint.rs`, pending UI metadata in `packages/opencode-plugin/src/metadata-store.ts`, and downloaded binaries in the cache directory managed by `packages/opencode-plugin/src/downloader.ts`. +**Storage:** Store undo snapshots in `crates/aft/src/backup.rs`, named checkpoints in `crates/aft/src/checkpoint.rs`, pending UI metadata in `packages/opencode-plugin/src/metadata-store.ts`, and downloaded binaries in the cache directory managed by `packages/aft-bridge/src/downloader.ts`. diff --git a/Cargo.lock b/Cargo.lock index 3b953a59..9d636a80 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -25,6 +25,7 @@ version = "0.29.1" dependencies = [ "aft-tokenizer", "ast-grep-core", + "base64 0.22.1", "blake3", "content_inspector", "crc32fast", diff --git a/Dockerfile.rust b/Dockerfile.rust new file mode 100644 index 00000000..c75392e6 --- /dev/null +++ b/Dockerfile.rust @@ -0,0 +1,23 @@ +# Dockerfile for Rust validation +# +# Used by scripts/docker-rust.ps1 to run Rust fmt/check/clippy/test +# inside a container, avoiding the need for native MSVC Build Tools +# on Windows. +# +# This is a minimal image: just Rust + rustfmt + clippy. +# If native dependencies fail during validation, add only the required +# apt packages and document why. +# +# Build (optional — the script pulls rust:1-bookworm directly): +# docker build -t aft-rust -f Dockerfile.rust . +# +# Override the default image via AFT_RUST_DOCKER_IMAGE: +# $env:AFT_RUST_DOCKER_IMAGE = 'aft-rust' + +FROM rust:1-bookworm + +WORKDIR /work + +RUN rustup component add rustfmt clippy + +ENV CARGO_TARGET_DIR=/target diff --git a/STRUCTURE.md b/STRUCTURE.md index 2b34bcd2..3598a7bf 100644 --- a/STRUCTURE.md +++ b/STRUCTURE.md @@ -5,9 +5,13 @@ ```text opencode-aft/ ├── crates/ # Rust workspace packages -│ └── aft/ # Core AFT library, CLI binary, command handlers, and integration tests +│ ├── aft/ # Core AFT library, CLI binary, command handlers, and integration tests +│ └── aft-tokenizer/ # Claude lookup-encoding tokenizer for code estimation ├── packages/ # JavaScript workspace packages -│ ├── opencode-plugin/ # OpenCode plugin that exposes and hoists AFT tools +│ ├── aft-bridge/ # Shared NDJSON bridge transport, binary resolution, ONNX runtime helpers +│ ├── aft-cli/ # Unified CLI — setup/doctor across all harnesses (@cortexkit/aft) +│ ├── opencode-plugin/ # OpenCode adapter that exposes and hoists AFT tools (@cortexkit/aft-opencode) +│ ├── pi-plugin/ # Pi coding agent adapter for AFT (@cortexkit/aft-pi) │ └── npm/ # Platform-specific npm binary packages ├── benchmarks/ # Bun-based benchmark runner and reporting code ├── scripts/ # Release and version-management scripts @@ -20,6 +24,11 @@ opencode-aft/ ## Directory Purposes +**`crates/aft-tokenizer/`:** +- Purpose: Provide Claude-compatible token counting for code estimation and context management. +- Contains: `src/` Rust sources, lookup-table encoding data generated at build time +- Key files: `crates/aft-tokenizer/src/claude.rs`, `crates/aft-tokenizer/build.rs` + **`crates/aft/`:** - Purpose: Keep the Rust execution engine, stdin/stdout protocol binary, and shared analysis logic together. - Contains: `src/` Rust modules, `tests/` integration suites, crate manifest @@ -38,17 +47,44 @@ opencode-aft/ **`packages/opencode-plugin/`:** - Purpose: Ship the OpenCode-facing package that resolves the binary and registers tools. - Contains: `src/` TypeScript sources, `dist/` build output, tests, package manifest -- Key files: `packages/opencode-plugin/src/index.ts`, `packages/opencode-plugin/src/bridge.ts`, `packages/opencode-plugin/package.json` +- Key files: `packages/opencode-plugin/src/index.ts`, `packages/opencode-plugin/src/config.ts`, `packages/opencode-plugin/package.json` **`packages/opencode-plugin/src/tools/`:** - Purpose: Group OpenCode tool definitions by capability area. - Contains: Thin adapters for hoisted, reading, import, structure, navigation, refactor, safety, AST, LSP, and conflict tools -- Key files: `packages/opencode-plugin/src/tools/hoisted.ts`, `packages/opencode-plugin/src/tools/reading.ts`, `packages/opencode-plugin/src/tools/refactoring.ts` +- Key files: `packages/opencode-plugin/src/tools/hoisted.ts`, `packages/opencode-plugin/src/tools/bash.ts`, `packages/opencode-plugin/src/tools/reading.ts`, `packages/opencode-plugin/src/tools/refactoring.ts` + +**`packages/pi-plugin/src/tools/`:** +- Purpose: Group Pi tool definitions by capability area, mirroring the opencode-plugin tool structure. +- Contains: Thin adapters for hoisted, reading, AST, bash, structure, navigation, import, refactor, safety, semantic, LSP, and conflict tools +- Key files: `packages/pi-plugin/src/tools/hoisted.ts`, `packages/pi-plugin/src/tools/reading.ts`, `packages/pi-plugin/src/tools/bash.ts` **`packages/opencode-plugin/src/__tests__/`:** - Purpose: Verify plugin behavior, resolver logic, tool registration, and end-to-end bridge flows. - Contains: Unit tests and `e2e/` test fixtures -- Key files: `packages/opencode-plugin/src/__tests__/tools.test.ts`, `packages/opencode-plugin/src/__tests__/structure.test.ts`, `packages/opencode-plugin/src/__tests__/e2e/` +- Key files: `packages/opencode-plugin/src/__tests__/tools.test.ts`, `packages/opencode-plugin/src/__tests__/e2e/` + +**`packages/aft-bridge/`:** +- Purpose: Share NDJSON bridge transport, binary resolution, ONNX runtime helpers, and URL fetch across all harness adapters. +- Contains: `src/` TypeScript sources, tests, package manifest +- Key files: `packages/aft-bridge/src/bridge.ts`, `packages/aft-bridge/src/pool.ts`, `packages/aft-bridge/src/downloader.ts`, `packages/aft-bridge/src/resolver.ts`, `packages/aft-bridge/src/onnx-runtime.ts`, `packages/aft-bridge/src/url-fetch.ts` +- Used by: `packages/opencode-plugin/` and `packages/pi-plugin/` (both import from `@cortexkit/aft-bridge`) + +**`packages/aft-cli/`:** +- Purpose: Provide the unified `npx @cortexkit/aft` CLI for setup, doctor, and filter management across all harnesses. +- Contains: `src/` TypeScript sources with harness-specific adapters and commands +- Key files: `packages/aft-cli/src/index.ts`, `packages/aft-cli/src/commands/doctor.ts`, `packages/aft-cli/src/commands/setup.ts`, `packages/aft-cli/src/adapters/opencode.ts`, `packages/aft-cli/src/adapters/pi.ts` + +**`packages/opencode-plugin/`:** +- Purpose: Ship the OpenCode-facing adapter that resolves the binary, manages the bridge pool, and registers AFT tools with the harness. +- Contains: `src/` TypeScript sources, `dist/` build output, tests, package manifest +- Key files: `packages/opencode-plugin/src/index.ts`, `packages/opencode-plugin/src/config.ts`, `packages/opencode-plugin/package.json` + +**`packages/pi-plugin/`:** +- Purpose: Ship the Pi coding agent adapter that registers AFT tools with the Pi harness. +- Contains: `src/` TypeScript sources, `dist/` build output, tests, package manifest +- Key files: `packages/pi-plugin/src/index.ts`, `packages/pi-plugin/src/config.ts`, `packages/pi-plugin/package.json` +- Same tool surface as opencode-plugin, adapted to Pi's plugin API **`packages/npm/`:** - Purpose: Publish one npm package per target platform so the plugin can resolve a bundled binary. @@ -67,11 +103,11 @@ opencode-aft/ ## Key File Locations -**Entry Points:** `packages/opencode-plugin/src/index.ts`: Register plugin tools and bridge configuration; `crates/aft/src/main.rs`: Start the Rust request loop; `.github/workflows/release.yml`: Drive tagged release publishing. +**Entry Points:** `packages/opencode-plugin/src/index.ts`: Register OpenCode plugin tools and bridge configuration; `packages/pi-plugin/src/index.ts`: Register Pi plugin tools; `packages/aft-cli/src/index.ts`: Dispatch CLI commands (`setup`, `doctor`); `crates/aft/src/main.rs`: Start the Rust request loop; `.github/workflows/release.yml`: Drive tagged release publishing. **Configuration:** `package.json`: Define Bun workspace scripts; `Cargo.toml`: Define the Rust workspace; `packages/opencode-plugin/src/config.ts`: Parse user and project AFT config. -**Core Logic:** `crates/aft/src/parser.rs`: Extract symbols and languages; `crates/aft/src/callgraph.rs`: Build navigation indexes; `crates/aft/src/edit.rs`: Run shared edit and diff logic; `packages/opencode-plugin/src/bridge.ts`: Manage subprocess transport. +**Core Logic:** `crates/aft/src/parser.rs`: Extract symbols and languages; `crates/aft/src/callgraph.rs`: Build navigation indexes; `crates/aft/src/edit.rs`: Run shared edit and diff logic; `crates/aft/src/semantic_index.rs`: Embed and search code by meaning; `crates/aft/src/vector_store.rs`: Vector storage abstraction; `packages/aft-bridge/src/bridge.ts`: Manage subprocess transport. **Tests:** `packages/opencode-plugin/src/__tests__/`: Plugin unit and e2e tests; `crates/aft/tests/integration/`: Rust integration tests. @@ -85,16 +121,26 @@ opencode-aft/ **New hoisted OpenCode file tool:** `packages/opencode-plugin/src/tools/hoisted.ts` — register the tool and map it onto a Rust command. -**New plugin tool group:** `packages/opencode-plugin/src/tools/[capability].ts` — export a `Record` and wire it into `packages/opencode-plugin/src/index.ts`. +**New plugin tool group (OpenCode):** `packages/opencode-plugin/src/tools/[capability].ts` — export a `Record` and wire it into `packages/opencode-plugin/src/index.ts`. + +**New plugin tool group (Pi):** `packages/pi-plugin/src/tools/[capability].ts` — export a `Record` and wire it into `packages/pi-plugin/src/index.ts`. + +**New shared transport / binary-resolution code:** `packages/aft-bridge/src/[module].ts` — keep shared primitives (bridge, pool, downloader, resolver, ONNX, URL fetch) that both harness adapters consume. + +**New unified CLI command:** `packages/aft-cli/src/commands/[command].ts` — add the handler and dispatch it from `packages/aft-cli/src/index.ts`. **New Rust command handler:** `crates/aft/src/commands/[command_name].rs` — expose the handler from `crates/aft/src/commands/mod.rs` and dispatch it from `crates/aft/src/main.rs`. -**New shared Rust engine code:** `crates/aft/src/[domain].rs` — keep reusable parser, formatter, import, or analysis logic outside command handlers. +**New shared Rust engine code:** `crates/aft/src/[domain].rs` — keep reusable parser, formatter, import, analysis, or semantic code outside command handlers. **New LSP behavior:** `crates/aft/src/lsp/[module].rs` — keep transport and server-management code inside the LSP subsystem. +**New tokenizer or Claude encoding code:** `crates/aft-tokenizer/src/[module].rs` — keep the tokenizer crate focused on Claude-compatible lookup encoding. + **New platform binary package:** `packages/npm/[platform-key]/` — add `package.json` and ship the platform binary in `bin/`. -**New plugin tests:** `packages/opencode-plugin/src/__tests__/` or `packages/opencode-plugin/src/__tests__/e2e/` — follow the existing `*.test.ts` naming. +**New plugin tests (OpenCode):** `packages/opencode-plugin/src/__tests__/` or `packages/opencode-plugin/src/__tests__/e2e/` — follow the existing `*.test.ts` naming. + +**New plugin tests (Pi):** `packages/pi-plugin/src/__tests__/` — follow the existing `*.test.ts` naming. **New Rust integration tests:** `crates/aft/tests/integration/` — follow the existing `*_test.rs` naming. diff --git a/benchmarks/compression-tokens/data/spike-output.json b/benchmarks/compression-tokens/data/spike-output.json index 89a973de..94838d3d 100644 --- a/benchmarks/compression-tokens/data/spike-output.json +++ b/benchmarks/compression-tokens/data/spike-output.json @@ -4,9 +4,9 @@ "command": "git status --short --branch", "category": "git", "tier": "rust modules", - "original_bytes": 214, + "original_bytes": 220, "compressed_bytes": 213, - "original_text": "## feature/compress-metrics...origin/feature/compress-metrics [ahead 3]\n M crates/aft/src/compress/mod.rs\n M crates/aft/src/commands/bash.rs\n M Cargo.lock\n?? benchmarks/compression-tokens/\n?? tmp/spike-output.json\n", + "original_text": "## feature/compress-metrics...origin/feature/compress-metrics [ahead 3]\r\n M crates/aft/src/compress/mod.rs\r\n M crates/aft/src/commands/bash.rs\r\n M Cargo.lock\r\n?? benchmarks/compression-tokens/\r\n?? tmp/spike-output.json\r\n", "compressed_text": "## feature/compress-metrics...origin/feature/compress-metrics [ahead 3]\n M crates/aft/src/compress/mod.rs\n M crates/aft/src/commands/bash.rs\n M Cargo.lock\n?? benchmarks/compression-tokens/\n?? tmp/spike-output.json" }, { @@ -14,9 +14,9 @@ "command": "git log --oneline --decorate -25", "category": "git", "tier": "rust modules", - "original_bytes": 560, + "original_bytes": 570, "compressed_bytes": 559, - "original_text": "e4e8f7e (HEAD -> feature/compress-metrics, origin/main) chore(release): v0.26.4\n9c4aa18 feat(compress): add builtin filters for kubectl and gh\n651bb01 fix(bash): preserve completion frames for background tasks\n37f9a72 test(compress): cover tsc pretty output\n0b51408 feat(compress): add biome compressor\nb11c850 docs: update v0.27 sqlite storage plan\n8a871dd refactor(config): normalize storage dir lookup\n4a1d7b8 feat(lsp): add pull diagnostics fallback\nf70c533 fix(imports): handle type-only namespace imports\n2c55219 perf(search): cap embedding batch memory\n", + "original_text": "e4e8f7e (HEAD -> feature/compress-metrics, origin/main) chore(release): v0.26.4\r\n9c4aa18 feat(compress): add builtin filters for kubectl and gh\r\n651bb01 fix(bash): preserve completion frames for background tasks\r\n37f9a72 test(compress): cover tsc pretty output\r\n0b51408 feat(compress): add biome compressor\r\nb11c850 docs: update v0.27 sqlite storage plan\r\n8a871dd refactor(config): normalize storage dir lookup\r\n4a1d7b8 feat(lsp): add pull diagnostics fallback\r\nf70c533 fix(imports): handle type-only namespace imports\r\n2c55219 perf(search): cap embedding batch memory\r\n", "compressed_text": "e4e8f7e (HEAD -> feature/compress-metrics, origin/main) chore(release): v0.26.4\n9c4aa18 feat(compress): add builtin filters for kubectl and gh\n651bb01 fix(bash): preserve completion frames for background tasks\n37f9a72 test(compress): cover tsc pretty output\n0b51408 feat(compress): add biome compressor\nb11c850 docs: update v0.27 sqlite storage plan\n8a871dd refactor(config): normalize storage dir lookup\n4a1d7b8 feat(lsp): add pull diagnostics fallback\nf70c533 fix(imports): handle type-only namespace imports\n2c55219 perf(search): cap embedding batch memory" }, { @@ -24,9 +24,9 @@ "command": "git diff -- crates/aft/src/compress/mod.rs", "category": "git", "tier": "rust modules", - "original_bytes": 997, + "original_bytes": 1019, "compressed_bytes": 996, - "original_text": "diff --git a/crates/aft/src/compress/mod.rs b/crates/aft/src/compress/mod.rs\nindex e2a94b1..8cbe201 100644\n--- a/crates/aft/src/compress/mod.rs\n+++ b/crates/aft/src/compress/mod.rs\n@@ -84,6 +84,17 @@ pub fn compress_with_registry(command: &str, output: &str, registry: &FilterRegi\n compress_with_registry(command, &output, &guard)\n }\n+\n+#[cfg(test)]\n+pub fn compress_for_spike(command: &str, output: &str) -> String {\n+ let registry = toml_filter::build_registry(builtin_filters::ALL, None, None);\n+ compress_with_registry(command, output, ®istry)\n+}\n+\n /// Thread-safe dispatch that does not need `AppContext`. Caller is responsible\n /// for the `experimental_bash_compress` gate (the registry has no opinion).\n@@ -99,7 +110,7 @@ pub fn compress_with_registry(command: &str, output: &str, registry: &FilterRegi\n- let compressors: [&dyn Compressor; 9] = [\n+ let compressors: [&dyn Compressor; 10] = [\n &GitCompressor,\n &CargoCompressor,\n &TscCompressor,\n", + "original_text": "diff --git a/crates/aft/src/compress/mod.rs b/crates/aft/src/compress/mod.rs\r\nindex e2a94b1..8cbe201 100644\r\n--- a/crates/aft/src/compress/mod.rs\r\n+++ b/crates/aft/src/compress/mod.rs\r\n@@ -84,6 +84,17 @@ pub fn compress_with_registry(command: &str, output: &str, registry: &FilterRegi\r\n compress_with_registry(command, &output, &guard)\r\n }\r\n+\r\n+#[cfg(test)]\r\n+pub fn compress_for_spike(command: &str, output: &str) -> String {\r\n+ let registry = toml_filter::build_registry(builtin_filters::ALL, None, None);\r\n+ compress_with_registry(command, output, ®istry)\r\n+}\r\n+\r\n /// Thread-safe dispatch that does not need `AppContext`. Caller is responsible\r\n /// for the `experimental_bash_compress` gate (the registry has no opinion).\r\n@@ -99,7 +110,7 @@ pub fn compress_with_registry(command: &str, output: &str, registry: &FilterRegi\r\n- let compressors: [&dyn Compressor; 9] = [\r\n+ let compressors: [&dyn Compressor; 10] = [\r\n &GitCompressor,\r\n &CargoCompressor,\r\n &TscCompressor,\r\n", "compressed_text": "diff --git a/crates/aft/src/compress/mod.rs b/crates/aft/src/compress/mod.rs\nindex e2a94b1..8cbe201 100644\n--- a/crates/aft/src/compress/mod.rs\n+++ b/crates/aft/src/compress/mod.rs\n@@ -84,6 +84,17 @@ pub fn compress_with_registry(command: &str, output: &str, registry: &FilterRegi\n compress_with_registry(command, &output, &guard)\n }\n+\n+#[cfg(test)]\n+pub fn compress_for_spike(command: &str, output: &str) -> String {\n+ let registry = toml_filter::build_registry(builtin_filters::ALL, None, None);\n+ compress_with_registry(command, output, ®istry)\n+}\n+\n /// Thread-safe dispatch that does not need `AppContext`. Caller is responsible\n /// for the `experimental_bash_compress` gate (the registry has no opinion).\n@@ -99,7 +110,7 @@ pub fn compress_with_registry(command: &str, output: &str, registry: &FilterRegi\n- let compressors: [&dyn Compressor; 9] = [\n+ let compressors: [&dyn Compressor; 10] = [\n &GitCompressor,\n &CargoCompressor,\n &TscCompressor," }, { @@ -34,9 +34,9 @@ "command": "git fetch origin main", "category": "git", "tier": "rust modules", - "original_bytes": 495, + "original_bytes": 505, "compressed_bytes": 122, - "original_text": "remote: Enumerating objects: 42, done.\nremote: Counting objects: 100% (42/42), done.\nremote: Compressing objects: 100% (18/18), done.\nremote: Total 24 (delta 14), reused 17 (delta 6), pack-reused 0\nUnpacking objects: 100% (24/24), 6.81 KiB | 697.00 KiB/s, done.\nFrom github.com:cortexkit/aft\n * branch main -> FETCH_HEAD\n e4e8f7e..4af3b19 main -> origin/main\nAuto packing the repository in background for optimum performance.\nSee \"git help gc\" for manual housekeeping.\n", + "original_text": "remote: Enumerating objects: 42, done.\r\nremote: Counting objects: 100% (42/42), done.\r\nremote: Compressing objects: 100% (18/18), done.\r\nremote: Total 24 (delta 14), reused 17 (delta 6), pack-reused 0\r\nUnpacking objects: 100% (24/24), 6.81 KiB | 697.00 KiB/s, done.\r\nFrom github.com:cortexkit/aft\r\n * branch main -> FETCH_HEAD\r\n e4e8f7e..4af3b19 main -> origin/main\r\nAuto packing the repository in background for optimum performance.\r\nSee \"git help gc\" for manual housekeeping.\r\n", "compressed_text": "From github.com:cortexkit/aft\n * branch main -> FETCH_HEAD\n e4e8f7e..4af3b19 main -> origin/main" }, { @@ -44,9 +44,9 @@ "command": "git push origin feature/compress-metrics", "category": "git", "tier": "rust modules", - "original_bytes": 623, + "original_bytes": 636, "compressed_bytes": 105, - "original_text": "Enumerating objects: 18, done.\nCounting objects: 100% (18/18), done.\nDelta compression using up to 10 threads\nCompressing objects: 100% (12/12), done.\nWriting objects: 100% (12/12), 3.21 KiB | 3.21 MiB/s, done.\nTotal 12 (delta 8), reused 0 (delta 0), pack-reused 0\nremote: Resolving deltas: 100% (8/8), completed with 5 local objects.\nremote: \nremote: Create a pull request for 'feature/compress-metrics' on GitHub by visiting:\nremote: https://github.com/cortexkit/aft/pull/new/feature/compress-metrics\nremote: \nTo github.com:cortexkit/aft.git\n * [new branch] feature/compress-metrics -> feature/compress-metrics\n", + "original_text": "Enumerating objects: 18, done.\r\nCounting objects: 100% (18/18), done.\r\nDelta compression using up to 10 threads\r\nCompressing objects: 100% (12/12), done.\r\nWriting objects: 100% (12/12), 3.21 KiB | 3.21 MiB/s, done.\r\nTotal 12 (delta 8), reused 0 (delta 0), pack-reused 0\r\nremote: Resolving deltas: 100% (8/8), completed with 5 local objects.\r\nremote: \r\nremote: Create a pull request for 'feature/compress-metrics' on GitHub by visiting:\r\nremote: https://github.com/cortexkit/aft/pull/new/feature/compress-metrics\r\nremote: \r\nTo github.com:cortexkit/aft.git\r\n * [new branch] feature/compress-metrics -> feature/compress-metrics\r\n", "compressed_text": "To github.com:cortexkit/aft.git\n * [new branch] feature/compress-metrics -> feature/compress-metrics" }, { @@ -54,9 +54,9 @@ "command": "cargo test", "category": "build-test", "tier": "rust modules", - "original_bytes": 1335, + "original_bytes": 1365, "compressed_bytes": 259, - "original_text": " Compiling agent-file-tools v0.26.4 (/Users/ufukaltinok/Work/OSS/opencode-aft/crates/aft)\nwarning: function `normalize_command` is never used\n --> crates/aft/src/compress/git.rs:218:4\n |\n218 | fn normalize_command(command: &str) -> String {\n | ^^^^^^^^^^^^^^^^^\n |\n = note: `#[warn(dead_code)]` on by default\nwarning: `agent-file-tools` (lib test) generated 1 warning\n Finished `test` profile [unoptimized + debuginfo] target(s) in 7.42s\n Running unittests src/lib.rs (target/debug/deps/aft-3e63e65b6f8e5a12)\n\nrunning 312 tests\ntest compress::git::tests::status_short_preserves_branch ... ok\ntest compress::cargo::tests::test_summary_keeps_failures ... ok\ntest commands::bash::tests::try_spawn_with_login_shell ... ok\ntest lsp::tests::pull_diagnostics_prefers_317 ... ok\ntest imports::tests::organize_groups_external_before_internal ... ok\ntest search_index::tests::incremental_cache_reuses_embeddings ... ok\n\ntest result: ok. 312 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 5.86s\n\n Running tests/compress_filters.rs (target/debug/deps/compress_filters-ea287c4a1a64c0e8)\n\nrunning 18 tests\ntest builtin_filters_are_parseable ... ok\ntest terraform_plan_filter_caps_middle ... ok\ntest kubectl_get_pods_strips_age_noise ... ok\n\ntest result: ok. 18 passed; 0 failed; finished in 0.09s\n", + "original_text": " Compiling agent-file-tools v0.26.4 (/Users/ufukaltinok/Work/OSS/opencode-aft/crates/aft)\r\nwarning: function `normalize_command` is never used\r\n --> crates/aft/src/compress/git.rs:218:4\r\n |\r\n218 | fn normalize_command(command: &str) -> String {\r\n | ^^^^^^^^^^^^^^^^^\r\n |\r\n = note: `#[warn(dead_code)]` on by default\r\nwarning: `agent-file-tools` (lib test) generated 1 warning\r\n Finished `test` profile [unoptimized + debuginfo] target(s) in 7.42s\r\n Running unittests src/lib.rs (target/debug/deps/aft-3e63e65b6f8e5a12)\r\n\r\nrunning 312 tests\r\ntest compress::git::tests::status_short_preserves_branch ... ok\r\ntest compress::cargo::tests::test_summary_keeps_failures ... ok\r\ntest commands::bash::tests::try_spawn_with_login_shell ... ok\r\ntest lsp::tests::pull_diagnostics_prefers_317 ... ok\r\ntest imports::tests::organize_groups_external_before_internal ... ok\r\ntest search_index::tests::incremental_cache_reuses_embeddings ... ok\r\n\r\ntest result: ok. 312 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 5.86s\r\n\r\n Running tests/compress_filters.rs (target/debug/deps/compress_filters-ea287c4a1a64c0e8)\r\n\r\nrunning 18 tests\r\ntest builtin_filters_are_parseable ... ok\r\ntest terraform_plan_filter_caps_middle ... ok\r\ntest kubectl_get_pods_strips_age_noise ... ok\r\n\r\ntest result: ok. 18 passed; 0 failed; finished in 0.09s\r\n", "compressed_text": " Finished `test` profile [unoptimized + debuginfo] target(s) in 7.42s\nrunning 312 tests\ntest result: ok. 312 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 5.86s\nrunning 18 tests\ntest result: ok. 18 passed; 0 failed; finished in 0.09s" }, { @@ -64,19 +64,19 @@ "command": "cargo build --release", "category": "build-test", "tier": "rust modules", - "original_bytes": 501, - "compressed_bytes": 500, - "original_text": " Compiling libc v0.2.177\n Compiling proc-macro2 v1.0.101\n Compiling unicode-ident v1.0.19\n Compiling quote v1.0.41\n Compiling serde_core v1.0.228\n Compiling memchr v2.7.6\n Compiling aho-corasick v1.1.3\n Compiling regex-syntax v0.8.8\n Compiling serde v1.0.228\n Compiling regex-automata v0.4.13\n Compiling tree-sitter v0.26.2\n Compiling agent-file-tools v0.26.4 (/Users/ufukaltinok/Work/OSS/opencode-aft/crates/aft)\n Finished `release` profile [optimized] target(s) in 38.74s\n", - "compressed_text": " Compiling libc v0.2.177\n Compiling proc-macro2 v1.0.101\n Compiling unicode-ident v1.0.19\n Compiling quote v1.0.41\n Compiling serde_core v1.0.228\n Compiling memchr v2.7.6\n Compiling aho-corasick v1.1.3\n Compiling regex-syntax v0.8.8\n Compiling serde v1.0.228\n Compiling regex-automata v0.4.13\n Compiling tree-sitter v0.26.2\n Compiling agent-file-tools v0.26.4 (/Users/ufukaltinok/Work/OSS/opencode-aft/crates/aft)\n Finished `release` profile [optimized] target(s) in 38.74s" + "original_bytes": 514, + "compressed_bytes": 512, + "original_text": " Compiling libc v0.2.177\r\n Compiling proc-macro2 v1.0.101\r\n Compiling unicode-ident v1.0.19\r\n Compiling quote v1.0.41\r\n Compiling serde_core v1.0.228\r\n Compiling memchr v2.7.6\r\n Compiling aho-corasick v1.1.3\r\n Compiling regex-syntax v0.8.8\r\n Compiling serde v1.0.228\r\n Compiling regex-automata v0.4.13\r\n Compiling tree-sitter v0.26.2\r\n Compiling agent-file-tools v0.26.4 (/Users/ufukaltinok/Work/OSS/opencode-aft/crates/aft)\r\n Finished `release` profile [optimized] target(s) in 38.74s\r\n", + "compressed_text": " Compiling libc v0.2.177\r\n Compiling proc-macro2 v1.0.101\r\n Compiling unicode-ident v1.0.19\r\n Compiling quote v1.0.41\r\n Compiling serde_core v1.0.228\r\n Compiling memchr v2.7.6\r\n Compiling aho-corasick v1.1.3\r\n Compiling regex-syntax v0.8.8\r\n Compiling serde v1.0.228\r\n Compiling regex-automata v0.4.13\r\n Compiling tree-sitter v0.26.2\r\n Compiling agent-file-tools v0.26.4 (/Users/ufukaltinok/Work/OSS/opencode-aft/crates/aft)\r\n Finished `release` profile [optimized] target(s) in 38.74s" }, { "file": "build-test/npm-install.txt", "command": "npm install", "category": "build-test", "tier": "rust modules", - "original_bytes": 639, + "original_bytes": 658, "compressed_bytes": 312, - "original_text": "npm WARN EBADENGINE Unsupported engine {\nnpm WARN EBADENGINE package: 'vite@7.2.2',\nnpm WARN EBADENGINE required: { node: '^20.19.0 || >=22.12.0' },\nnpm WARN EBADENGINE current: { node: 'v20.11.1', npm: '10.2.4' }\nnpm WARN EBADENGINE }\nnpm WARN deprecated inflight@1.0.6: This module is not supported, and leaks memory.\nnpm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported\n\nadded 428 packages, and audited 429 packages in 12s\n\n82 packages are looking for funding\n run `npm fund` for details\n\n3 moderate severity vulnerabilities\n\nTo address all issues, run:\n npm audit fix\n\nRun `npm audit` for details.\n", + "original_text": "npm WARN EBADENGINE Unsupported engine {\r\nnpm WARN EBADENGINE package: 'vite@7.2.2',\r\nnpm WARN EBADENGINE required: { node: '^20.19.0 || >=22.12.0' },\r\nnpm WARN EBADENGINE current: { node: 'v20.11.1', npm: '10.2.4' }\r\nnpm WARN EBADENGINE }\r\nnpm WARN deprecated inflight@1.0.6: This module is not supported, and leaks memory.\r\nnpm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported\r\n\r\nadded 428 packages, and audited 429 packages in 12s\r\n\r\n82 packages are looking for funding\r\n run `npm fund` for details\r\n\r\n3 moderate severity vulnerabilities\r\n\r\nTo address all issues, run:\r\n npm audit fix\r\n\r\nRun `npm audit` for details.\r\n", "compressed_text": "npm WARN deprecated inflight@1.0.6: This module is not supported, and leaks memory.\nnpm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported\n82 packages are looking for funding\n3 moderate severity vulnerabilities\n\nTo address all issues, run:\n npm audit fix\n\nRun `npm audit` for details." }, { @@ -84,9 +84,9 @@ "command": "pnpm install", "category": "build-test", "tier": "rust modules", - "original_bytes": 540, + "original_bytes": 558, "compressed_bytes": 180, - "original_text": "Scope: all 7 workspace projects\nLockfile is up to date, resolution step is skipped\nPackages: +821\n++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\nProgress: resolved 821, reused 814, downloaded 0, added 0\nProgress: resolved 821, reused 814, downloaded 0, added 138\nProgress: resolved 821, reused 814, downloaded 0, added 821, done\n\ndependencies:\n+ @modelcontextprotocol/sdk 1.18.1\n+ ai-tokenizer 1.0.6\n+ zod 4.1.12\n\ndevDependencies:\n+ @biomejs/biome 2.4.7\n+ typescript 5.8.3\n\nDone in 4.8s using pnpm v9.15.9\n", + "original_text": "Scope: all 7 workspace projects\r\nLockfile is up to date, resolution step is skipped\r\nPackages: +821\r\n++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\r\nProgress: resolved 821, reused 814, downloaded 0, added 0\r\nProgress: resolved 821, reused 814, downloaded 0, added 138\r\nProgress: resolved 821, reused 814, downloaded 0, added 821, done\r\n\r\ndependencies:\r\n+ @modelcontextprotocol/sdk 1.18.1\r\n+ ai-tokenizer 1.0.6\r\n+ zod 4.1.12\r\n\r\ndevDependencies:\r\n+ @biomejs/biome 2.4.7\r\n+ typescript 5.8.3\r\n\r\nDone in 4.8s using pnpm v9.15.9\r\n", "compressed_text": "Progress: resolved 821, reused 814, downloaded 0, added 0\nProgress: resolved 821, reused 814, downloaded 0, added 138\ndependencies:\ndevDependencies:\nDone in 4.8s using pnpm v9.15.9" }, { @@ -94,9 +94,9 @@ "command": "pytest -q", "category": "build-test", "tier": "rust modules", - "original_bytes": 1602, + "original_bytes": 1632, "compressed_bytes": 877, - "original_text": "============================= test session starts ==============================\nplatform darwin -- Python 3.12.4, pytest-8.3.3, pluggy-1.5.0\nrootdir: /Users/ufukaltinok/Work/OSS/example-service\nconfigfile: pyproject.toml\nplugins: anyio-4.6.0, asyncio-0.24.0, cov-5.0.0\ncollected 146 items\n\ntests/test_api.py ...................... [ 15%]\ntests/test_auth.py .............F.... [ 27%]\ntests/test_cache.py ........................ [ 43%]\ntests/test_cli.py ....................... [ 58%]\ntests/test_storage.py ............................... [ 79%]\ntests/test_workers.py .............................. [100%]\n\n=================================== FAILURES ===================================\n_______________________ test_refresh_token_rejects_reuse _______________________\n\nclient = \n\n async def test_refresh_token_rejects_reuse(client):\n first = await client.post('/auth/refresh', json={'token': TOKEN})\n second = await client.post('/auth/refresh', json={'token': TOKEN})\n> assert second.status_code == 401\nE assert 200 == 401\nE + where 200 = .status_code\n\ntests/test_auth.py:87: AssertionError\n=========================== short test summary info ============================\nFAILED tests/test_auth.py::test_refresh_token_rejects_reuse - assert 200 == 401\n======================== 1 failed, 145 passed in 9.41s =========================\n", + "original_text": "============================= test session starts ==============================\r\nplatform darwin -- Python 3.12.4, pytest-8.3.3, pluggy-1.5.0\r\nrootdir: /Users/ufukaltinok/Work/OSS/example-service\r\nconfigfile: pyproject.toml\r\nplugins: anyio-4.6.0, asyncio-0.24.0, cov-5.0.0\r\ncollected 146 items\r\n\r\ntests/test_api.py ...................... [ 15%]\r\ntests/test_auth.py .............F.... [ 27%]\r\ntests/test_cache.py ........................ [ 43%]\r\ntests/test_cli.py ....................... [ 58%]\r\ntests/test_storage.py ............................... [ 79%]\r\ntests/test_workers.py .............................. [100%]\r\n\r\n=================================== FAILURES ===================================\r\n_______________________ test_refresh_token_rejects_reuse _______________________\r\n\r\nclient = \r\n\r\n async def test_refresh_token_rejects_reuse(client):\r\n first = await client.post('/auth/refresh', json={'token': TOKEN})\r\n second = await client.post('/auth/refresh', json={'token': TOKEN})\r\n> assert second.status_code == 401\r\nE assert 200 == 401\r\nE + where 200 = .status_code\r\n\r\ntests/test_auth.py:87: AssertionError\r\n=========================== short test summary info ============================\r\nFAILED tests/test_auth.py::test_refresh_token_rejects_reuse - assert 200 == 401\r\n======================== 1 failed, 145 passed in 9.41s =========================\r\n", "compressed_text": "platform darwin -- Python 3.12.4, pytest-8.3.3, pluggy-1.5.0\nrootdir: /Users/ufukaltinok/Work/OSS/example-service\ncollected 146 items\n=================================== FAILURES ===================================\n_______________________ test_refresh_token_rejects_reuse _______________________\n\nclient = \n\n async def test_refresh_token_rejects_reuse(client):\n first = await client.post('/auth/refresh', json={'token': TOKEN})\n second = await client.post('/auth/refresh', json={'token': TOKEN})\n> assert second.status_code == 401\nE assert 200 == 401\nE + where 200 = .status_code\n\ntests/test_auth.py:87: AssertionError\n=========================== short test summary info ============================\n======================== 1 failed, 145 passed in 9.41s =========================" }, { @@ -104,9 +104,9 @@ "command": "eslint . --format stylish", "category": "lint", "tier": "rust modules", - "original_bytes": 619, + "original_bytes": 630, "compressed_bytes": 546, - "original_text": "\n/Users/ufukaltinok/Work/OSS/web/src/App.tsx\n 12:7 warning 'unused' is assigned a value but never used @typescript-eslint/no-unused-vars\n 48:13 error Unexpected any. Specify a different type @typescript-eslint/no-explicit-any\n 93:5 error React Hook useEffect has a missing dependency react-hooks/exhaustive-deps\n\n/Users/ufukaltinok/Work/OSS/web/src/lib/api.ts\n 21:10 error 'ResponsePayload' is defined but never used @typescript-eslint/no-unused-vars\n 77:3 warning Unexpected console statement no-console\n\n✖ 5 problems (3 errors, 2 warnings)\n", + "original_text": "\r\n/Users/ufukaltinok/Work/OSS/web/src/App.tsx\r\n 12:7 warning 'unused' is assigned a value but never used @typescript-eslint/no-unused-vars\r\n 48:13 error Unexpected any. Specify a different type @typescript-eslint/no-explicit-any\r\n 93:5 error React Hook useEffect has a missing dependency react-hooks/exhaustive-deps\r\n\r\n/Users/ufukaltinok/Work/OSS/web/src/lib/api.ts\r\n 21:10 error 'ResponsePayload' is defined but never used @typescript-eslint/no-unused-vars\r\n 77:3 warning Unexpected console statement no-console\r\n\r\n✖ 5 problems (3 errors, 2 warnings)\r\n", "compressed_text": "/Users/ufukaltinok/Work/OSS/web/src/App.tsx\n 12:7 warning @typescript-eslint/no-unused-vars 'unused' is assigned a value but never used\n 48:13 error @typescript-eslint/no-explicit-any Unexpected any. Specify a different type\n 93:5 error react-hooks/exhaustive-deps React Hook useEffect has a missing dependency\n/Users/ufukaltinok/Work/OSS/web/src/lib/api.ts\n 21:10 error @typescript-eslint/no-unused-vars 'ResponsePayload' is defined but never used\n 77:3 warning no-console Unexpected console statement\n\n✖ 5 problems (3 errors, 2 warnings)" }, { @@ -114,9 +114,9 @@ "command": "biome check .", "category": "lint", "tier": "rust modules", - "original_bytes": 900, + "original_bytes": 921, "compressed_bytes": 61, - "original_text": "src/hooks/useSession.ts:14:7 lint/correctness/noUnusedVariables ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\n ✖ This variable is unused.\n\n 12 │ export function useSession() {\n 13 │ const [session, setSession] = useState(null);\n > 14 │ const debugSession = session;\n │ ^^^^^^^^^^^^\n 15 │ return session;\n 16 │ }\n\nsrc/components/Button.tsx format ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\n ✖ Formatter would have printed the following content:\n\n 8 8 │ export function Button(props: Props) {\n 9 │ - return