diff --git a/.agents/skills/land-pr/SKILL.md b/.agents/skills/land-pr/SKILL.md index 3164ff0..b1cfd8f 100644 --- a/.agents/skills/land-pr/SKILL.md +++ b/.agents/skills/land-pr/SKILL.md @@ -5,16 +5,18 @@ description: > Makes the PR self-contained: rewrites the PR description from the PR's own contents, appends a changelog [Unreleased] entry when warranted, regenerates the English dev notes from the Chinese source when touched, and keeps the bilingual - README in sync. Then — only after explicit maintainer confirmation — merges the - PR. Does NOT tag or publish. Triggers on + README in sync. It also checks bilingual research notes and reminds the maintainer + when their English version needs refreshing. Then — only after explicit maintainer + confirmation — merges the PR. Does NOT tag or publish. Triggers on "合并 PR / land PR / 合并这个 PR / merge this PR". --- # Land a PR Merge a feature/bugfix PR into `main`, making the PR **self-contained** first: -its code, changelog entry, dev notes (zh + en), and README (zh + en) all in sync -within the PR. This skill never tags or publishes — that is the `release` skill's job. +its code, changelog entry, dev notes (zh + en), research-note translation status, +and README (zh + en) all accounted for within the PR. This skill never tags or +publishes — that is the `release` skill's job. ## Prerequisites @@ -31,38 +33,46 @@ file, e.g. `docs/changelogs/0.1.x.md` / `docs/dev_notes/{zh-CN,en}/0.1.x.md`. - `git status --porcelain` — empty (clean tree). - current branch is **not** `main` (`git branch --show-current`). -### 2. Ensure a PR exists +### 2. Check English research notes before PR creation +If the branch modified any `docs/research/zh-CN/.md`, check the +corresponding `docs/research/en/.md`. Explicitly remind the maintainer +that the English version must also be translated or refreshed, and report +whether it is in sync. Do this even when the English file was already updated. +If it is missing or stale, surface that clearly before opening or landing the +PR; do not silently treat the Chinese-only change as complete. + +### 3. Ensure a PR exists ```bash gh pr view --json number,url 2>/dev/null || gh pr create --fill ``` Note the PR number for later steps. -### 3. Append a changelog entry (when warranted) +### 4. Append a changelog entry (when warranted) Inspect `git diff main...HEAD`. If the changes are worth recording for users, add an entry under `## [Unreleased]` in `` (changelogs) in the right group (Added / Changed / Fixed / Removed). Purely internal/tooling changes may need **no** entry — decide, and surface the decision at the gate. -### 4. Sync English dev notes (if the Chinese source changed) +### 5. Sync English dev notes (if the Chinese source changed) If this PR modified `docs/dev_notes/zh-CN/.md`, regenerate the whole `docs/dev_notes/en/.md` by translating the Chinese source. The English file is generated — do not hand-edit beyond this regeneration. -### 5. Keep the bilingual README in sync +### 6. Keep the bilingual README in sync If this PR changed only one of `README.md` / `README.zh-CN.md`, mirror the change into the other (translate/align) so the pair stays consistent. Both READMEs are hand-written sources — propose the synced change and let the maintainer adjust the wording at the gate; do not blindly overwrite. If both were already changed, skip. -### 6. Commit and push the sync changes -Commit any changes from steps 3–5 to the feature branch and push: +### 7. Commit and push the sync changes +Commit any changes from steps 4–6 to the feature branch and push: ```bash git commit -am "docs: sync changelog/dev-notes/README for this PR" git push ``` -(Skip if steps 3–5 produced no changes.) +(Skip if steps 4–6 produced no changes.) -### 7. Rewrite the PR description from scratch +### 8. Rewrite the PR description from scratch Build the description **only** from the PR's actual contents — commits, changed files, `gh pr diff`. **Do not read or extend the old description** (it may be stale). Update via the GitHub API (`gh pr edit` can fail on deprecated Projects @@ -71,13 +81,14 @@ Classic): gh api repos/{owner}/{repo}/pulls/{number} -X PATCH -f body="..." --silent ``` -### 8. ⛔ Confirmation gate (mandatory) +### 9. ⛔ Confirmation gate (mandatory) Present to the maintainer: the rewritten **PR description**, the **changelog entry** added (or "none, because …"), and any **dev notes / README sync** done -(or a README drift warning). **Wait for explicit confirmation** (e.g. "确认 / go"). -Do not merge until approved. +(or a README drift warning), plus the **English research-note sync status** when +applicable. **Wait for explicit confirmation** (e.g. "确认 / go"). Do not merge +until approved. -### 9. Merge +### 10. Merge ```bash gh pr merge --merge ``` diff --git a/AGENTS.md b/AGENTS.md index e0896b6..36ad5af 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -37,6 +37,19 @@ The **only** exception is documentation explicitly designated as Chinese: Everything outside that list — including code under `src/`, this file, and all other docs — is English. +### Bilingual research notes + +Research notes are bilingual: + +- `docs/research/zh-CN/` contains the hand-written Chinese sources of truth. +- `docs/research/en/` contains English versions generated from the Chinese + sources; regenerate the whole corresponding file instead of hand-editing it. + +When a change to a Chinese research source is headed into a pull request, the +agent MUST remind the user that the corresponding English version also needs to +be translated or refreshed. Before opening or landing the PR, report whether +the English version is in sync, even when it has already been updated. + ## Commits & PRs All agents working in this repo MUST follow diff --git a/docs/changelogs/0.7.x.md b/docs/changelogs/0.7.x.md new file mode 100644 index 0000000..180d997 --- /dev/null +++ b/docs/changelogs/0.7.x.md @@ -0,0 +1,23 @@ +# Changelog — 0.7.x + +All notable changes in the **0.7.x** release series are documented here. + +## [Unreleased] + +### Added +- English translations for all research notes, organized alongside their + hand-written Chinese sources of truth. Development notes now link to the + matching research language, and the pull-request workflow reports whether + English research translations are in sync when Chinese sources change. + + diff --git a/docs/dev_notes/en/0.5.x.md b/docs/dev_notes/en/0.5.x.md index 8a39c6f..a9bc004 100644 --- a/docs/dev_notes/en/0.5.x.md +++ b/docs/dev_notes/en/0.5.x.md @@ -10,7 +10,7 @@ When the tool starts up without printing its version, I sometimes cannot tell wh ### Built-in tools in common code agents -For a survey of the built-in tools of mainstream code agents, see [Built-in Tools in Common Code Agents](../../research/agent_tools.md) (Chinese). +For a survey of the built-in tools of mainstream code agents, see [Built-in Tools in Common Code Agents](../../research/en/agent_tools.md). ### Designing the built-in tools @@ -18,7 +18,7 @@ As the survey shows, beyond the core file/command tools that something like pi o #### The read tool -We also surveyed the [read tool](../../research/read_tool.md) (Chinese): apart from codex, every code agent has a dedicated read tool. pi and opencode are model-agnostic, and you can see it in their read tools — their limits are line counts and file sizes, with no token limit. The other, model-specific code agents all have token limits; presumably because they only support their own vendor's models, where tokens are easier to count? +We also surveyed the [read tool](../../research/en/read_tool.md): apart from codex, every code agent has a dedicated read tool. pi and opencode are model-agnostic, and you can see it in their read tools — their limits are line counts and file sizes, with no token limit. The other, model-specific code agents all have token limits; presumably because they only support their own vendor's models, where tokens are easier to count? Functionally, read and bash both seem able to satisfy the need to read a file. Even accounting for files that may exceed a size limit, bash can still read them fine. Take codex: in actual testing, whatever the size of the file, it always starts with a `sed -n '1,260p' foo.md`, and then, if it finds there is more, it may follow up with `wc -l foo.md && sed -n '1,260p' foo.md` — reading chunk by chunk, which looks a lot like the `limit` of a dedicated read tool. But I'm not sure whether, for any given model, reading files is better done with a dedicated tool or with `bash`. diff --git a/docs/dev_notes/en/0.6.x.md b/docs/dev_notes/en/0.6.x.md index 5df3bef..1e74495 100644 --- a/docs/dev_notes/en/0.6.x.md +++ b/docs/dev_notes/en/0.6.x.md @@ -35,7 +35,7 @@ print("Hello, World!") Running it outputs: `Hello, World!` ```` -Based on the revised conclusion of the [write tool research](../../research/write_tool.md), the write tool should implement: +Based on the revised conclusion of the [write tool research](../../research/en/write_tool.md), the write tool should implement: - Structured `{path, content}` input: the content lands on disk as data, never passing through shell expansion, heredoc delimiters, or quote escaping — eliminating the whole class of failures where a heredoc silently corrupts a file; - Create the file when it does not exist, overwrite it whole when it does, and state which of the two happened in the result; diff --git a/docs/dev_notes/zh-CN/0.5.x.md b/docs/dev_notes/zh-CN/0.5.x.md index a9c07d4..3563592 100644 --- a/docs/dev_notes/zh-CN/0.5.x.md +++ b/docs/dev_notes/zh-CN/0.5.x.md @@ -8,7 +8,7 @@ 工具启动时，如果不打印版本号，有时候自己分辨不出来是哪个版本。先开发一下这个功能。 ### 常见 code agent 内置工具 -主流 Code Agent 的内置工具调研见 [《常见 Code Agent 内置工具》](../../research/agent_tools.md)。 +主流 Code Agent 的内置工具调研见 [《常见 Code Agent 内置工具》](../../research/zh-CN/agent_tools.md)。 ### 内置工具设计 @@ -16,7 +16,7 @@ #### read 工具 -我们对 [read工具](../../research/read_tool.md) 也做了调研，除了 codex 外，其他 code agent 都有专门的 read 工具。pi 和 opencode 是模型无关的，可以看出他们的 read 工具，limit 除了行数，还有文件大小，并没有 token 限制。而其他专用的 code agent，都有 token 限制，大概是只支持自家模型，token 更方便计算？ +我们对 [read工具](../../research/zh-CN/read_tool.md) 也做了调研，除了 codex 外，其他 code agent 都有专门的 read 工具。pi 和 opencode 是模型无关的，可以看出他们的 read 工具，limit 除了行数，还有文件大小，并没有 token 限制。而其他专用的 code agent，都有 token 限制，大概是只支持自家模型，token 更方便计算？ 功能上讲，似乎 read 和 bash 都能实现读文件需求。即使考虑文件大小可能超出限制，bash 也能正常读取。例如 codex，实际测试中发现 codex 读文件的操作，不管读什么大小文件，他都会先来个 `sed -n '1,260p' foo.md`，之后如果发现还有内容，可能会 `wc -l foo.md && sed -n '1,260p' foo.md`,相当于一块一块读，和专用 read 工具的 limit 似乎差不多。但不太确定各个模型，在读文件的时候，是用专用工具好，还是用 `bash` 更好。 diff --git a/docs/dev_notes/zh-CN/0.6.x.md b/docs/dev_notes/zh-CN/0.6.x.md index 2e664c0..170367d 100644 --- a/docs/dev_notes/zh-CN/0.6.x.md +++ b/docs/dev_notes/zh-CN/0.6.x.md @@ -33,7 +33,7 @@ print("Hello, World!") Running it outputs: `Hello, World!` ```` -基于 [write 工具调研](../../research/write_tool.md)修正后的结论，write 工具要实现的功能： +基于 [write 工具调研](../../research/zh-CN/write_tool.md)修正后的结论，write 工具要实现的功能： - 结构化 `{path, content}` 入参：内容作为数据直接落盘，不经过 shell 展开、heredoc delimiter 和引号转义，消除 heredoc 静默写坏文件这一整类失败； - 文件不存在则创建，存在则整文件覆盖，并在结果中说明是哪一种； diff --git a/docs/research/README.md b/docs/research/README.md index d8b9da8..8cdf4f7 100644 --- a/docs/research/README.md +++ b/docs/research/README.md @@ -5,14 +5,25 @@ building the equivalent here. Unlike changelogs and dev notes, these are not tied to a release series — each file stands on its own, and the decisions they feed into are recorded in [`../dev_notes/`](../dev_notes/). -These notes are **written and maintained in Chinese**; each file says so at the -top. +Research notes are bilingual, split by language: -- [`agent_tools.md`](agent_tools.md) — the built-in tools of Pi, Claude Code, - Codex, OpenCode and Grok Build. -- [`edit_tool.md`](edit_tool.md) — how five agent projects express a localized - edit, and what contract a first Edit tool here should commit to. -- [`read_tool.md`](read_tool.md) — how five agent projects read files, and what - a dedicated read tool buys over plain bash. -- [`write_tool.md`](write_tool.md) — how five agent projects write whole files, - and why a structured write beats a heredoc. +- [`zh-CN/`](zh-CN/) — **hand-written Chinese source** (source of truth) +- [`en/`](en/) — **English, generated from the Chinese source** (do not edit by + hand) + +When a pull request adds or updates a Chinese source, its corresponding English +version also needs to be translated or refreshed. The agent preparing or +landing the pull request must remind the user and report whether the two +versions are in sync. + +- Built-in tools of Pi, Claude Code, Codex, OpenCode, and Grok Build: + [English](en/agent_tools.md) | [Chinese](zh-CN/agent_tools.md) +- How five agent projects express a localized edit, and what contract a first + Edit tool here should commit to: + [English](en/edit_tool.md) | [Chinese](zh-CN/edit_tool.md) +- How five agent projects read files, and what a dedicated read tool buys over + plain Bash: + [English](en/read_tool.md) | [Chinese](zh-CN/read_tool.md) +- How five agent projects write whole files, and why a structured write beats a + heredoc: + [English](en/write_tool.md) | [Chinese](zh-CN/write_tool.md) diff --git a/docs/research/en/agent_tools.md b/docs/research/en/agent_tools.md new file mode 100644 index 0000000..fdca0d3 --- /dev/null +++ b/docs/research/en/agent_tools.md @@ -0,0 +1,154 @@ +# Built-in Tools in Popular Code Agents + +> Generated from the Chinese source [`../zh-CN/agent_tools.md`](../zh-CN/agent_tools.md). Do not edit by hand. + +nanoPyCodeAgent currently has only a Bash tool, while mainstream code agents typically provide other built-in tools as well. This document surveys the tool designs of Pi, Claude Code, Codex, OpenCode, and Grok Build. + +## Pi + +Pi has a relatively small core toolset. The [Pi Quick Start](https://pi.dev/docs/latest/quickstart#first-session) introduces four core tools: + +- `read`: read files +- `write`: create or overwrite files +- `edit`: edit files +- `bash`: run shell commands + +It also offers three optional extension tools: `grep`, `find`, and `ls`. + +## Claude Code + +Claude Code provides a larger set of tools. The [Claude Code tools reference](https://code.claude.com/docs/en/tools-reference) introduces the following major tools. + +### File operations + +- `Read`: read file contents, including text, images, PDFs, and Jupyter Notebooks +- `Write`: create or completely overwrite files +- `Edit`: make localized changes to files through exact string replacement +- `NotebookEdit`: edit Jupyter Notebooks by cell + +### Search and code intelligence + +- `Glob`: find files by filename pattern +- `Grep`: search file contents by pattern, powered by ripgrep +- `LSP`: provide code intelligence through language servers, such as go to definition, find references, and type-error reporting + +### Command execution + +- `Bash`: execute shell commands in the environment +- `PowerShell`: execute PowerShell commands natively, primarily on Windows +- `Monitor`: run commands in the background and stream output line by line, for monitoring logs, polling state, and subscribing to WebSocket events + +### Web access + +- `WebFetch`: fetch content from a URL, convert it to Markdown, and use a small model to extract information according to a prompt +- `WebSearch`: search the web and return result titles and links + +### Subagents and orchestration + +- `Agent`: spawn a subagent with an independent context window to complete a task +- `SendMessage`: send a message to an Agent Team teammate, or resume a specific subagent by ID +- `Workflow`: run dynamic workflow scripts that orchestrate multiple subagents in the background and return an aggregated result +- `Skill`: run a Skill—a reusable prompt workflow—in the main conversation + +### Task management + +- `TaskCreate`, `TaskGet`, `TaskList`, `TaskUpdate`: create, retrieve, list, and update tasks +- `TaskOutput`: retrieve background-task output; deprecated in favor of reading output files with `Read` +- `TaskStop`: stop a background task by ID +- `TodoWrite`: manage the session task list; disabled by default and superseded by the `Task*` family + +### Planning and workspaces + +- `EnterPlanMode`, `ExitPlanMode`: enter plan mode to design an approach, submit the plan for approval, and exit +- `EnterWorktree`, `ExitWorktree`: create and enter an isolated Git worktree, then return to the original directory on exit + +### Scheduling + +- `CronCreate`, `CronDelete`, `CronList`: create, cancel, and list scheduled tasks within the current session +- `ScheduleWakeup`: schedule the next iteration of `/loop` at an adaptive cadence +- `RemoteTrigger`: create and manage Routines on claude.ai—that is, scheduled cloud agents + +### User interaction and output + +- `AskUserQuestion`: ask the user multiple-choice questions to clarify requirements +- `Artifact`: publish an HTML or Markdown file as an Artifact page on claude.ai +- `PushNotification`: send desktop or mobile push notifications +- `SendUserFile`: send reports, screenshots, and other files from the session directly to the user's device +- `ReportFindings`: report code-review findings as a structured list +- `ShareOnboardingGuide`: upload `ONBOARDING.md` and generate a link for sharing it with the team + +### MCP-related tools + +- `ListMcpResourcesTool`, `ReadMcpResourceTool`: list and read resources exposed by MCP servers +- `ToolSearch`: search for and load deferred tools on demand; requires Tool Search to be enabled +- `WaitForMcpServers`: wait for MCP servers that are still connecting in the background to become ready + +## Codex + +Unlike Pi and Claude Code, Codex has no dedicated `Read`, `Glob`, or `Grep` tools; it reads and searches by using `exec_command` to invoke commands such as `rg` and `sed`. In addition to the [official tools documentation](https://developers.openai.com/api/docs/guides/tools), this document also refers to the source code of the open-source [Codex](https://github.com/openai/codex) project. + +Tools commonly available in a standard local coding turn include: + +- `exec_command`: execute shell commands +- `write_stdin`: send additional input to, or poll, a long-running command +- `apply_patch`: modify files with structured patches +- `update_plan`: update the task plan +- `view_image`: view local images +- `web_search`, or `web.run` under the newer namespace: search the web + +Conditionally available tools also include: + +- Interaction: `request_user_input` +- MCP resources: `list_mcp_resources`, `list_mcp_resource_templates`, `read_mcp_resource` +- Permissions: `request_permissions` +- Multi-agent collaboration: `spawn_agent`, `send_message`, `followup_task`, `wait_agent`, `interrupt_agent`, `list_agents` +- Search-based tool discovery: `tool_search` +- Image generation: `image_gen.imagegen` +- Dynamic tools injected by MCP, Apps, and plugins + +## OpenCode + +[OpenCode](https://github.com/anomalyco/opencode) v1.18.3 includes the following built-in tools: + +### Files and execution + +- `bash`: execute shell commands +- `read`: read text or images, or list directory contents +- `write`: create or overwrite files +- `edit`: perform exact string replacements +- `apply_patch`: use patches to add, modify, or delete multiple files +- `glob`: find files by filename pattern +- `grep`: search file contents with regular expressions + +### Web access + +- `webfetch`: fetch content from a specified URL +- `websearch`: search the internet + +### User interaction + +- `question`: ask the user questions during execution + +### Skills and task management + +- `skill`: load a `SKILL.md` file +- `todowrite`: create and update task lists + +## Grok Build + +The recently open-sourced [Grok Build](https://github.com/xai-org/grok-build) includes the following built-in tools: + +- Files and execution: `run_terminal_command`, `read_file`, `search_replace`, `write`, `list_dir`, `grep`, `lsp` +- Background tasks and subagents: `spawn_subagent`, `get_command_or_subagent_output`, `wait_commands_or_subagents`, `kill_command_or_subagent`, `monitor` +- Task management: `todo_write`, `update_goal` +- Scheduled tasks: `scheduler_create`, `scheduler_delete`, `scheduler_list` +- Plan mode: `enter_plan_mode`, `exit_plan_mode`, `ask_user_question` +- Web: `web_search`, `web_fetch` +- MCP meta-tools: `search_tool`, `use_tool` +- Memory: `memory_search`, `memory_get` +- Media: `image_gen`, `image_to_video`, `reference_to_video` + +The following may also be added dynamically at runtime: + +- `image_edit` +- Server-side `x_search` diff --git a/docs/research/en/edit_tool.md b/docs/research/en/edit_tool.md new file mode 100644 index 0000000..fee4be3 --- /dev/null +++ b/docs/research/en/edit_tool.md @@ -0,0 +1,412 @@ +# A Design Survey of Edit Tools in Five Code Agents + +> Generated from the Chinese source [`../zh-CN/edit_tool.md`](../zh-CN/edit_tool.md). Do not edit by hand. + +## Research Scope and Methodology + +This document surveys the model-callable editing tools in five Code Agents under `references/` and answers three questions: + +1. Does the project provide an Edit tool? If not, what protocol does it use for localized edits? +2. Why design an Edit tool in addition to Bash and whole-file `write`, and what capabilities does it provide? +3. Given that nanoPyCodeAgent already has `read`, `write`, and `bash`, what Edit contract should it adopt next? + +Here, “Edit tool” broadly refers to any model-facing protocol that expresses “make a localized change based on the current file,” including exact string replacement, anchored editing, and patches. Ordinary GUI or server-side `fs/writeFile` RPCs do not count as model tools. This document is based on the source revisions currently checked out. `claude-code` is a third-party source mirror used by this repository, not an official Anthropic open-source repository. + +| Project | Current commit | Commit date | +| --- | --- | --- | +| `grok-build` | [`eb267fe`](https://github.com/xai-org/grok-build/tree/eb267feff13129e568df38fb6fdf0ceb65f735d6) | 2026-08-13 | +| `pi` | [`b1efcf7`](https://github.com/earendil-works/pi/tree/b1efcf7d7c5d7394fbb12ede0174e04d39ee7004) | 2026-08-14 | +| `claude-code` | [`a371abb`](https://github.com/yasasbanukaofficial/claude-code/tree/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367) | 2026-04-05 | +| `opencode` | [`e23586a`](https://github.com/anomalyco/opencode/tree/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3) | 2026-08-14 | +| `codex` | [`5bc8da6`](https://github.com/openai/codex/tree/5bc8da6d78fe32343dc51eaf73b96fd288ae0e87) | 2026-08-14 | + +Two methodological qualifications must be stated up front: + +- **The snapshots are not equally recent.** `grok-build`, `pi`, `opencode`, and `codex` were all checked out in mid-August 2026, whereas the `claude-code` mirror stops at 2026-04-05, roughly four months earlier than the other four. The conclusions about Claude Code apply only to this mirror and must not be treated as a description of the current Claude Code release. +- **Disambiguate identically named tools by package.** Some repositories contain multiple Edit implementations. This survey covers only the implementation that the model actually calls in the relevant Code Agent. Pi's CLI is provided by `@earendil-works/pi-coding-agent` (`bin: pi` → `cli.ts` → `main.ts` → `AgentSession` → `createAllToolDefinitions`), which leads to [`packages/coding-agent/src/core/tools/edit.ts`](https://github.com/earendil-works/pi/blob/b1efcf7d7c5d7394fbb12ede0174e04d39ee7004/packages/coding-agent/src/core/tools/edit.ts); that is the implementation described here. The generic agent kernel `@earendil-works/pi-agent-core` has another implementation at [`packages/agent/src/harness/tools/edit.ts`](https://github.com/earendil-works/pi/blob/b1efcf7d7c5d7394fbb12ede0174e04d39ee7004/packages/agent/src/harness/tools/edit.ts). It shares the same matching algorithm but replaces I/O with a fully `env`-abstracted layer. It is currently referenced only by `server/create-harness.ts` and is not reachable from the CLI. The two OpenCode generations differ far more substantially, so they are recorded separately as V1 and V2. + +## Conclusions + +All five projects provide structured localized-editing capabilities, but they have not converged on one kind of Edit tool: + +| Project | Model tool named Edit? | Primary editing protocol | Design orientation | +| --- | --- | --- | --- | +| Grok Build | No `edit` by default; the default is `search_replace`, while other presets can select `hashline_edit`, OpenCode `edit`, or `apply_patch` | Exact old/new strings; optional line-hash anchors or Patch | Select the protocol by model/preset; uniformly apply permissions, hooks, plan gates, and events in the outer tool layer | +| Pi | Yes, `edit` | Single-file `edits[]`, each containing `oldText`/`newText` | Submit multiple non-overlapping localized replacements at once; lightweight, embeddable, and backend-replaceable | +| Claude Code | Yes, `Edit`, plus `NotebookEdit` | One `old_string`/`new_string` pair, with optional `replace_all` | Strong Read-before-Edit, stale-state protection, history, permissions, LSP, and IDE integration | +| OpenCode | Both V1 and V2 have `edit` as well as `apply_patch` | One old/new pair plus `replaceAll`; V1 uses Patch instead for GPT models | V1 emphasizes fuzzy matching and ecosystem integration; V2 emphasizes exact matching, canonical paths, and in-process expected-bytes conditional writes | +| Codex | **No** string-based `edit` | Free-form `apply_patch` | Use one Patch protocol for multi-file add/update/delete/move operations, reducing overlapping tools | + +The true commonality among these “Edit tools” is therefore neither their name nor their schema. It is this: **the model expresses only the change and its preconditions; the runtime reads the current file, validates those preconditions, and preserves everything outside the requested change.** + +Compared with whole-file `write`, Edit's first benefit is sending fewer tokens and rewriting less unrelated content. Compared with Bash, its first benefit is enabling the Agent runtime to understand which resource is changing, which old content is the precondition, and what diff was actually produced. Edit is still not inherently a security boundary or a transaction: if Bash can write arbitrary files, path rules can be bypassed; if validation and persistence are not one strongly consistent storage transaction, external processes can still create races. + +## Why Design an Edit Tool? + +### 1. Replace Whole-File Retransmission with Incremental Expressions + +When changing three lines of code, `write(path, complete_content)` requires the model to resend the entire file. As files grow, so do the input-token cost, the probability of unintentionally changing an unexamined region, the retry cost after conflicts, and the UI cost of recomputing the diff. Edit sends only the target old and new text; Patch sends only hunks with context. + +This is why Claude Code, Pi, and OpenCode reserve `write` for file creation or complete rewrites and use Edit for ordinary modifications. Codex goes further: it exposes no whole-file Write at all, expressing new files through `*** Add File` patches as well. + +### 2. Turn the Old State Assumed by the Model into a Verifiable Precondition + +A localized edit does not mean “write at line 37.” It means “replace this old content only if it still exists and can be located uniquely.” If the user, a formatter, or another tool has already changed it, Edit should fail and make the model read again, rather than overwrite the current version as a whole-file Write would. + +The projects choose preconditions of different strengths: + +- String Edit: the old text exists and is unique by default; +- Patch: the context hunk can still be located; +- Hashline: the line number and content hash returned by Read still validate; +- Claude Code: additionally tracks session Read state and compares mtime/content; +- OpenCode V2: reads the original bytes after permission approval, then performs `writeIfUnchanged(expectedBytes)` at commit time. + +String Edit and Patch use “semantic preconditions”; Claude Code and OpenCode V2 additionally bind the operation to a read version. All are better than unconditional overwrite, but none automatically constitutes cross-writer CAS or a file-system transaction. + +### 3. Fail by Default Instead of Guessing a Location + +When old text occurs multiple times, replacing only the first occurrence by default is dangerous. Mainstream string Edit tools generally require a unique match; the model must either add a small amount of context or explicitly enable `replace_all`. Batch protocols also reject overlapping ranges so that applying one replacement cannot change the meaning of another. + +This fail-closed behavior also makes errors recoverable: the tool can explicitly report “not found,” “found N occurrences,” “file changed,” or “anchor is stale,” rather than handing the model only a fragment of shell stderr. + +### 4. Establish an Approvable, Observable File-Mutation Lifecycle + +Structured input tells the Agent runtime the target path before execution and lets it obtain the old/new content or a unified diff afterward, enabling an auditable lifecycle. Projects differ in when they request authorization; a fuller sequence looks like this: + +```text +Resolve target and canonical path +→ Authorize path/read access (before reading sensitive content) +→ Read current content and validate preconditions +→ Compute final diff / approve content / optional pre-write checkpoint +→ Revalidate before commit or perform an expected-bytes conditional write +→ Persist +→ Formatter / LSP / file event / history / undo / audit +→ Summary for the model and structured result for the Agent runtime +``` + +Arbitrary Bash can of course modify files too, but the Agent runtime cannot reliably infer the final write set, pre-execution diff, and ownership of each hunk from a dynamic shell program. + +### 5. Adapt to the Model's Training Distribution Instead of Pursuing One Universal Protocol + +OpenCode exposes `apply_patch` to modern non-OSS GPT models and `edit`/`write` to other models. Grok Build likewise uses presets to switch among search/replace, Hashline, OpenCode Edit, and Codex Patch. This demonstrates that the tool schema is a model-adaptation surface; it cannot be chosen solely according to implementation convenience in the Agent runtime. + +String replacement is simplest for a model; Patch is better suited to multi-file changes in one call; Hashline most strongly encodes “the old content is still the version I saw,” but requires Read output and Edit input to adopt a new shared anchor language. No interface is optimal for every model. + +## Implementations by Project + +### 1. Grok Build + +#### Does It Have Edit? + +It has localized-editing capabilities, but the default tool is named `search_replace`. The default Grok Build toolset registers `read_file` + `search_replace`, with the workspace version additionally registering `write`; the Codex preset uses `apply_patch` instead. The file tools can also be switched as a group to `hashline_read`/`hashline_edit`/`hashline_grep`, and the registry explicitly rejects mixing standard and Hashline file tools. The [preset configuration](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-agent/src/config.rs#L170-L347) and [mutual-exclusion validation](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/registry/types.rs#L878-L908) embody this design of replacing tools as a protocol-level set. + +#### `search_replace` + +Its input is: + +```text +file_path: string +old_string: string +new_string: string +replace_all: boolean = false +``` + +It performs an exact string search first. Matches must be unique by default; only `replace_all=true` replaces all non-overlapping matches. `old_string == new_string` fails. If the file contains any CRLF, matching moves into a logical LF space and the entire output is then converted back to CRLF. Ordinary CRLF files therefore retain their style, while mixed line endings are normalized. The [schema and description](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/grok_build/search_replace/mod.rs#L53-L139) and the [matching, newline, and persistence code](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/grok_build/search_replace/mod.rs#L522-L738) define the actual semantics. + +After exact matching fails, the default behavior is only to return diagnostics about nearby lines, possible user modifications, and Unicode typography. Unicode-confusable normalization fallback is enabled only by an optional configuration. This fallback maps smart quotes, dashes, special spaces, and similar characters before matching, maps the resulting range back to the original UTF-8 bytes, and fails closed on partial expansions, overlaps, and ambiguity. The [normalization fallback](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/grok_build/search_replace/helpers.rs#L136-L263) is deliberately more conservative than “fuzzily find something close and edit it.” + +An empty `old_string` doubles as file creation/whole-file writing. One compatibility detail is easy to misread: `empty_old_string_does_not_override` still defaults to false, so it can overwrite an existing non-empty file, while the success message still calls the file “created.” Only after the guard is explicitly enabled do the intended semantics narrow to create-or-fill-empty. Even that guard is not strictly fail-closed: the creation branch swallows read errors and treats an unreadable target as nonexistent. The [versioned parameter](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/grok_build/search_replace/mod.rs#L94-L139) and [empty-string branch](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/grok_build/search_replace/mod.rs#L294-L405) explain this historical compatibility tradeoff. + +A successful result contains modification context and line numbers; the `patch` field in the result structure is currently always `None`. The tool emits `FileWritten` with previous/new content for the hunk tracker, rewind, and UI. The outer session then uniformly applies the plan-mode gate, pre-tool hooks, and edit permissions. The core search/replace implementation is therefore relatively thin, with most control-plane logic in the orchestration layer. However, the repository's `FileOperationLockManager` is not currently connected to this execution path. + +The boundaries are as follows: it has no session-level prior-Read revision, mtime, expected-bytes CAS, or shared path lock; other tools and external processes can still race with the read-compute-write sequence. CRLF restoration normalizes line endings across the entire output, so a file with mixed line endings can acquire unrelated changes. UTF-8 decoding remains lossy by default, creating a risk of replacement characters when editing invalid UTF-8 text. Permissions match lexical paths, while an existing target is canonicalized before I/O; if a symlink points into another permission domain, safety must come from the outer OS sandbox rather than the tool rules alone. + +#### `hashline_edit` + +Hashline Read returns a `LINE:HASH[:CONTEXT_HASH]` anchor for each line. Edit accepts multiple operations in one call: + +- `replace(anchor, end_anchor?, content)`: replace/delete one line or a range; +- `insert_after(anchor, content)`: accepts an ordinary anchor, BOF `0:`, or `EOF`; +- `write(content)`: replace the whole file without an anchor. + +See [`types.rs`](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/grok_build_hashline/edit/types.rs#L7-L116) for the schema. Every anchor is validated against the same pre-edit snapshot; if any anchor is stale, ambiguous, not found, or overlaps another range, the entire logical batch fails before disk is written. Once validation passes, all operations are applied once from bottom to top, and localized excerpts with fresh anchors are returned for subsequent edits. See [batch validation and application](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/grok_build_hashline/edit/apply.rs#L143-L305). + +It addresses ordinary line numbers drifting after insertions/deletions and long old text wasting tokens. A stale anchor also triggers a search for relocated candidates nearby, but the tool returns only a fresh anchor so the model can retry rather than editing on its own initiative. The cost is that Read and Edit must be coupled as one protocol. The default three-character hash, which ignores some whitespace differences, is not a cryptographic revision either. The model may also accidentally copy anchor prefixes into the content, so the implementation explicitly detects and reports this error. A logical batch's “validate everything, then write once” behavior does not imply atomic visibility, crash durability, or cross-writer CAS. Unlike the default `search_replace`, the current Hashline Edit does not emit a `FileWritten` event, and ordinary replace/insert operations normalize CRLF to LF. + +#### Other Compatibility Presets + +Grok Build also retains two implementations ported from other Agent protocols rather than merely renaming its tools: + +- `Codex:apply_patch` accepts a JSON `patch` field whose Patch body supports multi-file Add/Delete/Update/Move operations. It first parses and computes all changes in memory, then performs I/O sequentially. A hunk failure can stop the operation before any write, but a failure during persistence does not roll back earlier writes; Add and Move destinations can also overwrite existing files. The [tool and three-phase execution](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/codex/apply_patch/tool.rs#L25-L105) and [commit phase](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/codex/apply_patch/tool.rs#L313-L477) show that it is “full preflight + non-transactional commit.” +- `OpenCode:edit` uses `filePath`/`oldString`/`newString`/`replaceAll`, but this port performs only exact matching and does not include OpenCode V1's current multi-layer fuzzy matching. An empty old string can create a file or overwrite an empty file; ordinary edits require uniqueness by default and emit `FileWritten`. See the [schema and execution](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/opencode/edit/mod.rs#L45-L88) and [replacement path](https://github.com/xai-org/grok-build/blob/eb267feff13129e568df38fb6fdf0ceb65f735d6/crates/codegen/xai-grok-tools/src/implementations/opencode/edit/mod.rs#L333-L472). + +These two compatibility implementations likewise have no prior-Read revision, shared file lock, or atomic replacement. Their main value is matching the tool-training distribution of particular models, not providing stronger file-system guarantees. + +### 2. Pi + +#### Does It Have Edit? + +Yes. The default toolset contains `read`, `write`, `edit`, and `bash`. The current schema has evolved from an earlier single old/new pair to: + +```text +path: string +edits: Array<{ + oldText: string + newText: string +}> +``` + +`edits` must be non-empty. Every `oldText` is matched against the same original file rather than against the result of the previous item. The tool prompt asks the model to put multiple distant changes in one call and merge adjacent or overlapping changes into a single block. See the [schema and model guidance](https://github.com/earendil-works/pi/blob/b1efcf7d7c5d7394fbb12ede0174e04d39ee7004/packages/coding-agent/src/core/tools/edit.ts#L34-L64). + +Pi once supported both single-edit and multi-edit schemas, but models repeatedly produced invalid calls by mixing the two shapes. It ultimately retained only `edits[]`. When restoring an old session, `prepareArguments` still folds top-level `oldText`/`newText` into an array and tolerates some models double-encoding the array as a JSON string. This [compatibility conversion](https://github.com/earendil-works/pi/blob/b1efcf7d7c5d7394fbb12ede0174e04d39ee7004/packages/coding-agent/src/core/tools/edit.ts#L105-L135) shows that schema simplicity is itself a reliability feature. + +#### Capabilities and Algorithm + +All replacements in one call are validated first, then applied in reverse order with only one write: + +- Empty `oldText` is forbidden, so Edit does not create files; +- Each `oldText` must match exactly once; zero matches and multiple matches are rejected. The batch fails if its final content is unchanged, but there is no per-item no-op validation; +- Overlapping ranges among edits are rejected; +- After exact matching fails, it falls back to NFKC, per-line trailing-whitespace, smart-quote, dash, and special-space normalization; +- It preserves the UTF-8 BOM and restores ordinary LF/CRLF files to the detected line-ending style; +- On a fuzzy match, only the lines actually touched are rewritten; untouched lines are copied back from the original content to avoid normalizing the entire file; +- It returns a display diff, a standard unified patch, and the first changed line. When the arguments are complete, the TUI asynchronously previews the diff before execution. + +See [`edit-diff.ts`](https://github.com/earendil-works/pi/blob/b1efcf7d7c5d7394fbb12ede0174e04d39ee7004/packages/coding-agent/src/core/tools/edit-diff.ts#L295-L374) for the core algorithm and [`edit.ts`](https://github.com/earendil-works/pi/blob/b1efcf7d7c5d7394fbb12ede0174e04d39ee7004/packages/coding-agent/src/core/tools/edit.ts#L298-L367) for the execution path. Notably, the model-facing description says “exact,” while the implementation has fuzzy fallbacks. This improves the success rate when copied text differs slightly, but also means that a human cannot infer the actual matching boundary from the prompt alone. + +Pi abstracts `readFile`, `writeFile`, and `access` as injectable operations, allowing the same protocol to target backends such as SSH and VMs. All `write`/`edit` operations also share an in-process mutation queue keyed by the `realpath` of existing files: operations on the same real file are serialized, while operations on different files can still run concurrently. When an operation is aborted, the queue is released only after the underlying I/O has settled, preventing a write already reported as canceled from interleaving with the next write. The [operations](https://github.com/earendil-works/pi/blob/b1efcf7d7c5d7394fbb12ede0174e04d39ee7004/packages/coding-agent/src/core/tools/edit.ts#L81-L103) and [mutation queue](https://github.com/earendil-works/pi/blob/b1efcf7d7c5d7394fbb12ede0174e04d39ee7004/packages/coding-agent/src/core/tools/file-mutation-queue.ts#L1-L60) reflect Pi's lightweight, embeddable orientation. + +The boundaries are as follows: the queue coordinates only participating Pi tools, not Bash or external editors. There is no mtime/hash/expected-bytes check and no temp + fsync + rename sequence. Paths are not restricted to the cwd by default. The fuzzy fallback normalizes entire touched lines, so it can still change trailing whitespace or Unicode forms on those lines outside `oldText`. Mixed line endings are normalized to the first detected style, and bare CR becomes LF. UTF-8 decoding is lossy, with no NUL/binary guard. Injectable I/O helps remote backends, but the surrounding tool infrastructure has not caught up: both the mutation queue's `realpath` call and the TUI's pre-execution diff preview directly use local `node:fs`, bypassing `operations`. The abstraction is therefore incomplete. + +### 3. Claude Code + +#### Does It Have Edit? + +Yes. It has `Edit` and separately provides `NotebookEdit` for `.ipynb` files. The ordinary Edit schema is: + +```text +file_path: string # the schema text requires an absolute path; the runtime also accepts relative paths and ~ +old_string: string +new_string: string +replace_all: boolean = false +``` + +Claude Code's `Write` prompt directly instructs the model to prefer Edit when modifying an existing file because Edit “only sends the diff”; Write is reserved for creating a file or rewriting it completely. The [Write tool prompt](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileWriteTool/prompt.ts#L10-L17) gives the product-level rationale for Edit: compress model output while avoiding retransmission of unchanged regions. + +`old_string` must be unique by default. If it occurs more than once, the model is told to add context or explicitly use `replace_all`. The tool prompt explicitly warns against copying Read's line-number prefixes into the string. Although the schema text requires an absolute path, the runtime's `expandPath` also accepts relative paths and `~`, so the described contract and implementation boundary do not align completely. The [schema](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileEditTool/types.ts#L5-L34), [prompt](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileEditTool/prompt.ts#L4-L27), and [path expansion](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/utils/path.ts#L8-L84) demonstrate detailed constraints tailored to Claude's output habits, while also exposing interface drift. + +#### Enforced Read-before-Edit and Stale-State Protection + +Claude Code does not merely recommend Read in the prompt; it enforces it at runtime. An ordinary existing file must have session Read state, and a partial view automatically injected by the system is insufficient. The tool rejects the edit if the current mtime is later than the read time. The implementation also attempts to accept a pure touch when the cache state has no recorded offset/limit and the content is unchanged, but FileRead records `offset=1` by default, so an ordinary full read generally cannot reach this fallback either. Immediately before the real write, it synchronously reads the current metadata/content again and deliberately inserts no asynchronous yield between that check and persistence, narrowing the race window within a single JS event loop. Its [input validation](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileEditTool/FileEditTool.ts#L137-L361) and [pre-commit revalidation](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileEditTool/FileEditTool.ts#L425-L491) constitute the heaviest session-level protection among the five projects. + +An empty `old_string` is a restricted creation path: it is allowed if the target does not exist or the existing file is blank, and rejected for non-empty files, preventing implicit whole-file overwrite through Edit. FileEdit's core matcher tries exact matching first; on failure, it normalizes only straight/curly quotes and restores the new string to the file's original typography style. Earlier in the call chain, before the input reaches the tool, a fixed set of desanitization rules restores a small number of tokens hidden or abbreviated by the API. It still does not perform arbitrary indentation or anchor-similarity fallback like OpenCode V1. See [quote matching](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileEditTool/utils.ts#L18-L135) and [input restoration](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileEditTool/utils.ts#L526-L657). When deleting, `new_string=""` also has a convenience behavior that removes an immediately following newline. However, this special branch is defective when combined with `replace_all`: for example, deleting every `x` in `x\nx` may leave the final occurrence in place even though the result text still claims that all occurrences were replaced. See the [deletion implementation](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileEditTool/utils.ts#L206-L228). + +#### Control Plane and Output + +Edit reuses Claude Code's complete mutation lifecycle: path/deny/UNC and symlink permission checks, team-memory secret validation, optional history, UTF-8/UTF-16LE and dominant-line-ending restoration, structured patches, VS Code/LSP notification when available, diagnostic cleanup, and telemetry. After success it updates the Read state so the next consecutive edit is based on the new content. It does not automatically run a formatter. See [post-write integration and results](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileEditTool/FileEditTool.ts#L490-L593). The low-level write prefers a temporary file in the same directory, flushes it, preserves the mode, and then renames it. On failure, it silently falls back to direct overwrite, so this is only best-effort atomic replacement rather than a strong guarantee. See the [write implementation](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/utils/file.ts#L369-L477). + +It is still not cross-writer CAS: a window remains between the mtime/content check and the final write. The synchronous write path provides in-process “no await” ordering, not a kernel transaction. Enforcing a prior Read also adds tool-call cost, and a path policy applied only to Edit can still be bypassed through Bash. Another contract boundary worth noting is that API message normalization silently removes per-line trailing whitespace from non-Markdown `new_string` values, so the bytes written need not exactly match the model arguments. See [input normalization](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileEditTool/utils.ts#L526-L657). + +`NotebookEdit` deserves separate treatment because a Notebook is not an ordinary JSON text-replacement problem. It performs replace, insert, and delete operations using either real cell IDs or synthetic `cell-N` IDs from FileRead. When changing a code cell, it clears outputs and the execution count and maintains nbformat 4.5+ cell IDs. It also requires Read-before-Edit and fresh mtime. The [validation](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/NotebookEditTool/NotebookEditTool.ts#L176-L294) and [cell operations](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/NotebookEditTool/NotebookEditTool.ts#L295-L453) demonstrate why structured files benefit from domain-specific Edit tools rather than forced reuse of text replacement. However, it checks for staleness only during initial validation, without the ordinary Edit's final revalidation after waiting for permissions/history, and it re-stringifies the entire notebook. Editing one cell can therefore still overwrite concurrent changes and produce a large formatting diff. Its [prompt still describes](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/NotebookEditTool/prompt.ts#L1-L8) `cell_number`, which is no longer present in the schema—another example of drift between prose and implementation. + +### 4. OpenCode + +The OpenCode repository contains two generations of the implementation: V1/legacy under `packages/opencode` and the new Location-scoped V2 architecture under `packages/core`. They share names but not semantics and must not be conflated. V1 chooses between `apply_patch` and `edit` + `write` according to the model; the current [V2 built-ins](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/core/src/tool/builtins.ts#L20-L42) register all three, then filter them by permission. + +#### Why Have Both `edit` and `apply_patch`? + +The V1 registry selects by model: modern non-OSS, non-GPT-4 GPT models see `apply_patch`, while other models see `edit` + `write`; it does not give one model all three overlapping interfaces. See [tool selection](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/opencode/src/tool/registry.ts#L286-L297). This directly demonstrates that the two interfaces primarily serve different model capabilities: old/new is easier to generate, while Patch is better suited to multi-file edits and GPT's training distribution. + +#### V1 `edit`: Permissive Matching and Deep Integration + +Its inputs are `filePath`, `oldString`, `newString`, and optional `replaceAll`. A per-file semaphore covers read-compute-write. For an existing file, an empty old string is forbidden; for a nonexistent target, an empty old string creates the file. It preserves the BOM and the file's original LF/CRLF style, first generates a diff to request `edit` permission, then persists, runs the formatter, recomputes the final diff from the post-formatter result, publishes file/watcher events, triggers LSP, and returns diagnostics to the model. See the [execution lifecycle](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/opencode/src/tool/edit.ts#L35-L215). + +The matching strategy tries, in order: exact, per-line trim, first/last-line block anchors with Levenshtein similarity, whitespace normalization, flexible indentation, escape normalization, boundary trim, context-aware matching, and multiple occurrences. After the replacer selects one concrete actual string, the outer layer requires that string to be unique by default; only `replaceAll` changes every occurrence. However, block-anchor fuzzy matching selects the highest-scoring candidate among several, retains the first on a tie, and does not require the original fuzzy candidates to be globally unique. A separate guard rejects a matched span that is excessively large relative to `oldString`, preventing a short fuzzy input from consuming a large block. See the [replacer implementation](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/opencode/src/tool/edit.ts#L217-L736). + +This strategy is highly tolerant of small model transcription errors, but also carries the greatest risk: the prompt's claimed “exact edit” is far removed from the actual runtime boundary, and a block anchor with 0.65 similarity can accept a middle region that has already changed substantially. A permission diff can provide a backstop when a human approves the operation; permissive fallbacks deserve more caution under automatic approval. V1's [prompt also claims that Read is required before modification](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/opencode/src/tool/edit.txt#L1-L8), but the corresponding FileTime/read-state check has been removed from the current implementation—another counterexample of drift between the prompt contract and runtime. + +V1 `apply_patch` uses a `{ patchText }` JSON wrapper around a Codex-like Patch language and supports multi-file add/delete/update/move in one call. Matching falls back progressively through exact, `trimEnd`, `trim`, and Unicode-punctuation normalization. All changes are computed and approved together first, after which files are persisted one by one, formatted, and sent through events and LSP. Add and Move destinations overwrite existing files. The commit does not roll back and has no expected-bytes CAS. Because formatting occurs after the approved diff, the final actual changes can exceed the approval preview. Although a Move destination enters metadata and can separately trigger external-directory authorization, it is not added to the `edit` permission's path patterns, leaving the approval scope for internal destinations incomplete as well. The [tool execution](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/opencode/src/tool/apply_patch.ts#L18-L278) and [Patch matching](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/opencode/src/patch/index.ts#L430-L506) show this combination of “Patch expressiveness + deep V1 integration.” + +#### V2 `edit`: Exact + Expected Bytes + +V2 keeps the same single old/new + `replaceAll` shape but does not use V1's similarity-based fuzzy matching. The source lists V1 fuzzy matching, formatting, watcher integration, snapshot/undo, and LSP as future TODOs. It: + +- Canonicalizes paths through `LocationMutation`, preventing relative paths and workspace symlinks from escaping the workspace; an explicitly external absolute path requests `external_directory` first; +- Forbids an empty old string and no-op changes, requiring uniqueness or explicit replace-all; +- Strips the UTF-8 BOM before matching, converts `oldString` and `newString` together to the file's detected line ending, and restores the BOM when writing; +- Returns replacements, a unified patch, additions/deletions, and a bounded old/new diff preview for the model; +- Reads source bytes only after permission approval, then compares current bytes with expected bytes inside an in-process canonical-path lock at commit time, reporting stale state if they differ. + +To be precise, V2's exact matching is not raw-byte exact. It is exact **after removing the BOM and converting old/new to the file's line endings**: `detectLineEnding` classifies a file as CRLF if `\r\n` occurs even once, and `convertToLineEnding` first normalizes input to LF before converting it back to CRLF when needed. See the [normalization helpers](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/core/src/tool/edit.ts#L42-L53) and [pre-match conversion](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/core/src/tool/edit.ts#L161-L165). This conversion has a deterministic direction and an enumerable scope, and it rewrites only the matched span while leaving unmatched regions as the original bytes. It therefore solves both “Read output stripped `\r`, so the model cannot provide the original CRLF text” and “do not casually normalize the whole file.” Among the five projects, this is the most restrained treatment of the problem. The cost is a coarse classification: one occurrence of `\r\n` causes the tool to treat the entire file as CRLF when converting `oldString` and `newString`, so pure-LF fragments in a mixed-line-ending file cannot match; the tool must fail closed and make the model read again. + +See [`V2 edit.ts`](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/core/src/tool/edit.ts#L22-L223) for execution and [`file-mutation.ts`](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/core/src/file-mutation.ts#L54-L171) for the in-process keyed lock and expected-bytes conditional write. The comparison and ordinary write are contiguous only with respect to OpenCode mutations that cooperate through that lock; an external process or Bash can still race between them. This reflects a strategy of first clarifying explainable exact matching and in-process concurrency semantics, then gradually adding UX features. + +V2 `apply_patch` supports add/update/delete in one call. It first parses and resolves every target, authorizes them in a batch, then performs preflight reads. Add uses create-only `wx`; Update uses expected bytes. Commits remain sequential, so a later failure does not roll back earlier successful items. Delete has no expected-content CAS, and Move is not yet supported. See [V2 Patch](https://github.com/anomalyco/opencode/blob/e23586af2623f1bc2e8e6965d2d7acf7bd03d5c3/packages/core/src/tool/apply-patch.ts#L17-L218). “Batch preflight” must therefore not be described as a “multi-file transaction.” + +### 5. Codex + +#### No String-Based Edit + +Codex does not register `edit`/`edit_file`. [When an environment exists and the model metadata declares `apply_patch_tool_type`](https://github.com/openai/codex/blob/5bc8da6d78fe32343dc51eaf73b96fd288ae0e87/codex-rs/core/src/tools/spec_plan.rs#L1090-L1110), it registers free-form `apply_patch`, currently its only dedicated file-mutation tool. The schema is not JSON old/new; it is text constrained by a Lark grammar: + +```text +*** Begin Patch +*** Add File: path ++new content +*** Update File: old-path +*** Move to: new-path +@@ optional context +-old ++new +*** Delete File: path +*** End Patch +``` + +The tool definition explicitly says that it is “for editing files” and must not be wrapped in JSON, and it marks the free-form custom tool as suitable for GPT-5. The provider-side Lark grammar supports multi-hunk add/delete/update/move operations. See the [tool spec](https://github.com/openai/codex/blob/5bc8da6d78fe32343dc51eaf73b96fd288ae0e87/codex-rs/core/src/tools/handlers/apply_patch_spec.rs#L5-L27) and [grammar](https://github.com/openai/codex/blob/5bc8da6d78fe32343dc51eaf73b96fd288ae0e87/codex-rs/core/src/tools/handlers/apply_patch.lark#L1-L19). The old JSON/function calling surface was later [removed](https://github.com/openai/codex/commit/e783341b705721728a8fa422416c10c3a09c7716), preventing both models and tests from having to handle two equivalent protocols. Patch is executed by a built-in Rust parser and does not depend on a system-installed `patch(1)`. + +This is an important counterexample: needing a dedicated localized-editing protocol does not necessarily mean needing a string-based Edit tool. Add File covers creation, Update suits ordinary modifications, and multiple files share one parser and permission entry point. The cost is that the model must generate the Patch language reliably. + +#### Capabilities and Boundaries + +Before execution, Codex parses the entire Patch, reads old content for Update/Delete, computes new content and a unified diff for every Update hunk, and rejects multiple operations that use the same source path. Only then does it classify safety, request approval, and execute in the sandbox based on the complete change set. The [pre-validation](https://github.com/openai/codex/blob/5bc8da6d78fe32343dc51eaf73b96fd288ae0e87/codex-rs/apply-patch/src/invocation.rs#L200-L280) and [handler](https://github.com/openai/codex/blob/5bc8da6d78fe32343dc51eaf73b96fd288ae0e87/codex-rs/core/src/tools/handlers/apply_patch.rs#L359-L447) embody “understand the changes first, then decide whether they may execute.” The same rule does not fully cover conflicts among Move destinations or between destinations and other sources, so sequential execution can still partially fail. + +Context location tries, in order: exact, ignoring trailing whitespace, trimming both sides, and normalizing Unicode punctuation/special spaces. An EOF anchor searches from the end of the file first. See [`seek_sequence`](https://github.com/openai/codex/blob/5bc8da6d78fe32343dc51eaf73b96fd288ae0e87/codex-rs/apply-patch/src/seek_sequence.rs#L1-L115). Even if the model invokes the tool through a restricted `apply_patch <<'PATCH' ...` shell form, Codex recognizes it and routes it through the same Patch approval/sandbox chain rather than treating it as arbitrary Bash. + +Important Patch limitations include: + +- Actual file operations commit in file-action order; failures do not roll back, and the return value carries the deltas already committed; +- Add can overwrite an existing target rather than being create-only; +- Move writes the destination first and then deletes the source, so failure to delete the source can leave both copies; +- There is no expected-bytes CAS between validation of an Update and actual persistence. After approval, the runtime rereads the file and recomputes the Patch. If an external change does not invalidate the hunk, it can continue applying the Patch to the new file, and the final result can differ from the approval preview; +- The final operation is a whole-file text rewrite and does not express binary deltas, `chmod`, or similar metadata. + +See [`apply-patch/src/lib.rs`](https://github.com/openai/codex/blob/5bc8da6d78fe32343dc51eaf73b96fd288ae0e87/codex-rs/apply-patch/src/lib.rs#L438-L663) for sequential commit and Move behavior. The outer tool layer also provides writable-root safety classification, platform sandboxing, hooks, streaming diff events, and remote file systems, but these are Codex control-plane capabilities rather than atomicity inherent in the Patch grammar. + +## Capability Comparison + +| Capability | Grok `search_replace` / Hashline | Pi | Claude Code | OpenCode V1 / V2 | Codex | +| --- | --- | --- | --- | --- | --- | +| Ordinary interface | One old/new pair; Hashline can batch | Batched old/new in `edits[]` | One old/new pair | One old/new pair | Multi-file Patch | +| Unique match by default | Yes | Required for every item | Yes | V1 block fuzzy can select the best candidate; V2 yes | A hunk takes the first context candidate | +| Replace all | `replace_all` | No direct switch | `replace_all` | `replaceAll` | Multiple hunks/contexts | +| Multiple locations/files in one call | Search can change identical literals; Hashline supports multiple operations; Patch supports multiple files | Multiple operations in one file | `replace_all` can change identical literals; no multiple distinct operations/files | Edit can change identical literals; Patch supports multiple files | Multiple files | +| Creation | Empty old in `search_replace`; Hashline write | No | Restricted creation with empty old | Empty old in V1; no V2 Edit creation; Patch Add | Patch Add | +| Fuzzy fallback | Optional Unicode for search; Hash anchor recovery | NFKC/trailing whitespace/Unicode | Quote normalization + fixed desanitization | Multi-layer fuzzy in V1; V2 exact only after BOM/line-ending conversion | Hunk trim/Unicode | +| CRLF/BOM | Search preserves LF or normalizes to CRLF; Hashline converts to LF; no special BOM protection | BOM and ordinary LF/CRLF; mixed line endings are normalized | UTF-8/UTF-16LE and dominant LF/CRLF | V1 Edit preserves them; V2 strips BOM for matching and converts old/new to file line endings; Patch may mix line endings | Converts to LF by default; an experimental feature can preserve line endings; no special BOM handling | +| Stale-state protection | Old/anchor precondition | Old precondition | Prior Read + mtime/content | V1 Edit: old; V2 Edit/Patch Update: expected bytes; Add: `wx`; Delete: none | Hunk precondition, no CAS | +| In-process serialization for the same file | No shared path lock | Realpath queue | No `await` in the final section, but no path lock | V1: Edit semaphore only; V2: canonical lock per target | No path lock/CAS | +| Diff/UI | Context; Search has `FileWritten` events | Preview + diff + patch | Structured patch + IDE | V1/V2 diff | Diff event + A/M/D summary | +| Formatter/LSP/history | Search connects to hunk/rewind; no in-tool formatter | Extension hooks; no built-in LSP | No formatter; optional history/LSP | Present in V1; TODO in V2 | General events/hooks; no in-tool formatter | + +## Design Tradeoffs + +### Exact vs. Fuzzy + +Exact matching is explainable and testable: the tool changes precisely the precondition submitted by the model. Its drawback is that small differences in Read formatting, CRLF, trailing whitespace, smart quotes, and similar details cause retries. + +Fuzzy matching can improve first-attempt success, but expands the range the tool is effectively authorized to change. Grok's Unicode offset map maps a match under limited normalization back to the original bytes and rejects candidates that cannot be mapped back completely. Pi instead replaces within normalized text and restores only untouched blocks of original lines, so touched lines can also acquire NFKC, punctuation, or trailing-whitespace changes. OpenCode V1's block similarity is more aggressive and requires additional span guards and an approval diff. At the other end of the spectrum, OpenCode V2 deliberately limits itself to two deterministic, enumerable conversions—BOM and line endings—with no similarity judgment, so the matched range still corresponds exactly to the literal submitted by the model. For a small Agent without a pre-execution approval UI, exact-first/fail-closed is more appropriate. The allowed conversions must be a short, enumerable list that can be written into the tool description, not a chain of heuristics. + +### Single Replacement, Batched Replacements, and Patch + +- Single old/new: the smallest schema and a high model success rate; multiple calls repeat I/O and cannot atomically validate a group of changes as one logical unit. +- Same-file `edits[]`: one read, full validation, one write; the contract must define whether all matches use the original content or progressive results and must handle overlaps. +- Patch: the greatest multi-file expressiveness, but also the highest costs for parsing, error recovery, partial commits, and model adaptation. + +Pi's evolution also shows that one schema should not expose both top-level old/new and `edits[]` as equivalent shapes; models will mix them. Choose one stable public interface and confine compatibility to a runtime input-migration layer. + +### “Validate Then Write” vs. True Atomicity + +Five concepts must be distinguished: + +1. **Complete logical validation**: first confirm in memory that every replacement can apply; +2. **Single-file atomic visibility**: concurrent observers see either the complete old version or the complete new version, usually provided by temp + rename on the same file system; +3. **Crash durability**: after success is reported, the file and directory entry remain reliable even after power loss, usually requiring file/directory `fsync` as well; +4. **Cross-writer concurrent CAS**: commit only if the file still has a specified revision/byte sequence; +5. **Multi-file transaction**: either every operation succeeds or all are rolled back. + +Pi/Hashline batched Edit provides item 1. Claude's low-level write makes a best effort to use temp + rename but can silently fall back, so it only attempts item 2 and does not fully provide items 3 or 4. OpenCode V2's `writeIfUnchanged` is a conditional write protected by an in-process lock, not cross-writer CAS. None of the multi-file Patch implementations above—Grok compatibility presets, OpenCode, or Codex—provides item 5. Reports and tool errors must not present one of these layers as another. + +## Design Recommendations for nanoPyCodeAgent + +### Current Constraints + +nanoPyCodeAgent currently exposes only `read`, `write`, and `bash`, with Claude Sonnet 4.6 as the default model. A single-threaded loop executes tool calls in the order returned by the model. The [system prompt and toolset](../../../src/nanopycodeagent/agent.py#L37-L50) and [sequential dispatch](../../../src/nanopycodeagent/agent.py#L169-L179) mean there is currently no in-process concurrent lost-update problem. + +The existing `write` is explicitly last-writer-wins: it calls `Path.write_bytes` directly, does not require a prior Read, does not check a revision, provides no atomic replacement, and follows symlinks that point to regular files. See [`write_tool.py`](../../../src/nanopycodeagent/write_tool.py#L89-L153). `read` loads at most 10 MB for an entire file and strips CRLF's `\r` in its display. The [size limit](../../../src/nanopycodeagent/read_tool.py#L13-L23) and [line-ending view](../../../src/nanopycodeagent/read_tool.py#L117-L129) mean that multi-line old text copied by the model usually contains only LF. A new Edit must be compatible with these actual semantics; it cannot suddenly claim a security boundary that Bash can bypass and Write does not provide. + +### Conclusion: Add a Thin, Exact, Single-Replacement Edit Now + +`write` already handles new files and whole-file rewrites, but ordinary code modifications still require retransmitting the entire file or falling back to Bash. Edit now offers direct benefits in token usage, preservation of untouched content, and old-text preconditions, making it worthwhile as a fourth built-in tool. + +The first version should use: + +```text +Edit( + path: string, + old_text: string, + new_text: string, + replace_all: boolean = false, +) +``` + +`path` keeps the interface consistent with this project's Read/Write tools; snake_case matches Python and the current schema style. Although Claude Code uses `old_string`, a clear tool description is enough to teach the default Claude model. There is no need to fragment this project's field naming merely to copy one product. + +The exact contract for the first version should be: + +1. Edit only existing regular UTF-8 files; missing targets, directories, FIFOs, and devices fail. Creation and whole-file overwrite remain the responsibility of `write`. +2. `old_text` must not be empty, and `old_text == new_text` fails. `new_text=""` is valid and means exact deletion, with no hidden behavior such as “also remove the next newline.” +3. Perform an exact literal match first (item 6 permits one, and only one, subsequent CRLF retry). Zero matches fail. More than one match with `replace_all=false` fails and reports the match count. `replace_all=true` replaces every non-overlapping match and returns the actual count. +4. Do not use regex, trim, indentation, similarity, or Unicode fuzzy matching. The only permitted input conversions are the BOM and line-ending conversions in items 5 and 6. They are deterministic and enumerable and must be stated truthfully in the tool description; unlike OpenCode V1 and Pi, the tool must not claim exact matching to the model while implementing a wider matching boundary. Errors should prompt the model to Read and expand or correct the context. The first version has no pre-execution diff approval, so an extra retry is preferable to silently expanding the authorized mutation range. +5. Decode UTF-8 strictly and reject invalid UTF-8 and files containing NUL. `read` uses the replacement character for invalid bytes to make them viewable; if Edit round-tripped the file that way, it would permanently corrupt the original bytes. Strip a UTF-8 BOM before matching and restore it unchanged when writing. This cannot be omitted: `read` currently does not strip U+FEFF, so the model sees an invisible character at the start of the first line; failing to strip it would make `old_text` targeting that line mysteriously fail. Pi and OpenCode V2 handle it this way as well. +6. To match Read's line-ending view, follow OpenCode V2's approach with **one deterministic conversion**, not several parallel candidates. First perform raw exact matching with `old_text` unchanged. Only if the match count is zero, the file contains `\r\n`, and `old_text` contains `\n` but no `\r`, retry exactly once with the LF→CRLF form of `old_text`; in that case, also convert `new_text` from LF to CRLF before writing. Do not take the union of both passes: uniqueness checks and `replace_all` counts occur only within the pass that actually matched, avoiding ambiguity from candidate deduplication and overlapping spans across encodings. (Taking a union would create exactly this ambiguity: for content `"a\r\na\na"` with `old_text="a\na"`, the two encodings respectively match `[3,6)` and `[0,4)`, which overlap.) If `old_text` explicitly contains `\r`, perform raw exact matching only, treating the model as expressing the original bytes. Both passes rewrite only matched spans, always preserving unmatched regions as the original bytes. Because the second pass runs only after the first has zero matches, one call can match only one line-ending style. In a mixed-line-ending file, fragments using the other style will not match in that call—an intentional fail-closed behavior, and the error must state that the CRLF retry was attempted. This limitation affects only multi-line `old_text`: single-line `old_text` without `\n` is independent of line endings, can match across both styles in the first pass, and does not trigger the retry. +7. Reuse the 10 MB `MAX_READ_BYTES` size limit because the implementation requires a whole-file read-compute-write cycle. Errors should explicitly recommend Bash or a purpose-built script for larger files. +8. Align path expansion, regular-file checks, symlink behavior, and error formatting with `read`/`write`. The first version still writes back directly and explicitly provides no mtime/CAS/atomic replacement. + +### Capabilities Not Recommended for the First Version + +- **Do not enforce a prior Read.** The current session has no read-revision registry, and Read may expose only a window. A unique `old_text` match already serves as a localized precondition. Recording mtime by force would add state without eliminating TOCTOU. +- **Do not add batched `edits[]`.** The current loop executes multiple tool calls sequentially. Start with a single replacement for model reliability and a small implementation. Upgrade like Pi only if either tool round-trip latency or the need for multiple edits within one file becomes a measured bottleneck, and retain only one public schema when doing so. +- **Do not add `apply_patch`.** A parser, multi-file partial commits, move/delete operations, and permission target sets would substantially enlarge the nano agent's core. The default Claude model is also more familiar with old/new Edit. +- **Do not add localized versions of a formatter, LSP, history, approval, or workspace sandbox.** These belong to the overall mutation/exec control plane. Path protection on Edit alone can be bypassed through Bash. +- **Do not add a per-file queue yet.** Tool execution is currently single-threaded. When parallel tool calls are actually introduced, key a shared mutation queue by canonical/real path and place the entire Read-Compute-Write sequence inside it. + +### Model Prompt, Results, and Terminal Display + +The tool description should tell the model directly: + +- Prefer Edit for ordinary localized changes; use Write for new files or complete rewrites; use Bash for bulk mechanical transformations; +- `old_text` must match file content literally and be unique by default. Two to four lines are usually enough. Do not include Read's line-number prefixes. The only exceptions are line endings and the BOM: Read displays LF, and the tool converts CRLF files itself, so the model need not and should not construct `\r` manually; +- On a mismatch, Read the latest content first rather than repeatedly submitting the same call; +- Use `replace_all` only when every identical literal is intentionally being changed. + +Keep a successful result concise, for example: + +```text +[edited src/app.py: replaced 1 occurrence, lines 42-44] +``` + +In the terminal, show `[edit] path` and a collapsed small old/new diff. Do not place the complete large strings in the tool result again, because they are already present in the assistant's tool input. If generating a unified diff, give it a hard limit similar to Bash/Read and mark truncation explicitly. + +Errors should contain recovery information: not-found should recommend reading again; duplicate should return the match count and recommend adding context or explicitly using `replace_all`; invalid UTF-8, oversize files, and non-regular files should each explain why they cannot be round-tripped safely instead of collapsing into “edit failed.” + +### Recommended Test Matrix + +The first version should cover at least: + +- Unique replacement, deletion, Unicode, empty files, and no-op changes; +- Not-found, duplicate matches, `replace_all`, and returned counts; +- LF, CRLF, no final newline, and preservation of untouched content in files with mixed line endings; +- Multi-line LF-form `old_text` matching a CRLF file and writing back as CRLF; one call matching only one style in a mixed-line-ending file while failing closed on the other; single-line `old_text` matching across both styles without triggering a retry; `old_text` that explicitly contains `\r` using raw exact matching only; correct `replace_all` counts in each pass; +- Stripping the UTF-8 BOM before matching and restoring it when writing, including successful `old_text` matching against the first line; rejection of invalid UTF-8 and NUL; +- Missing targets, directories, FIFOs/devices, and files above the size limit; +- `~`, relative/absolute paths, and symlink behavior consistent with Write; +- Collapsed terminal previews and correct `is_error` values in tool results; +- Multiple Edit calls in the same model response observing the result of each previous call in order. + +### Conditions for Further Evolution + +When the product introduces parallel calls, an approval UI, undo, remote file systems, or a restricted mode, establish a shared mutation core for Write/Edit/Patch: canonical paths, uniform permissions, expected digests, a per-file queue, best-effort atomic replacement, mode/BOM/line-ending policies, before/after events, and bounded diffs. Bash must be brought under the same OS/container/VM file-system boundary at the same time, or the control plane remains bypassable. + +The final recommendation can be summarized as follows: + +> What nanoPyCodeAgent needs now is a thin Edit that “fails on inexact matches and describes its semantics honestly,” not the full control plane of a mature Agent compressed into one Python file. First use a unique old-text precondition to address token costs and accidental overwrites in localized modifications. Upgrade the mutation core only when the architecture genuinely develops concurrency, approval, and remote-access requirements, rather than simulating safety in advance. diff --git a/docs/research/en/read_tool.md b/docs/research/en/read_tool.md new file mode 100644 index 0000000..2f30c57 --- /dev/null +++ b/docs/research/en/read_tool.md @@ -0,0 +1,337 @@ +# Comparing File-Reading Tools Across Five Agent Projects + +> Generated from the Chinese source [`../zh-CN/read_tool.md`](../zh-CN/read_tool.md). Do not edit by hand. + +## Scope + +This document counts only **built-in tools that the model can invoke directly**. It does not conflate ordinary internal functions, test helpers, or RPCs intended for GUI clients with Agent tools. The conclusions are based on the following revisions checked out during the research. Source snapshots are not committed to this repository, so this document does not cite their local paths: + +| Project | Current commit | Commit date | +| --- | --- | --- | +| `grok-build` | `c68e39f` | 2026-07-16 | +| `pi` | `5e336cfa` | 2026-07-15 | +| `claude-code` | `a371abb` | 2026-04-05 | +| `opencode` | `fab213312` | 2026-07-18 | +| `codex` | `1bbdb32789` | 2026-07-15 | + +## Conclusions + +| Project | Model-facing tool name | Core role | Text pagination | Non-text files | +| --- | --- | --- | --- | --- | +| Grok | `read_file` | Full-featured semantic file reader | `offset` and `limit`; up to 1,000 lines and 25,000 tokens by default | Images, PDFs, and PPTX; `.ipynb` read as text | +| Pi | `read` | Compact, embeddable reader with a replaceable backend | `offset` and `limit`; up to 2,000 lines or 50 KiB | Images | +| Claude Code | `Read` | Reader deeply integrated with permissions, context, and the skill system | `offset` and `limit`, with dual protection based on file size and token count | Images, PDFs, and Jupyter Notebooks | +| OpenCode | `read` | Location-scoped structured reader shared by files, images, and directories | `offset` and `limit`; up to 2,000 lines or 50 KiB for large files or explicit pagination | JPEG, PNG, GIF, and WebP; directory listings | +| Codex | **No dedicated text-reading tool**; normally uses `exec_command`, or `shell_command` in legacy configurations; uses `view_image` for images | Reuses the shell ecosystem, placing safety, session, and output controls in the command-execution layer | Implemented by commands such as `sed`, `rg`, and `head` | `view_image` supports images; no dedicated branch for PDFs or Notebooks | + +The most important difference is that Grok, Pi, Claude Code, and OpenCode model “reading a file” as an explicit, read-only operation with structured parameters. Codex does not add another abstraction for text; instead, it lets the model invoke programs such as `rg`, `sed`, and `cat` through a general-purpose command-execution tool. + +## Invocation Interfaces and Model-Visible Return Formats + +Two layers must be distinguished here: a tool's internal implementation may return a struct or union type, while the model may ultimately see only a formatted segment of text. “Return format” in the table below refers specifically to the result actually visible in the model context, not the implementation's internal type. + +| Project | Model invocation interface | Model-visible text or content | Return-format assessment | +| --- | --- | --- | --- | +| Grok | `read_file({ target_file, offset?, limit?, pages?, format? })` | Text comes from the internal `FileContent.content`; the first visible line and every line whose number is divisible by ten receive a `line number→` anchor. The unformatted source is also stored in `raw_output`, but it is not sent to the model as an ordinary text result | Formatted text; images and PDF pages use multimodal content blocks | +| Pi | `read({ path, offset?, limit? })` | The body remains unchanged and line numbers are not added automatically; when truncation or explicit pagination occurs, a note with the current range and the next `offset` is appended | Primarily plain text; images use a separate image content block | +| Claude Code | `Read({ file_path, offset?, limit?, pages? })` | Before the internal structured result enters the model context, it is converted to `tool_result.content`; every line receives a `line number→` or `line number` prefix, and a `` may be appended | Formatted text; images, PDFs, and Notebooks use dedicated content blocks | +| Classic OpenCode implementation | `read({ filePath, offset?, limit? })` | Returns an `output: string` wrapped in ``, ``, and ``, with an `N: ` prefix added to every line of the body | XML-style formatted text | +| OpenCode V2 | `read({ path, offset?, limit? })` | Returns a structured union of `FileSystem.Content`, `TextPage`, or `ListPage`; the body is in the `content` field without line numbers, while pagination is expressed through `offset`, `truncated`, and `next` | Structured JSON; images additionally use a file content block | +| Codex | No dedicated text-reading tool; normally invokes `exec_command({ cmd, ... })` | The tool result is an object containing fields such as `output`, `exit_code`, and `session_id`; whether `output` is the original text, text with line numbers, or some other format depends entirely on the command executed | Structured command result; the format of the file body is not fixed | + +Therefore, if “plain text” means the file body without added line numbers, tags, or a JSON wrapper, Pi comes closest. Claude Code, Grok, and classic OpenCode actively add positional or structural markers. OpenCode V2 preserves the original body but places it in a structured result. Codex delegates the format choice to the specific shell command. + +## Code Agent File-Reading Tool Implementations + +### 1. Grok: `read_file` + +#### Tool entry point and parameters + +- The model-facing tool name is `read_file`. +- Its parameters are: + - `target_file`: a workspace-relative or absolute path; + - `offset`: the starting line; accepts positive values, `0`, and negative values, with negative values locating lines backward from the end of the file; + - `limit`: the number of lines to read; + - `pages`: a PDF page number or range; + - `format`: PDF output as either `image` or `text`. + +#### Text reading + +- The tool first reads the file as bytes, then converts it to text with fault-tolerant UTF-8 decoding; recognized binary files are rejected. +- By default, a single result contains at most 1,000 lines. Even if the caller supplies a larger `limit`, the configured ceiling takes precedence. A further 25,000-token ceiling protects the output. +- Exact windowed reads are supported. When a limit is exceeded, the tool suggests narrowing `offset`/`limit` or switching to a search tool. If the window contains only one extremely long line, it also suggests extracting characters with shell tools such as `jq` or `cut`. +- The actual text format does not prefix every line. The current implementation gives the first visible line and every line whose number is divisible by ten a `line number→` anchor, reducing the token cost of line numbers themselves. +- Text results can be streamed in chunks of approximately 4 KiB, with splits made at character boundaries, but the file itself is still loaded into memory in full before chunking. +- `SKILL.md` is an intentional exception: the tool ignores the supplied `offset`/`limit` and bypasses the normal line and token limits, ensuring that skill instructions are loaded in full. + +#### Images and documents + +- Images are detected by magic bytes rather than by trusting the extension alone. +- Images are converted to PNG, JPEG, or WebP accepted by the model endpoint, and automatically resized and compressed according to dimensions, total pixel count, and payload size. By default, the longest edge is at most 2,000 px, total area is approximately 1.05 Mpx or less, and the base64 payload is at most 768 KiB. +- PDFs are rendered as page-by-page images by default, or their text can be extracted with `format="text"`. When no page is specified, at most 10 pages are read automatically; with an explicit selection, at most 20 pages can be read at once. The file-size limit is 50 MiB and the processing timeout is 60 seconds. +- For PPTX files, the tool unpacks the archive and extracts the DrawingML text and speaker notes from each slide. Compressed input is likewise limited to 50 MiB, with a 60-second processing timeout. +- The tool description claims support for Jupyter Notebooks, but the current `read_file` implementation has no Notebook-specific branch; `.ipynb` files actually follow the ordinary JSON text path. + +#### Additional integration with the Agent framework + +- Depending on configuration, the tool can refuse to read files matched by `.gitignore`, reducing the risk of inadvertently reading secrets, build artifacts, and similar content. +- It attempts to correct Unicode filenames and produces friendlier suggestions for nonexistent paths. +- It can return structured errors such as “file does not exist,” “is a directory,” and “permission denied,” instead of only a mixed block of stderr. +- When reading a file, it can append matching Cursor rules so that path-specific rules enter the context alongside the source code. + +### 2. Pi: `read` + +#### Tool entry point and parameters + +- The model-facing name is `read`. +- Its parameter set is small: `path`, a 1-based `offset`, and `limit`. +- Paths may be relative or absolute. The path layer also handles `~` and `@` prefixes, Unicode spaces, narrow no-break spaces in macOS screenshot names, NFD filenames, and curly-quote variants. + +#### Text reading + +- By default, the tool reads the complete file into a `Buffer`, then selects the requested line range and truncates the output. +- Returned content is limited to 2,000 lines or 50 KiB, whichever limit is reached first, while avoiding partial lines whenever possible. +- A truncated result explicitly reports the current line range, the file's total number of lines, and the `offset` to use next. +- If the first line alone exceeds 50 KiB, the tool does not silently truncate that line. Instead, it returns an executable `sed | head -c` suggestion. +- `limit` first selects the window requested by the user, after which the shared line/byte limit is applied. An out-of-range `offset` produces an error that includes the total line count. +- Unlike Claude Code and Grok, Pi does not automatically add line numbers to ordinary text returned to the model. This saves tokens, at the cost of requiring the caller to establish its own positional anchors when referring to a particular line. + +#### Images + +- JPEG, PNG, GIF, WebP, and BMP are supported and detected from file signatures; a non-image disguised with an image extension does not enter the image branch. +- Formats such as BMP that the model API does not accept directly are converted to PNG first. +- By default, images are resized to no more than 2000×2000 and the base64 payload is kept to approximately 4.5 MiB or less. The tool tries PNG and JPEG, multiple JPEG quality levels, and progressively smaller dimensions. +- Images are attached to the model as genuine image content blocks rather than embedding base64 text in ordinary output. If the model does not support visual input, the result explicitly states that the image was omitted. + +#### Additional integration with the Agent framework + +- `ReadOperations` abstracts `readFile`, `access`, and MIME detection into replaceable operations. The same tool can therefore connect to SSH or another remote filesystem without changing the model protocol. +- It supports `AbortSignal`, allowing the session to cancel long-running operations. +- The TUI applies syntax highlighting by extension and collapses `SKILL.md`, `AGENTS.md`, `CLAUDE.md`, and Pi documentation into compact displays. This mainly improves the human interface and does not change the file body sent to the model. + +### 3. Claude Code: `Read` + +#### Tool entry point and parameters + +- The tool is exposed to the model as `Read`. +- Its parameters are the absolute path `file_path`, a 1-based `offset`, `limit`, and the PDF-specific `pages`. +- The tool declares itself read-only and safe for concurrent execution, and includes the path in permission matching. + +#### Text reading + +- Returned content uses a format similar to `cat -n`, with a line number on every line. The default uses a right-aligned `line number→content` format; an experimental flag can instead use a compact tab-separated format. +- Ordinary files smaller than 10 MiB take a one-shot fast path. Large files, pipes, and special files use streaming scans. The streaming path retains only lines within the requested window and merely counts lines outside it, so reading a small window from a 100 GiB file does not cause memory use to grow with the file size. +- Both paths remove a UTF-8 BOM, normalize CRLF to LF, and support session cancellation. +- The default output limit is 25,000 tokens. When `limit` is omitted, a 256 KiB total-file-size gate also rejects large files early; when `limit` is explicit, a small window can be read from a large file, while the final result remains subject to the token limit. +- The tool prompt says “2,000 lines maximum by default,” but the current core `call` implementation does not set a default `limit` of 2,000. Instead, it primarily relies on the 256 KiB and 25,000-token limits. A 2,000-line constant is explicitly used by the automatic attachment-ingestion path. This is a discrepancy between the implementation and the prompt in the checked-out source. +- When the same range of the same file is read again and its modification time has not changed, subsequent reads return only a short “file unchanged” message. This reuses content from earlier in the conversation, reducing context usage and prompt-cache cost. + +#### Images, PDFs, and Notebooks + +- Images are read as visual content and automatically resized, downsampled, or compressed according to the model's token budget. Original and displayed dimensions are returned as well, allowing the model to map coordinates back to the original image. +- PDFs can be sent directly as document blocks, or pages can be extracted and rendered as images according to `pages`. A page range is mandatory for PDFs longer than 10 pages, and at most 20 pages can be read at once. For files larger than 3 MiB, or when the model does not natively support PDFs, the tool attempts page extraction. +- `.ipynb` has a genuine Notebook branch: cells are parsed, with code, Markdown, output, and visualizations preserved, then mapped to a structured tool result instead of treating the Notebook as JSON text. +- Common binary extensions are rejected, except for natively supported types such as images, PDFs, and SVGs. + +#### Additional integration with the Agent framework + +- Before reading, the tool checks `Read(...)` allow/deny rules. Actual I/O for UNC paths is deferred until after user authorization, avoiding unapproved network authentication attempts. +- It explicitly blocks `/dev/zero`, `/dev/random`, stdin/tty, standard file-descriptor aliases, and other device files that could produce infinite output or block indefinitely. +- When a path does not exist, the tool tries macOS screenshot-space variants, candidate paths under the current working directory, and similar filenames, then offers a `Did you mean ...` suggestion. +- Reading a path can trigger skill-directory discovery and conditional skill activation. Automatically managed memory files also receive freshness information. +- Before text enters the model context, a malicious-code-analysis reminder is appended. Read events are also sent through hooks, listeners, and telemetry. + +### 4. OpenCode: `read` + +#### Tool entry point and layers + +- The model-facing name is `read`; this section discusses the current V2 Location-scoped built-in tool. +- Its parameters are `path`, a positive 1-based integer `offset`, and a positive integer `limit`; the schema itself caps `limit` at 2,000. `path` can refer to a file or directory, so the current implementation no longer exposes a separate `list` tool. +- The implementation is deliberately split into two layers: `read.ts` handles the model schema, Location paths, permissions, file/directory dispatch, image normalization, and error projection; `read-filesystem.ts` handles independently testable I/O, pagination, format detection, and structured results. +- The return value is not a preformatted string, but a union of three structured results: `FileSystem.Content` for ordinary files, `TextPage` for paginated text, and `ListPage` for directories. Only images additionally produce a native media content block. + +#### Text reading + +- If a file is no larger than 50 KiB and the caller supplies neither `offset` nor `limit`, the tool reads the whole file in 64 KiB chunks and returns UTF-8 `FileSystem.Content`. Small files are therefore returned in full by default, and the 2,000-line limit does not apply to this fast path. +- Files larger than 50 KiB, or any request with an explicit range, automatically enter a streaming pagination path. It starts at line 1 by default, returns at most 2,000 lines and at most 50 KiB of body text, stops when either limit is reached, and reports `truncated` plus the `next` line number for a subsequent call. +- Pagination does not load an entire large file into memory. It scans from the beginning in 64 KiB chunks and retains only the current window. A high `offset` still requires a sequential scan of the preceding content, however; no byte index or seek is used. +- Each line retains at most 2,000 characters; if it is longer, an explicit truncation marker is appended. CRLF is normalized during line-by-line processing. UTF-8 decoding is fatal, so invalid bytes cause the read to fail instead of being replaced with `�` and allowing processing to continue. +- The text body does not receive automatic line numbers. A small-file result is `FileSystem.Content` JSON containing `content`, `encoding`, and `mime`; only paginated results additionally contain `offset`, `truncated`, and `next`. Positional anchors depend on pagination metadata rather than line-by-line prefixes as in Claude Code. +- Binary detection combines extensions, PDF magic bytes, NUL bytes, and the proportion of non-printing control characters. PDFs, Office documents, archives, executables, and similar files are rejected. `.ipynb` has no dedicated branch and is read as ordinary UTF-8 JSON text. + +#### Images and directories + +- Images are identified as JPEG, PNG, GIF, or WebP from their content signatures, which take precedence over extensions; a valid image disguised as `.bin` can therefore still be read. Original media ingestion is limited to 20 MiB, with checks against both the `stat` size and the number of bytes actually streamed, preventing file growth during a read from bypassing the limit. +- Images are limited by default to 2000×2000 and a 5 MiB base64 payload. When a limit is exceeded, Photon/WASM with Lanczos3 progressively downsizes the image and tries PNG followed by several JPEG quality levels. These thresholds and automatic resizing can be configured through `attachments.image`. If the resizer itself is unavailable, the current policy retains the original image instead of failing an ordinary read. +- An image result contains a “read succeeded” message and a native `file` content block; base64 is not disguised as ordinary text. General text-output truncation does not remove the media block. +- A directory read resolves the real path of every direct child and retains only regular files and directories that remain within the directory. It therefore excludes broken links, special files, and symbolic links pointing outside the directory. Entries are sorted with directories first and by name within each kind; at most 2,000 entries are returned, and `next` supports continued pagination. + +#### Paths, permissions, and the output lifecycle + +- Relative paths must stay within the current Location, and absolute paths within that Location are also accepted. Relative `..` traversal outside the Location, or escape through symbolic links, is rejected. An explicit absolute path outside the Location first requires `external_directory` authorization, followed by `read` authorization for the target resource. +- During pagination, the tool itself constrains the text window to 2,000 lines/50 KiB. During finalization, the Tool Registry adds a general model-output safeguard with the same defaults of 2,000 lines/50 KiB. If the complete structured result still exceeds the limit, the full content is saved to a managed `tool-output` file, while the model receives a head-and-tail preview and its path. The default retention period is 7 days. +- Not all expected errors are exposed verbatim. Binary files, media-ingestion limits, image-decoding failures, and dimension errors retain their specific messages, while path errors, ordinary filesystem errors, permission errors, invalid UTF-8, out-of-range offsets, and similar conditions are currently projected to `Unable to read `. This reduces leakage of internal errors but makes recovery guidance for some failures less specific than Pi's or Claude Code's. + +### 5. Codex: No Dedicated Text `read_file` + +#### Tools the model actually uses + +The current Codex tool registry has no ordinary file reader. The `read_file` references in the source are merely remote tool names in MCP server examples and tests, not model tools built into Codex. When Unified Exec is supported, the default tool plan exposes: + +- `exec_command`: execute shell commands; +- `write_stdin`: continue interacting with a command that is still running; +- `view_image`: read and view local images. + +When Unified Exec is not supported, or a legacy model configuration is used, Codex exposes `shell_command` instead. + +Codex therefore normally reads text by asking `exec_command` to run commands such as: + +```text +rg -n '^' path/to/file +sed -n '100,180p' path/to/file +head -n 200 path/to/file +``` + +#### What `exec_command` adds beyond Bash + +- It provides structured fields for `workdir`, the shell, whether to use a login shell, whether to allocate a PTY, how long to wait before returning, the output token budget, and more. +- It waits 10 seconds by default and has a model-facing output budget of 10,000 tokens; internal collection also has a 1 MiB limit. Overlong output is truncated in the middle and its original token count is reported. Retaining the beginning and end is usually more useful than a simple `head` for seeing a command's conclusion or errors. +- Long-running commands do not block the entire Agent: the tool returns a `session_id`, which the model can use with `write_stdin` to poll, send input, or continue collecting output. +- Commands run inside filesystem and network sandboxes. Out-of-scope access goes through structured approval, and controlled command-prefix rules can be recorded. +- Local and attached remote environments are supported, and the tool returns structured metadata including `exit_code`, elapsed time, session ID, and truncation state. + +#### Implementation details of output budgets, middle truncation, and session continuation + +The “10,000-token budget, middle truncation, and session continuation” described above are distributed across three layers. Together they form a combination of byte-level truncation in the collection layer, token-level truncation in the rendering layer, and session continuation through a process repository. + +**Collection layer: byte-level head/tail buffering (`HeadTailBuffer`)** + +- Each process has a single 1 MiB output buffer, split equally between a head and a tail of 512 KiB each. +- Writes fill the head first, after which all output goes into the tail. The tail is a rolling queue: once over budget, it discards bytes from the front and retains only the final 512 KiB. Discarded bytes only increment an `omitted_bytes` counter. +- A read assembles `head + "... N bytes omitted ..." + tail`. +- This layer provides a hard memory ceiling. Background tasks continuously write PTY output into the buffer, but even a process with infinite output, such as `yes`, still uses a constant 1 MiB of memory. + +**Rendering layer: token-level middle truncation (`truncate_middle_with_token_budget`)** + +- Token counting does not use an actual tokenizer. It simply rounds up “number of bytes ÷ 4” (`APPROX_BYTES_PER_TOKEN = 4`). +- After converting the token budget to a byte budget, it divides that budget equally between the beginning and end, cuts at character boundaries, and replaces the middle with an `…N tokens truncated…` marker. +- The final text also receives a `Warning: truncated output (original token count: N)` header. During assembly, the implementation checks for an existing byte-level marker to avoid inserting a duplicate. The two truncation markers are independent and may both appear. +- The 10,000-token budget is calculated as `min(caller-provided max_output_tokens or the default 10000, truncation_policy ceiling for the model tier)`: the model may explicitly request more output in the parameters, but the policy still caps it. This budget applies only to the rendering layer; the collection layer remains fixed at 1 MiB. + +**Session layer: process repository and deadline-bounded collection** + +- `exec_command` first assigns a random process_id from 1000 to 100000. After spawning a PTY, it immediately stores the process in a repository (HashMap). Storing it before waiting for output is intentional: if the user interrupts the current turn, releasing the last reference must not accidentally kill the background process. +- The core `collect_output_until_deadline` loop repeatedly drains the shared buffer, waits for notifications of new output, and detects process exit. After exit, it keeps a 50 ms grace period to capture residual output. This continues until the deadline; the wait duration is clamped to 250 ms–30 s and defaults to 10 s. +- If the process has exited by the deadline, the tool returns its exit_code and releases the process_id. If it is still running, the response includes a session ID and the process remains in the repository. +- `write_stdin` retrieves the same set of output handles by process_id, writes to stdin, and then enters the same deadline-bounded collection loop. Output produced between tool calls is not lost: the background task continuously writes into the shared buffer, which the next call drains immediately. An empty `write_stdin` has polling semantics, with a broader wait window of 5 s–300 s. +- The repository retains up to 64 concurrent sessions. When full, it prunes by LRU while protecting the 8 most recently used sessions and preferring to remove processes that have already exited. + +The implementation's complexity is distributed very unevenly. Head/tail buffering and token truncation together take fewer than 350 lines, have no unusual dependencies, and are easy to port. Session continuation is the genuinely heavy part: PTYs, background collection tasks, the process repository, and exit-race handling account for most of the code. + +#### `view_image` + +`view_image` is Codex's only model tool dedicated to the contents of local files. It: + +- checks whether the model supports image input; +- reads an image inside the selected environment and filesystem sandbox; +- uses the `high` detail level by default and can request `original` when the model supports it; +- converts the file to a data URL and sends it to the model as a genuine image content item. + +#### Do not confuse it with `fs/readFile` + +The Codex app server also provides an `fs/readFile` JSON-RPC method. It accepts an absolute path and returns the raw bytes as base64. This method is intended for app-server clients, is not part of the model tool plan, and has no `offset`, `limit`, line numbers, text-token controls, or document parsing. It therefore should not be counted as a built-in file-reading tool for the Codex Agent. + + +In summary: + +- **Claude Code `Read`**: has the richest semantics, particularly for streaming scans of large files, deduplicating repeated reads, permissions, and Notebook/PDF integration. The tradeoff is a complex implementation tightly coupled to Claude Code's internal state. +- **Grok `read_file`**: broadly covers images, PDFs, and PPTX; has clear output anchors and token protection; and explicitly handles rules, skills, and extremely long lines. Ordinary text is still loaded into memory in full before processing. +- **OpenCode `read`**: V2 has clear layering and structured-result boundaries; text and directories both support resumable pagination; large files are not loaded in full; and Location permissions and general oversized-output retention are unified. It supports fewer document types, text has no line numbers, and some recoverable errors are collapsed into generic messages. +- **Pi `read`**: has the most direct implementation, predictable 50 KiB/2,000-line limits, useful image handling, and a practical remote-backend abstraction. It supports few document types and does not automatically add line numbers to text. +- **Codex `exec_command` + `view_image`**: derives its text capabilities from Unix/PowerShell composition, making it flexible and able to reuse the surrounding ecosystem. Safety, sessions, approvals, and output budgets are handled uniformly by the execution layer, but it lacks a dedicated read protocol for text ranges, total line counts, PDFs, and Notebook semantics. + +## Comparing Bash and Read Tools + +To investigate whether a dedicated file-reading tool is more effective than Bash, this research also examined tests, design documents, system prompts, changelogs, and source comments in all five projects. It specifically searched `eval`, `bench`, and `benchmark` directories, as well as tests containing both Read and `cat`/Bash. No genuine end-to-end A/B evaluation was found: there was no test in which the same model completed the same task once with only a dedicated Read tool and once with only Bash, followed by measurements of task success rate, token usage, latency, or error rate. + +The existing evidence supports much narrower claims: + +- A dedicated Read tool can provide bounded pagination, deterministic continuation positions, structured permissions, native multimodal content, and recoverable errors more consistently. +- Some framework-level optimizations apply only to dedicated Read tools and do reduce repeated context or prompt-cache cost. +- These behaviors are covered by extensive unit and regression tests. +- They do not, however, directly establish that “an Agent using Read has a higher overall task success rate than an Agent using Bash.” + +Text in a system prompt that says to “prefer Read” demonstrates only the product's design intent and is not, by itself, evidence of effectiveness. + +The following sections therefore compare the advantages and disadvantages of dedicated file-reading tools. + +### Advantages of a Dedicated File-Reading Tool + +#### 1. Less invocation ambiguity and shell risk + +A dedicated tool represents the path, starting line, line count, and PDF page number as typed fields. The model does not have to assemble quoting, escape spaces, handle `$()`, or account for cross-platform command differences. Nor does it incidentally gain the full expressive power of a shell merely to “read one file.” The permission system can also identify the operation unambiguously as read-only. + +#### 2. Proactive context-window protection + +By default, `cat` dumps all content to stdout. If the file is huge, the Agent can only deal afterward with a result the platform has already truncated. At the read-protocol layer, a dedicated tool knows the total number of lines, the current window, and the next offset, and limits its result by line count, byte count, or token budget. This brings three direct benefits: + +- A single tool result does not consume a disproportionate amount of context. +- It avoids oversized API requests, out-of-memory failures, and long blocking operations. +- Even after truncation, it gives a deterministic position from which reading can continue, rather than an unidentified fragment. + +Although Codex has no dedicated text reader, `exec_command` still provides a default 10,000-token budget, middle truncation, and original-size information at the command-output layer. + +#### 3. Stable positional anchors for the model + +Claude Code numbers every line, while Grok anchors the first line of a window and every tenth line. OpenCode leaves the body unchanged but identifies window boundaries with structured `offset`/`next` fields. The model can refer to code locations, plan subsequent range reads, and map observations to edit operations. Ordinary `cat` provides no line numbers. Bash can add them with `nl` or `cat -n`, but the model must remember to choose a consistent format every time. + +#### 4. Native multimodal and document understanding + +Reading an image with Bash normally produces binary garbage or base64, and tools such as `pdftotext`, `jq`, and `unzip` may not be installed. A dedicated tool can: + +- represent an image as visual tokens instead of text; +- automatically correct orientation, resize, transcode, and compress; +- render a PDF by page or extract its text; +- split a Notebook into cells; +- split a PPTX into slides and notes. + +These capabilities are more than command shortcuts: they transform “file bytes” into “content the model can understand” at the protocol boundary. + +#### 5. Friendlier failure recovery + +A dedicated tool knows the current working directory, total line count, supported formats, and invocation parameters. It can therefore return action-oriented errors such as “offset out of range,” “this is a directory,” “the file is excluded by `.gitignore`,” or “did you mean this similar path?” Ordinary Bash usually returns only an errno or a particular command's stderr. + +#### 6. Integration with the Agent lifecycle and state + +Claude Code can avoid resending unchanged files, activate path-specific skills, and record freshness for memory files. Grok can inject path rules. Pi can swap in a remote read backend. OpenCode connects Location permissions, image normalization, and oversized-result retention to a unified finalization pipeline. Codex's command tool can leave a running command in a resumable session, accept later input, and integrate with approvals. A one-off `cat` process has none of these session semantics on its own. + +#### 7. Clearer observability and policy controls + +The framework can record “which file was read, how much was read, whether the result was truncated, and whether a permission rule matched” as structured events. If the system sees only an arbitrary shell script, it must first parse the command and still may not be able to determine precisely what a pipeline, redirection, or subprocess ultimately read. + + +### Drawbacks of a Dedicated File-Reading Tool + +- **Implementation and maintenance cost grows linearly with file types**: each format needs its own branch, and tool descriptions can easily drift from implementations. Grok's claimed Notebook support without a dedicated implementation branch and the discrepancy between Claude Code's prompt saying “2,000 lines maximum by default” and its core implementation are both examples. +- **Tool definitions consume context**: each dedicated tool's schema and description enter the system prompt. When its capabilities overlap with Bash, prompt guidance is also needed for the model to select the right tool consistently; otherwise, it may oscillate between the two. +- **Expressiveness has a ceiling**: line-based offset/limit pagination cannot handle extremely long individual lines or field-based extraction. When faced with a long line, both Pi and Grok explicitly recommend falling back to `sed | head -c` or `jq`. The dedicated tools thus acknowledge their own boundaries and complement Bash rather than replace it. + + +Bash remains appropriate in the following situations: + +- When only content matching is needed, `rg` uses fewer tokens than reading an entire section. +- For a single extremely long JSON record, field- or character-based extraction with `jq`, `cut -c`, or a script is more effective than line pagination. +- Mature CLI tools are more flexible when filtering, sorting, decompression, and deserialization must be composed. +- Pi and Grok both load an entire ordinary file into memory first. For a small window into an extremely large file, Claude Code's or OpenCode's streaming implementation, or `sed`/`awk`, uses less memory. +- Codex's general-purpose shell approach can immediately use a new format processor already installed on the machine, without first adding another built-in branch to the Agent. + +The Codex example must be interpreted correctly. It does not prove that “Bash alone is enough”; it shows that “with sufficiently substantial infrastructure in the command-execution layer—sandboxes, approvals, output budgets, and session management—Bash can perform text reading.” The complexity has not disappeared; it has merely moved from the read tool to the exec tool. Codex has not entirely escaped dedicated reading tools either: `view_image` is one, because image content cannot be expressed through text output at the protocol layer. The real scope of a Bash-only approach is therefore limited to text. + +Another easily overlooked variable is the model's training distribution: tool shapes and models co-evolve. Claude-family models are trained with extensive use of Read/Edit pairs, where line-number anchors directly support subsequent edits. Codex models are trained specifically for shell use. “Codex works well with Bash” is partly true because its models were trained that way; the conclusion does not transfer to other models for free. + +A more accurate conclusion is not “dedicated tools replace Bash,” but rather: + +- **Routine reading of source code, configuration, and multimedia**: dedicated tools are safer, more stable, and more context-efficient. +- **Search, extremely long lines, and ad hoc format conversion**: Bash/CLI is more capable. +- **The best Agent implementations** usually retain both, prompting the model to prefer the dedicated reader and fall back to the shell only for windows or transformations the dedicated tool cannot express. +- The fundamental distinction is not feature coverage—Bash can do almost anything—but **which layer bears the complexity** and **whether permission semantics can be expressed statically**. Dedicated tools move safety and budgets forward into the protocol layer, while the Bash approach requires equally strong sandboxing and approval infrastructure in the execution layer. diff --git a/docs/research/en/write_tool.md b/docs/research/en/write_tool.md new file mode 100644 index 0000000..27ff651 --- /dev/null +++ b/docs/research/en/write_tool.md @@ -0,0 +1,606 @@ +# Comparing File-Writing Tool Designs Across Five Agent Projects + +> Generated from the Chinese source [`../zh-CN/write_tool.md`](../zh-CN/write_tool.md). Do not edit by hand. + +## Scope of the Research + +This document examines the model-callable file-writing tools in five mainstream code-agent projects, focusing on two questions: + +1. Bash can already create and overwrite files, so why design a separate `write` tool? +2. What exactly do the `write`, `edit`, `apply_patch`, and other file-mutation tools in these projects implement, and why do they implement those features? + +Although the subject is `write`, whole-file writes cannot be considered in isolation. `write` usually forms a file-mutation protocol together with `edit` and `apply_patch`: the former expresses “what the final file should be,” while the latter two express “what should change relative to the current file.” This document therefore examines all three categories, but excludes ordinary `fs/writeFile` RPCs intended for GUI clients rather than for model use. + +The conclusions are based on the following source snapshots. Links are pinned to the checked-out commits. `claude-code` is the third-party source mirror used in the current directory, not an official Anthropic open-source repository. + +| Project | Current commit | Commit date | +| --- | --- | --- | +| `grok-build` | [`500129c`](https://github.com/xai-org/grok-build/tree/500129c714ad1b10e6095481f4a8387a2ec52649) | 2026-07-29 | +| `pi` | [`c13ffe1`](https://github.com/earendil-works/pi/tree/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87) | 2026-07-30 | +| `claude-code` | [`a371abb`](https://github.com/yasasbanukaofficial/claude-code/tree/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367) | 2026-04-05 | +| `opencode` | [`8c38d26`](https://github.com/anomalyco/opencode/tree/8c38d260eb6555d2824230be100fb2a7eadd7513) | 2026-07-30 | +| `codex` | [`578c1b2`](https://github.com/openai/codex/tree/578c1b2230288104041e880a86d0f7f3a5ca6e47) | 2026-07-30 | + +## Conclusions + +The user's intuition is correct: **creating or overwriting a file with a sequence of bytes is not complicated, and Bash is entirely capable of doing it.** A standalone `write` tool usually adds no new filesystem-level capability; what it adds is a layer of **structured mutation protocol and control plane**. + +The difference between the two paths is roughly: + +```text +Write / Edit / Patch + → structured intent + → schema validation + → path resolution and permissions + → conflict checks and concurrency coordination + → filesystem mutation + → diff, history, events, LSP, UI + +Bash + → arbitrary command string + → shell expansion, pipelines, redirection, subprocesses + → arbitrary side effects +``` + +Bash can manually reproduce any step above, but when an agent framework sees an arbitrary shell program, it is difficult to answer the following questions reliably before execution: + +- Which file will ultimately be changed, and which file will it be after symbolic links are resolved? +- Is this a create, overwrite, local replacement, move, or delete operation? +- What will the file look like when the user approves the operation? +- Has the file been changed by the user, a formatter, or another tool since the model read it? +- Which diff, diagnostics, history snapshot, and audit event should be sent to the UI? +- Are parallel calls in the same turn writing to the same underlying file? +- How can the same semantics be ported to Windows, SSH, a VM, or a virtual filesystem? + +A more accurate summary is therefore: + +> Bash answers “can it write?”; dedicated file tools answer “with what semantics does the model write, who approves it, how are incorrect writes prevented, and how does the system know what was written?” + +The converse must also be emphasized: **a dedicated `write` tool is not itself a security boundary.** If the agent also has unrestricted Bash access, path protections implemented only in `write`/`edit` can still be bypassed through `printf > file`, Python, or `sed -i`. A real security boundary must cover every write channel—for example, by disabling Bash, placing Bash inside the same filesystem sandbox, or isolating the entire process at the OS, container, or VM layer. + +## Overview + +| Project | Model-facing file-mutation interface | Design orientation of whole-file writes | Key characteristics | +| --- | --- | --- | --- | +| Grok Build | Exposes `write`, `search_replace`, `apply_patch`, or Hashline `edit_file`, depending on the preset | The tool itself is thin; heavier capabilities live in session orchestration and adjacent editing protocols | Path permissions, plan gate, hooks, batch locks keyed by the path string in the arguments, mutation events, hunk/rewind support; `search_replace` and Hashline provide stronger preconditions | +| Pi | Enables `read`, `bash`, `edit`, and `write` by default | Lightweight, embeddable, and backend-replaceable | Automatic directory creation, tolerant path handling, a serial queue for the same underlying file, abort support, TUI preview, extension hooks, SSH/VM/`ExecutionEnv` support | +| Claude Code | `Write`, `Edit`, plus `NotebookEdit` | The heaviest “managed whole-file replacement” | Mandatory full Read first, mtime/content stale-write protection, path and symlink permissions, history backups, best-effort atomic replacement, permission-bit preservation, diff, LSP, editor notifications | +| OpenCode | Both V1 and V2 provide `write`, `edit`, and `apply_patch` | Two generations coexist in one repository; V1 has thick integration, while the V2 mutation core is stricter | Location boundaries, canonical paths, fine-grained permissions, BOM/newline handling; V2 edit provides byte-level CAS, while V1 provides formatters, events, LSP, and fuzzy editing | +| Codex | **No standalone whole-file `Write`**; uses `apply_patch` with a free-form syntax | Treats structured Patch as sufficient for both creation and modification; ordinary commands remain the responsibility of `exec_command` | Add/Delete/Update/Move, multi-file patches, context validation, approval and sandboxing, remote FS, structured diff/events; not a multi-file transaction | + +These five projects effectively provide three different answers: + +1. **Heavyweight dedicated tool**: Claude Code incorporates read state, permissions, history, disk writes, and IDE integration into the `Write` lifecycle. +2. **Lightweight structured interface**: Pi and Grok keep the whole-file `write` implementation simple; most of its value comes from schemas, hooks, UI, queues, events, and replaceable backends. +3. **No whole-file Write**: Codex provides only `apply_patch`; creating a file is an `Add File` patch, and a complete rewrite can also be expressed as a patch. + +## Why Build a Dedicated Writing Tool? + +### 1. Treat File Contents as Data, Not as Part of a Shell Program + +`write({ path, content })` still requires correctly generated JSON, but `content` is no longer processed by shell variable expansion, command substitution, globs, heredocs, or redirection syntax. + +Writing a file that contains backticks, `$()`, quotes, an arbitrary heredoc delimiter, or a binary NUL with Bash requires the model to handle both the “content language” and the “command language” correctly. A dedicated tool separates the two, and malformed JSON can be returned to the model as a schema error before execution so that it can be corrected. + +This is also a cross-platform concern: a unified file API does not depend on Bash, `sed`, or `perl` happening to be installed on the target machine, nor does it require the model to master the quoting rules of POSIX shells, PowerShell, and `cmd.exe` separately. + +### 2. Turn “Where to Write” into an Authorizable Resource + +For a structured call, the framework knows the target path before execution and can: + +- Normalize relative paths, `..`, and the home directory. +- Resolve the real paths of existing targets and parent directories. +- Check whether symbolic links escape the workspace. +- Authorize in-workspace files separately from explicitly external absolute paths. +- Reject all mutations in plan mode. +- Show the UI which file will be modified. + +For arbitrary Bash, static analysis of `command` can usually handle only simple shapes such as `>`/`>>` reliably. Variables, functions, `eval`, subprocesses, interpreter scripts, dynamic filenames, and symlinks make it unreliable to derive the exact write set before execution. + +This merely makes the policy enforcement point clearer; it does not create security automatically. Pi's security documentation explicitly states that tools inherit the permissions of the running process and have no built-in sandbox. Its path-protection extension example intercepts only `write`/`edit`, so it can be bypassed if Bash remains enabled. [Pi's default tools](https://github.com/earendil-works/pi/blob/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87/packages/coding-agent/src/core/agent-session.ts#L2556-L2595) and its [security-boundary documentation](https://github.com/earendil-works/pi/blob/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87/packages/coding-agent/docs/security.md#L31-L53) make this especially clear. + +### 3. Turn “Which Old Version I Am Modifying” into a Precondition + +One of the most dangerous write errors for an agent is not a syntax error, but a silent clobber: + +1. The model reads old content. +2. The user, a formatter, or a parallel tool changes the file. +3. The model overwrites the entire file based on the old content. +4. The intervening update is silently lost. + +A dedicated protocol can choose different strengths of protection: + +- Claude Code requires an existing file to have been read in full and compares its mtime after that read; immediately before writing, it reads synchronously again and falls back to content comparison when necessary. +- OpenCode V2's `edit` reads the original bytes after permission is approved, then calls `writeIfUnchanged(expectedBytes)` while holding the same canonical-path lock at commit time. +- Edit/SearchReplace/Patch operations require the old text or context to still match; some Edit/SearchReplace implementations also require the match to be unique and reject the operation otherwise. +- Pi serializes in-process `write`/`edit` calls through a mutation queue for the same underlying path. + +These mechanisms provide different levels of protection, and none is equivalent to a cross-process, kernel-level atomic compare-and-swap. An external program can still race between “check” and “write”; ordinary cross-process file locks usually coordinate only writers that participate in the same locking protocol. Stronger guarantees require mandatory revision/CAS semantics, a transactional backend, or complete isolation—not treating an advisory lock as a boundary that all processes must obey. + +### 4. Coordinate Changes to the Same File Across Parallel Agents + +Modern agents often execute multiple tool calls concurrently within a single turn. If two edits both perform “read old file → compute new content → write back,” the later write overwrites the earlier one. + +Dedicated file tools can queue operations by canonical path or `realpath`: + +- Serialize operations on the same underlying file. +- Continue processing different files in parallel. +- Try to coalesce symlink aliases into the same queue. +- Hold the lock for the entire read-compute-write sequence, not just the final `write()`. +- On abort, wait for the underlying I/O to settle before releasing the lock so that the canceled call does not interleave with a subsequent write. + +Pi's implementation and documentation explicitly describe this queue as a mechanism for preventing lost updates: [mutation queue](https://github.com/earendil-works/pi/blob/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87/packages/coding-agent/src/core/tools/file-mutation-queue.ts#L1-L60), [extension documentation](https://github.com/earendil-works/pi/blob/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87/packages/coding-agent/docs/extensions.md#L1865-L1873). + +Bash writes and external editors do not automatically participate in such an in-process queue. Giving Bash the same guarantee requires proxying all file I/O through the same mutation service or relying on workspace isolation at a higher layer. + +### 5. Generate Reviewable Information Before User Approval + +A structured edit or patch can calculate the following before writing to disk: + +- Whether the operation is a create, update, delete, or move. +- A unified diff. +- Additions and deletions. +- The number of matches and the fuzzy-matching method used. +- The set of canonical resources that will be affected. + +The UI can therefore display an actual mutation preview instead of merely showing a shell command that might indirectly invoke ten programs. After the write, it can also return the final diff, so post-processing by a formatter does not cause the approved result and the actual result to diverge silently. + +Whole-file `write` preview strategies vary: Pi shows only the new content to be written; Claude Code and OpenCode V1 read the old content and generate a diff; OpenCode V2 whole-file write does not generate a diff. + +### 6. Integrate History, Events, Formatters, LSP, and Editors + +A dedicated tool call provides a stable lifecycle boundary: + +```text +before_tool / permission +→ before_file_edit snapshot +→ mutate +→ formatter +→ file-written event +→ LSP didChange / didSave / diagnostics +→ UI diff / history / telemetry +→ after_tool +``` + +Claude Code, OpenCode V1, and Grok each connect some of these stages to their file tools. When only arbitrary Bash is run, the framework generally has no option but to scan the entire working tree after the command finishes; that cannot provide a reliable pre-execution diff, and it is difficult to distinguish changes made by the current command from changes made by background processes, the user's editor, or a formatter. + +### 7. Apply the Same Semantics to Different Execution Backends + +Pi injects operations such as `readFile`, `writeFile`, and `mkdir` into its tools, while the Harness version requires all built-in tools to go exclusively through a host-provided `ExecutionEnv`. Repository examples reuse the same `write`/`edit` protocol for SSH hosts and Gondolin VMs: [operations interface](https://github.com/earendil-works/pi/blob/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87/packages/coding-agent/src/core/tools/write.ts#L21-L40), [Harness constraint](https://github.com/earendil-works/pi/blob/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87/packages/agent/docs/agent-harness.md#L82-L84). + +Codex likewise routes final Patch execution to a local, sandboxed, or remote filesystem implementation rather than requiring a complete shell on the remote system. This turns file mutation into a capability interface that the host can provide. + +## Implementations by Project + +### 1. Grok Build + +#### The Tools Actually Exposed Depend on the Preset + +Grok Build maintains multiple mutation protocols simultaneously rather than giving every model the full set of overlapping tools: + +- The original `default_grok_build_toolset()` lists only `search_replace`. +- The current `grok-build` workspace preset adds an OpenCode-style `write` to that set. +- The default `AgentBuilder` also has `write_file_enabled` set to true and dynamically adds the write tool when none is present, so the default runtime toolset is likewise `search_replace` + `write`. +- The Codex preset uses `apply_patch`. +- The OpenCode preset uses `edit`/`write`. +- The optional Hashline configuration replaces the standard read/search/edit slots, but retains a standalone `write` when the default write feature is enabled. + +These combinations are registered centrally in the [agent preset configuration](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-agent/src/config.rs#L170-L228) and [tool registry](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-tools/src/registry/types.rs#L674-L757); the logic that dynamically adds Write by default is in [`AgentBuilder`](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-agent/src/builder.rs#L706-L765). This design shows that tool shape is also part of the model-adaptation layer: different models have different training distributions for whole-file writes, string replacement, patches, or hash anchors. + +#### `write` + +The model input consists of `file_path` and `content`. During execution, the tool: + +1. Maps the model-provided path to the currently displayed workspace or forked worktree. +2. Reads the old content and existence state on a best-effort basis. +3. Creates parent directories recursively. +4. Writes the full content through the filesystem backend. +5. Emits a `FileWritten` event containing the previous and new content plus `is_new`. +6. Returns the create/update type, a structured whole-file edit, and line-count statistics. + +The main implementation is [`OpenCodeWriteTool`](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-tools/src/implementations/opencode/write/mod.rs#L20-L195). + +Here, the old content is read to populate the mutation event and result; it is not a read-before-write guard. The current whole-file `write`: + +- Does not check whether the model called Read first. +- Does not compare an mtime, hash, or expected bytes. +- Does not use a temporary file followed by rename. +- Uses straightforward last-writer-wins semantics. +- Retries only transient Windows sharing/lock failures in the local backend. + +A read failure is treated as if the target did not exist, and processing continues rather than failing closed; this internal Read therefore cannot be interpreted as a safety check. + +The corresponding local disk write is implemented by [`LocalFs::write_file`](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-tools/src/computer/local/file_system.rs#L56-L87). + +Heavier capabilities live in the outer session executor: a plan-mode gate, a pre-tool hook, edit permission requested for the target file, and a same-path mutex within one batch of model calls. [Permissions and hooks](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-shell/src/session/acp_session_impl/tool_calls.rs#L952-L1160) and the [batch lock](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-shell/src/session/acp_session_impl/tool_calls.rs#L453-L504) are reused by all structured writing tools. The lock key comes from the raw path string in the arguments and is valid only within the current parallel batch; `a.py`, `./a.py`, an absolute path, and a symlink alias can still receive different locks. + +`FileWritten` is in turn consumed by the notification bridge for hunk tracking, auditing, and rewind. Thus, even though the `write` core is simple, the system still knows the before and after contents. The [event structure](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-tools/src/notification/types.rs#L185-L209) explicitly stores the previous content, while the [notification bridge](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-shell/src/tools/notification_bridge.rs#L353-L376) passes it to the hunk tracker and rewind snapshot. + +#### `search_replace` and Hashline + +`search_replace` expresses a local edit with `old_string`/`new_string`: + +- By default, the old string must be unique unless `replace_all` is explicitly set. +- `old_string=""` can also create or completely overwrite a file, and the protection that prevents an empty old string from overwriting an existing nonempty file is not enabled by default. +- It preserves CRLF. +- It can perform a limited fallback for Unicode confusables. +- It can reject targets matched by `.gitignore`. +- It returns a structured diff and context. +- “Read first” is primarily encouraged through the tool description and dependency metadata; no session read revision is enforced. + +The implementation entry point and parameter semantics are documented in [`search_replace`](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-tools/src/implementations/grok_build/search_replace/mod.rs#L59-L139). + +The optional Hashline protocol makes Read return “line number + content hash” anchors. A single edit can combine replace, insert-after, and whole-file write operations. All anchors are validated against the current content before any write; a stale, ambiguous, overlapping, or missing anchor causes the entire logical batch of mutations to fail before writing. If validation succeeds, the new content is written once. The [Hashline interface](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-tools/src/implementations/grok_build_hashline/edit/mod.rs#L25-L48) and [batch validation](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-tools/src/implementations/grok_build_hashline/edit/apply.rs#L143-L305) embody the idea of “preventing the model from editing against stale context through explicit preconditions.” Hashline's built-in whole-file write carries no anchor and remains a complete overwrite. + +Here, “the entire batch fails” describes in-memory mutation-validation semantics; it does not mean that the underlying filesystem write is transactional or crash-atomic. The current HashlineEdit metadata and run path also do not emit `FileWritten`, so although a structured diff is available, Grok's hunk-attribution and rewind before-snapshot notification chain does not run as it does for OpenCode Write/SearchReplace. The [Hashline execution path](https://github.com/xai-org/grok-build/blob/500129c714ad1b10e6095481f4a8387a2ec52649/crates/codegen/xai-grok-tools/src/implementations/grok_build_hashline/edit/mod.rs#L213-L434) shows that it reads, computes, writes, and returns the result directly. + +### 2. Pi + +#### `write` Itself Is Very Thin + +The public schema contains only: + +```text +path: string +content: string +``` + +The tool promises to create or overwrite a file and automatically create parent directories. Its prompt tells the model to use it only for new files or complete rewrites, leaving local modifications to `edit`. See the [schema and description](https://github.com/earendil-works/pi/blob/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87/packages/coding-agent/src/core/tools/write.ts#L14-L40). + +The execution layer implements: + +- Tolerant handling of relative and absolute paths, `~`, `file://`, a leading `@`, and special Unicode spaces. +- A per-file mutation queue entered after resolving an existing file through `realpath`. +- Recursive `mkdir`. +- AbortSignal checks before and after I/O. +- Injectable `writeFile`/`mkdir` operations. +- The written length in a successful result. +- A streaming TUI display of the target path and syntax-highlighted new content, collapsed to the first ten lines by default. + +The core call is in [`write.ts`](https://github.com/earendil-works/pi/blob/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87/packages/coding-agent/src/core/tools/write.ts#L181-L260). + +The abort semantics are worth noting: once the underlying I/O begins, it may not be cancelable. Pi waits for it to settle before reporting the abort and releasing the queue, preventing an unfinished write from interleaving with the next one. This provides concurrency ordering, not rollback. + +#### `edit` Handles the Complexity of Local Mutations + +Pi's `edit` accepts multiple `{ oldText, newText }` entries in one call: + +- All old text is matched against the same original snapshot. +- Empty targets, missing matches, multiple matches, overlaps, and no-ops are rejected. +- Once all entries pass validation, they are applied in reverse order and written only once. +- If exact matching fails, it can tolerate trailing whitespace, Unicode NFKC, smart quotes, dashes, and special spaces. +- It preserves the BOM and the original newline style. +- It returns a compact diff for the TUI, a standard unified patch, and the first changed line. +- When the arguments are complete, it can generate an asynchronous diff preview before actual execution. + +The multi-replacement algorithm is in [`edit-diff.ts`](https://github.com/earendil-works/pi/blob/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87/packages/coding-agent/src/core/tools/edit-diff.ts#L251-L374), and execution and results are in [`edit.ts`](https://github.com/earendil-works/pi/blob/c13ffe1877c3a47ce9f2fc98d9880447d64a0e87/packages/coding-agent/src/core/tools/edit.ts#L287-L430). + +#### Extensions and Boundaries + +A `tool_call` hook can change arguments or block a call, while a `tool_result` hook can replace the result; an extension with the same name can also override a built-in implementation. This gives path protection, approval, auditing, SSH/VM backends, and custom renderers stable integration points. + +Pi's limitations are equally explicit: + +- Whole-file `write` has no prior-Read, mtime, hash, or expected-content check. +- The local implementation calls `writeFile` directly, with no temp + fsync + rename sequence. +- `write` does not read the old file to generate a diff; it previews only the new content. +- The mutation queue coordinates only Pi tools using the same queue, not Bash or external processes. +- Paths are not restricted to the cwd by default; `realpath` exists to coalesce queue entries, not to provide a sandbox. +- Tool arguments are not schema-validated again after an extension hook changes them. +- Bash is enabled by default, so hook-level path protection cannot independently form a permission boundary. + +Pi's design emphasis is not “writing files better than Bash,” but obtaining a TUI, hooks, concurrency, and a replaceable execution environment through the smallest practical protocol. + +### 3. Claude Code + +#### A Clear Division of Responsibility Between Write and Edit + +`Write` takes only an absolute `file_path` and the complete `content`. Its prompt explicitly says: + +- Use `Write` only for new files or complete rewrites. +- Prefer `Edit` for modifying existing files because it sends only a diff. +- Existing files must first be read in full, or the tool fails. + +See the [`FileWriteTool` description](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileWriteTool/prompt.ts#L1-L19). The system prompt also explains the product motivation directly: dedicated tools make work easier for users to understand and review, so files should not be created with heredocs or `echo` redirection. See the [system tool guidance](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/constants/prompts.ts#L286-L309). + +#### Read First Is Actually Enforced at Runtime + +This is not merely a prompt instruction: + +1. Expand the path and check whether the content writes secrets into team memory. +2. Match deny rules early. +3. Do not stat a UNC path before permission is granted, avoiding Windows SMB/NTLM credential leakage. +4. Require a complete Read state for an existing file. +5. Reject the operation if the file's mtime postdates the prior Read. +6. Immediately before writing, synchronously read metadata and mtime again. +7. In environments such as Windows where mtime can produce false positives, fall back to content comparison if the file was read in full. +8. Deliberately avoid an asynchronous yield between the check and the write, narrowing the in-process race window. + +Input checks are in [`validateInput`](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileWriteTool/FileWriteTool.ts#L153-L218), and the final pre-write recheck is in [the same tool's call](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileWriteTool/FileWriteTool.ts#L249-L305). + +Permission checks consider not only the input string, but also the original path, the symlink chain, the nearest existing parent of a dangling target, special files, and the final resolved path, preventing authorization based only on the superficial path. See [file-operation permission resolution](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/utils/fsOperations.ts#L288-L381). + +#### Writing to Disk and Post-Processing + +Execution also: + +- Creates parent directories automatically. +- Saves a file-history snapshot before writing. +- Preserves the encoding of an existing file. +- Treats the model-provided content as a complete replacement, respecting its explicit newlines instead of silently retaining the old file's CRLF style. +- Writes through a symlink to its target rather than replacing the link itself. +- Writes and flushes a temporary file in the same directory, preserves the original permission bits, and then renames it over the target. +- Cleans up the temporary file and falls back to a direct flushed write if the atomic-write path fails. +- Notifies LSP with `didChange`/`didSave` and clears stale diagnostics. +- Notifies the VS Code diff view. +- Updates Read state so that subsequent edits use the new version. +- Returns create/update status, a structured patch, the original content, and line-count statistics. + +The best-effort atomic write and fallback are implemented by [`writeFileSyncAndFlush`](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/utils/file.ts#L354-L477); LSP, editor integration, and diff results are in [`FileWriteTool.call`](https://github.com/yasasbanukaofficial/claude-code/blob/a371abbe75ffa0d0a3c92290e2bbf56a7ef54367/src/tools/FileWriteTool/FileWriteTool.ts#L297-L416). + +This is the most complete whole-file `write` lifecycle among the five projects, but it still has boundaries: + +- If atomic rename fails, it falls back to a non-atomic overwrite. +- Read/mtime/content comparison is not a cross-process transactional CAS. +- Disk writes themselves use synchronous I/O. +- Whole-file content has a high token cost, so the prompt still instructs the model to prefer `Edit` for existing files. +- Bash is a separate capability channel, so security still depends on a unified permission and sandbox design. + +### 4. OpenCode + +The current OpenCode repository contains two implementations at the same time: + +- V1/legacy: `packages/opencode/src/tool/*`, still used by the old Session/CLI/TUI path. +- Core V2: `packages/core/src/tool/*`, the newer Location-scoped, schema-first architecture. + +They must not be conflated. V1's file tools have more complete integrations, while V2's mutation primitives and concurrency semantics are clearer. The individual V2 file-tool implementations still lack formatter, explicit file-edit/watcher events, and LSP integration, and snapshot/undo integration in those implementations remains marked TODO. However, the V2 session layer already has a general Snapshot capture/diff/restore service, so it would be incorrect to say that V2 as a whole has no snapshots. The [V2 `write` TODO](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/core/src/tool/write.ts#L19-L47) and [session-layer Snapshot calls](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/core/src/session/runner/llm.ts#L217-L333) illustrate these two layers respectively. + +#### Why It Provides write, edit, and apply_patch Together + +- `write`: Create a file or explicitly overwrite it when the model already knows the complete final content. +- `edit`: Express a small change with old/new strings, saving tokens and preserving the rest of the content. +- `apply_patch`: Express multi-file add/update/delete operations in one call, suited to models that work well with diffs. + +The V1 registry selects interfaces by model: modern non-OSS GPT models receive `apply_patch`, while other models mainly see `edit` + `write`, avoiding presenting every overlapping tool to every model simultaneously. See [V1 tool selection](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/opencode/src/tool/registry.ts#L286-L306). + +#### Core V2 Location and Permissions + +The three mutation tools share `LocationMutation`: + +- Relative paths must remain within the current Location; crossing the boundary with `../` fails immediately. +- Existing targets are resolved through `realPath`. +- For a nonexistent target, the nearest existing ancestor directory becomes the canonical anchor. +- A workspace symlink pointing outside the workspace produces `location_escape`. +- An explicit external absolute path first requests `external_directory` permission for its parent directory, then `edit` permission for the specific resource. +- Permission resources use Location-relative identities for internal paths and canonical absolute identities for external paths. + +See [`LocationMutation`](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/core/src/location-mutation.ts#L90-L152). + +#### Core V2 `write` + +V2 whole-file `write` resolves and authorizes the target, creates parent directories automatically, preserves a UTF-8 BOM, and returns `created`/`wrote` status, the canonical target, the permission resource, and `existed`. See the [execution flow](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/core/src/tool/write.ts#L63-L97). + +It intentionally retains clear last-writer-wins semantics: + +- It does not require a prior Read. +- It has no mtime, hash, or expected-bytes check. +- It does not generate a diff. +- It does not perform a temporary-file rename. +- It does not explicitly manage or return the mode. Overwriting an existing regular file generally retains the underlying file permissions; permissions on a new file depend on backend defaults and the umask. +- It provides only in-process serialization for the same canonical target. + +V2 `edit` is stricter: + +- It requires an exact match for the old string. +- It prohibits an unchanged replacement and an empty old string. +- The match must be unique by default, unless `replaceAll` is set explicitly. +- It preserves the BOM and original newline style. +- It returns replacements, a unified diff, and addition/deletion counts. +- After authorization, it reads the original bytes, then commits under a canonical-path lock using `writeIfUnchanged(expectedBytes)` and reports stale if the file has changed. + +See [`edit`](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/core/src/tool/edit.ts#L42-L159) and [`FileMutation`](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/core/src/file-mutation.ts#L69-L166). + +V2 `apply_patch` first parses and resolves all hunks, authorizes them in a batch, and reads and preflights every update/delete before committing them in order. Add uses create-only `wx`, so it cannot overwrite a file that appears during approval; Update uses expected bytes; Move is not yet supported. See the [Patch flow](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/core/src/tool/apply-patch.ts#L85-L202). + +It is explicitly not transactional: if a later operation fails during the commit phase, earlier successful mutations remain and are listed in the error. Delete also has no expected-content CAS at commit time. + +#### V1's Thick Integration + +V1 `write` reads the old content to generate a permission diff, preserves the BOM, writes after authorization, runs the formatter, publishes file/watcher events, touches the LSP, and waits for diagnostics. See [V1 `write`](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/opencode/src/tool/write.ts#L46-L122). + +V1 `edit` also implements multiple layers of fuzzy replacers, a per-file semaphore, a final post-formatter diff, file events, and LSP error feedback. V1 Patch supports move, but both Add and Move can overwrite their targets, and neither provides V2's byte-level CAS. + +One implementation/prompt-text mismatch worth recording is that V1 `write.txt` and `edit.txt` claim existing files must first be read or the tool will fail. The actual code does not query session Read history or mtime; it simply reads the current file itself and proceeds. The [prompt text](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/opencode/src/tool/write.txt#L1-L8) and [actual implementation](https://github.com/anomalyco/opencode/blob/8c38d260eb6555d2824230be100fb2a7eadd7513/packages/opencode/src/tool/write.ts#L46-L122) do not agree. + +This demonstrates that “having a separate tool” merely provides a place to implement a guard; it does not guarantee that the guard actually exists. Research must distinguish among prompts, TODOs, and runtime code. + +### 5. Codex + +#### No Whole-File Write + +Codex's standard local Coding Turn does not expose `write_file` or `edit_file` to the model; it provides only `apply_patch` with free-form input. App Server also has an `fs/writeFile` RPC, but that is intended for the host client rather than as a model tool and does not belong in this comparison. + +`apply_patch` does not ask the model to assemble a shell command. It is a free-form tool with its own grammar: + +- `*** Add File` +- `*** Delete File` +- `*** Update File` +- `*** Move to` +- Multi-file patches +- Context lines and an EOF anchor + +The tool schema and syntax are defined by [`apply_patch_spec`](https://github.com/openai/codex/blob/578c1b2230288104041e880a86d0f7f3a5ca6e47/codex-rs/core/src/tools/handlers/apply_patch_spec.rs#L18-L31) and the [Lark grammar](https://github.com/openai/codex/blob/578c1b2230288104041e880a86d0f7f3a5ca6e47/codex-rs/core/src/tools/handlers/apply_patch.lark#L1-L19). + +This provides the most direct counterexample to the question at hand: **needing structured file mutations does not imply needing a standalone whole-file `Write`.** Add File can already create a complete file, while Update Patch is better suited to ordinary code changes. + +#### Implemented Capabilities + +The execution chain: + +1. Parses the complete patch. +2. Reads old files for Update/Delete, computing the new content and a unified diff for Update; Add directly carries the target content. +3. Validates all hunks before executing any action. +4. Searches for context in stages: exact → ignore trailing whitespace → trim both sides → normalize Unicode punctuation. +5. Classifies safety under the permission profile, requests approval when necessary, and uses a platform sandbox in managed configurations. +6. Executes against a local, sandboxed, or remote filesystem. +7. Sends begin/end events for the execution lifecycle; when the corresponding feature is enabled, it can also stream patch-diff updates while arguments are being generated. +8. Returns stable A/M/D results and integrates with hooks. + +Parsing and prevalidation are implemented in the [`apply-patch` invocation](https://github.com/openai/codex/blob/578c1b2230288104041e880a86d0f7f3a5ca6e47/codex-rs/apply-patch/src/invocation.rs#L180-L239), context matching in [`seek_sequence`](https://github.com/openai/codex/blob/578c1b2230288104041e880a86d0f7f3a5ca6e47/codex-rs/apply-patch/src/seek_sequence.rs#L1-L96), and safety classification in [`safety.rs`](https://github.com/openai/codex/blob/578c1b2230288104041e880a86d0f7f3a5ca6e47/codex-rs/core/src/safety.rs#L32-L86). + +Even if the model invokes it through a shell-shaped call such as `apply_patch <<'PATCH' ...`, Codex recognizes that constrained form and routes it through the same Patch safety chain rather than treating it as arbitrary shell side effects. + +#### Important Boundaries + +- A multi-file Patch is not transactional: hunks are written in order, and a later failure does not roll back previously successful items. +- Add File can currently overwrite an existing target; it is not create-only. +- Move writes the target first and then deletes the source, so a failed deletion can leave both files in place. +- The end result is still a whole-file text rewrite; binary deltas and mode metadata are unsupported. +- `apply_patch` does not run in parallel by default and acquires an exclusive lock on the shared execution gate for the tool batch. This is not a session-wide, process-wide, or file-level global lock, however, and it provides no cross-process revision CAS. +- An external user process can race between validation and the final write. +- No tool-local patch-size limit is evident in the `apply-patch` parser/runtime, although model, API, and context layers still constrain practical input size. + +Sequential execution and retention of already-committed deltas after failure are implemented in the [`apply-patch` library](https://github.com/openai/codex/blob/578c1b2230288104041e880a86d0f7f3a5ca6e47/codex-rs/apply-patch/src/lib.rs#L390-L510). + +Codex places much of the infrastructure in its general command-execution layer and Patch runtime instead of building separate model interfaces for Read, Write, and Edit. This reduces the number of tools but does not eliminate the complexity of permissions, sandboxing, output, events, remote filesystems, and concurrency control; it merely moves that complexity to a different layer. + +## Mapping Features to Design Goals + +| Feature | Primary problem addressed | Project examples | +| --- | --- | --- | +| `{ path, content }` schema | Separating content from commands, argument validation, cross-platform quoting | Pi, Claude Code, OpenCode, Grok | +| Semantic division among Write/Edit/Patch | Controlling tokens, preserving untouched content, adapting to different models | All five projects; Codex retains only Patch | +| Canonical-path / symlink permission checks | Permission aliases and path escape | Claude Code, OpenCode V2 | +| Queue coalescing by `realpath` | Making path aliases share an in-process write queue where possible | Pi | +| Automatic parent-directory creation | Saving one tool call and avoiding a preliminary `mkdir` | The create path in all five projects | +| Prior Read / mtime / expected bytes | Preventing stale-context overwrites of newer changes | Claude Code, OpenCode V2 edit | +| Exact old text / patch context / hash anchor | Turning assumptions about the old version into verifiable preconditions | Pi/OpenCode/Grok edit, Codex Patch, Grok Hashline | +| Per-file queue / lock | Preventing lost updates from parallel calls by the same agent | Pi, OpenCode V2; OpenCode V1 only for edit; Grok only for identical raw path strings in the same batch; Codex uses a coarser batch execution gate | +| Temp + flush + rename | Reducing the risk of a crash or interruption leaving a truncated file | Claude Code; most other whole-file Write implementations overwrite directly | +| Mode / permission-bit management | Avoiding damage to executable bits and other permissions when replacing a file | Claude Code | +| BOM / newline-style handling | Avoiding unrelated diffs from small changes or damage to encoding markers | Pi edit, OpenCode; Claude Code preserves encoding but respects model-specified newlines | +| Diff / additions / deletions | Approval, review, model feedback, telemetry | Claude Code, OpenCode edit/patch, Pi edit, Codex Patch | +| History / rewind / file events | Undo, auditing, UI synchronization, hunk attribution | Claude Code, Grok, OpenCode V1 | +| Formatter / LSP diagnostics | Creating an automatic mutation–diagnosis–repair loop after writing | Claude Code, OpenCode V1 | +| Operations / ExecutionEnv / FS trait | SSH, VMs, containers, remote or virtual filesystems | Pi, Codex, Grok | +| Hook and/or permission lifecycle | Blocking, rewriting, approving, and recording a mutation | All five projects have some subset; Pi relies mainly on extension hooks and has no built-in sandbox/approval | + +## Costs and Pitfalls of Dedicated Writing Tools + +### 1. They Can Easily Create Duplicate Implementations + +If `write`, `edit`, `apply_patch`, and Bash each have separate path-resolution, permission, disk-write, and event logic, inconsistencies emerge: + +- One tool checks symlinks while another does not. +- One tool replaces atomically while another truncates directly. +- One tool generates the final diff while another generates only the pre-approval diff. +- One tool participates in a same-file queue while Bash does not. +- The prompt claims Read is mandatory, but the runtime does not check state. + +The parity gaps between OpenCode V1 and V2, and Bash bypasses in Pi, both show that the more dedicated tools there are, the more important it becomes to have a shared mutation core rather than duplicating multiple `writeFile()` calls. + +### 2. “Structured” Does Not Mean “Atomic” or “Transactional” + +These three concepts should be separated: + +- **Structured**: the framework knows the operation's intent and target. +- **Single-file crash atomicity**: observers generally see either the complete old version or the complete new version. +- **Multi-file transaction**: the entire batch succeeds or rolls back as a unit. + +Claude Code provides only a best-effort single-file temp + rename path and falls back on failure; Codex and OpenCode Patch are explicitly not multi-file transactions; Pi, Grok, and OpenCode whole-file writes primarily overwrite directly. + +### 3. Whole-File Write Can Cost More Tokens + +If the model must resend the complete content to modify three lines in a large file: + +- It uses more input tokens. +- It is more likely to accidentally change parts of the file that were not meant to be touched. +- Conflict retries are more expensive. +- The UI must recompute a diff from the whole file. + +Claude Code, Pi, and OpenCode therefore position `Write` for new files or complete rewrites and prefer `Edit`/Patch for ordinary changes. Codex goes one step further and does not provide a whole-file Write at all. + +### 4. Models Can Waver Between Overlapping Tools + +Putting three similar schemas into the system prompt consumes context and increases tool-selection errors. OpenCode switches between Patch and Edit/Write by model, while Grok uses presets to select SearchReplace, OpenCode, or Hashline protocols. This shows that tool interfaces need to be designed together with the model's training distribution. + +### 5. They Can Create a False Sense of Security + +Adding a “cannot change `.env`” hook to `write` is easy, but as long as Bash can still execute `python -c` or use redirection, the rule is not a boundary. The correct approach is to: + +- Disable Bash in restricted mode; or +- Put Bash and file tools inside the same workspace/filesystem sandbox; or +- Control the entire agent at the container, VM, or OS-permission layer; +- Use hooks only as a friendlier policy and approval layer, without presenting them as the sole security mechanism. + +## When Bash Is Already Enough + +Using Bash alone is a reasonable product choice, especially when all of the following are true: + +- The execution environment is trusted and handles only a temporary local workspace. +- The agent is single-threaded and will not modify the same file concurrently. +- The user does not need pre-execution diffs, per-file approvals, history rollback, or real-time IDE synchronization. +- There is no need to disable arbitrary commands while exposing only the ability to modify files. +- There is no need to support backends without a complete shell, such as SSH/VM/browser virtual filesystems. +- The model is well trained for shell use. +- The project is willing to invest in security, approval, and output control at the exec layer. + +Codex proves that “no whole-file Write” is entirely viable, but it is not “bare Bash”: it still has structured `apply_patch`, and its command-execution layer itself provides sandboxing, approval, sessions, output budgets, and remote-environment support. + +Bash also remains better suited to: + +- Running formatters, code generators, compilers, or database migration tools. +- Applying mechanical transformations across many files. +- Managing chmod, symlinks, special modes, pipelines, and complex file selection. +- Using mature CLIs for operations not represented by dedicated tools. + +## Recommendations for nanoPyCodeAgent + +> **Revision dated 2026-08-04**: The first version of this section reduced the value of a thin `write` over Bash to “minor quoting convenience” and therefore recommended remaining Bash-first. After review, that framing was found to understate two benefits. The revised conclusion is: **a thin `write` is worthwhile now because it separates content from commands and improves terminal presentation—not because it provides a control plane.** + +The principle “do not add a thin wrapper merely because mainstream agents all have one” still holds; the critical question is whether the rationale is genuine. There are two reasons to add one now: + +**First, separating content from commands prevents silently corrupted files, not merely inconvenient quoting.** The typical failure when using a heredoc to write a file is not an error followed by a retry: if the delimiter is unquoted, `$var`, backticks, and `$()` inside the content are expanded by the shell. The resulting file is corrupted while the exit code remains 0, so the model receives no error to retry. If the content happens to contain the delimiter, the file is silently truncated. For a nano agent without diff review or post-write verification, these silent corruptions are precisely the failures that are hardest to detect. `write({path, content})` treats content as data and eliminates this entire class of failures from day one, without depending on any control plane. + +**Second, this repository has already used `read` to reject the argument that “if Bash can do it, it does not need a dedicated tool.”** The `read` tool adds no filesystem capabilities beyond `cat`/`sed` either. It exists to standardize line numbers and output limits, and to return not only an explanation when something fails, but also an immediately actionable next step—for example, reporting the total line count or giving a Bash command for splitting overlong lines. These are purely structural and UX benefits. The value of `write` is exactly analogous: structured input, explicit error semantics, and terminal presentation. After tool-output shading, a Bash command carrying hundreds of lines of heredoc content is unreadable in the terminal, whereas a `write` call can follow Pi in collapsing its display to “target file + first few lines of content.” + +The revised tradeoff between the two paths is therefore as follows. + +### Path A (Revised): Bash + a Thin `write` + +This is appropriate for nanoPyCodeAgent's current stage, which emphasizes a minimal implementation and a trusted local workspace: + +- Retain Bash as the general-purpose interface for text and system operations; formatters, bulk transformations, appends, and similar work still use Bash. +- Implement the workspace sandbox, approval, and output budget at the exec layer. +- Add a thin `write`, with size and style aligned to `read_tool.py`: a `{path, content}` schema, rejection of directories and non-regular files, automatic parent-directory creation, and a collapsed terminal display. The tool description must not claim any security properties. +- Do not introduce prior-Read checks, mtime, `expected_revision`, or atomic replacement: in the current single-threaded design without an approval UI, they would be false assurances, and the semantics are simply last-writer-wins. +- Implement `write` before `edit`: `write` is the simplest implementation and has the least ambiguous semantics, whereas the complexity of `edit` lies entirely in matching semantics—uniqueness, fuzzy fallback, newline style—and `apply_patch` additionally requires its own parser. Add `edit` only when the token cost of whole-file rewrites becomes a real pain point. + +### Path B: Build a Unified File-Mutation Control Plane + +When the product needs diff approval, a restricted mode, parallel tool calls, remote environments, undo, or IDE integration, upgrade the file tools into a unified control plane. At that point the important consideration is not the tool names but establishing, first, a core shared by every mutation tool: + +1. Normalize and canonicalize the target, with an explicit workspace and symlink policy. +2. Separate create from overwrite, supporting `must_not_exist` or `expected_revision`/`expected_digest`. +3. Recheck preconditions after permission approval. +4. Serialize read-compute-write by canonical path. +5. Write to a temporary file in the same directory, flush, preserve the mode, and attempt an atomic rename. +6. Return create/update/delete/move status, diffs, and old/new revisions uniformly. +7. Emit before/after, history, LSP/formatter, and other events uniformly. +8. Have `Write`, `Edit`, and `Patch` all call this core. +9. Keep Bash constrained by the same filesystem sandbox so it cannot bypass the boundary. + +On top of this architecture, the model interface can remain small: + +```text +Write(path, content, expected_revision?) +Edit(path, old_text, new_text, replace_all?, expected_revision?) +ApplyPatch(patch_text) +``` + +The revised final assessment is: + +- **Building `write` as a control plane is not worthwhile yet: permissions, review, concurrency, history, and remote execution should still follow Path B when those needs arise (the original judgment stands).** +- **Building a thin `write` for content/command separation and terminal presentation is worthwhile now: it eliminates the entire class of silently corrupted heredoc output, based on the same value judgment already validated by the `read` tool.** +- **A thin `write` must be honest: it must not claim to be a security boundary or introduce stale-write and atomicity mechanisms that add no meaningful value in a single-threaded design.** +- **Defer `edit`/`apply_patch` and the control plane until real needs arise; when they do, all mutation tools should share a mutation core rather than wrapping `writeFile()` as an isolated feature (the original judgment stands).** diff --git a/docs/research/agent_tools.md b/docs/research/zh-CN/agent_tools.md similarity index 97% rename from docs/research/agent_tools.md rename to docs/research/zh-CN/agent_tools.md index e2b3d0c..95ff846 100644 --- a/docs/research/agent_tools.md +++ b/docs/research/zh-CN/agent_tools.md @@ -1,6 +1,6 @@ # 常见 Code Agent 内置工具 -> 本文明确指定使用中文编写和维护。 +> 本文件为**手写中文源文件**（source of truth）；英文版 [`../en/agent_tools.md`](../en/agent_tools.md) 由其生成。 目前 nanoPyCodeAgent 只有 Bash 工具，但主流 Code Agent 通常还会提供其他内置工具。本文调研 Pi、Claude Code、Codex、OpenCode 和 Grok Build 的工具设计。 diff --git a/docs/research/edit_tool.md b/docs/research/zh-CN/edit_tool.md similarity index 98% rename from docs/research/edit_tool.md rename to docs/research/zh-CN/edit_tool.md index 577740f..180394d 100644 --- a/docs/research/edit_tool.md +++ b/docs/research/zh-CN/edit_tool.md @@ -1,6 +1,6 @@ # 五个 Code Agent 的 Edit 工具设计调研 -> 本文明确指定使用中文编写和维护。 +> 本文件为**手写中文源文件**（source of truth）；英文版 [`../en/edit_tool.md`](../en/edit_tool.md) 由其生成。 ## 研究范围与口径 @@ -330,9 +330,9 @@ Pi/Hashline 批量 Edit 做到第 1 项；Claude 的底层写入尽力采用 tem ### 当前约束 -nanoPyCodeAgent 当前只暴露 `read`、`write`、`bash`，默认模型是 Claude Sonnet 4.6；工具调用由单线程循环按模型返回顺序执行。[系统提示与工具集](../../src/nanopycodeagent/agent.py#L37-L50)和[顺序 dispatch](../../src/nanopycodeagent/agent.py#L169-L179)意味着目前没有同一进程内的并行 lost-update 问题。 +nanoPyCodeAgent 当前只暴露 `read`、`write`、`bash`，默认模型是 Claude Sonnet 4.6；工具调用由单线程循环按模型返回顺序执行。[系统提示与工具集](../../../src/nanopycodeagent/agent.py#L37-L50)和[顺序 dispatch](../../../src/nanopycodeagent/agent.py#L169-L179)意味着目前没有同一进程内的并行 lost-update 问题。 -现有 `write` 是明确的 last-writer-wins：直接 `Path.write_bytes`，不要求先 Read、不检查 revision、没有 atomic replace，并跟随指向普通文件的 symlink。[`write_tool.py`](../../src/nanopycodeagent/write_tool.py#L89-L153)。`read` 最多整文件加载 10 MB，并在展示时把 CRLF 的 `\r` 去掉。[大小上限](../../src/nanopycodeagent/read_tool.py#L13-L23)与[换行视图](../../src/nanopycodeagent/read_tool.py#L117-L129)意味着模型复制出的多行旧文本通常只含 LF。新 Edit 必须与这些真实语义相容，不能突然宣称一套 Bash 可以绕过、Write 也没有的安全边界。 +现有 `write` 是明确的 last-writer-wins：直接 `Path.write_bytes`，不要求先 Read、不检查 revision、没有 atomic replace，并跟随指向普通文件的 symlink。[`write_tool.py`](../../../src/nanopycodeagent/write_tool.py#L89-L153)。`read` 最多整文件加载 10 MB，并在展示时把 CRLF 的 `\r` 去掉。[大小上限](../../../src/nanopycodeagent/read_tool.py#L13-L23)与[换行视图](../../../src/nanopycodeagent/read_tool.py#L117-L129)意味着模型复制出的多行旧文本通常只含 LF。新 Edit 必须与这些真实语义相容，不能突然宣称一套 Bash 可以绕过、Write 也没有的安全边界。 ### 结论：现在增加一个薄、精确、单替换 Edit diff --git a/docs/research/read_tool.md b/docs/research/zh-CN/read_tool.md similarity index 99% rename from docs/research/read_tool.md rename to docs/research/zh-CN/read_tool.md index 228da3d..57fd206 100644 --- a/docs/research/read_tool.md +++ b/docs/research/zh-CN/read_tool.md @@ -1,6 +1,6 @@ # 五个 Agent 项目的读文件工具对比 -> 本文明确指定使用中文编写和维护。 +> 本文件为**手写中文源文件**（source of truth）；英文版 [`../en/read_tool.md`](../en/read_tool.md) 由其生成。 ## 研究范围 diff --git a/docs/research/write_tool.md b/docs/research/zh-CN/write_tool.md similarity index 99% rename from docs/research/write_tool.md rename to docs/research/zh-CN/write_tool.md index eb60967..012bc72 100644 --- a/docs/research/write_tool.md +++ b/docs/research/zh-CN/write_tool.md @@ -1,6 +1,6 @@ # 五个 Agent 项目的写文件工具设计对比 -> 本文明确指定使用中文编写和维护。 +> 本文件为**手写中文源文件**（source of truth）；英文版 [`../en/write_tool.md`](../en/write_tool.md) 由其生成。 ## 研究范围