From af9d8815b11e6c9cd52b91fcefbb7bfb050464b2 Mon Sep 17 00:00:00 2001 From: minixalpha Date: Wed, 19 Aug 2026 22:48:54 +0800 Subject: [PATCH 1/6] docs: start the 0.8.x dev note Record why the next step is a headless entry point: the benchmark surveys both conclude that nothing can be measured until the agent can be driven programmatically end to end. --- docs/dev_notes/zh-CN/0.8.x.md | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 docs/dev_notes/zh-CN/0.8.x.md diff --git a/docs/dev_notes/zh-CN/0.8.x.md b/docs/dev_notes/zh-CN/0.8.x.md new file mode 100644 index 0000000..ccbee8f --- /dev/null +++ b/docs/dev_notes/zh-CN/0.8.x.md @@ -0,0 +1,11 @@ +# 开发笔记 — 0.8.x + +> 本文件为**手写中文源文件**(source of truth);英文版 [`../en/0.8.x.md`](../en/0.7.x.md) 由其生成。 + +## 0.8.0 + +目前已经开发了一些基本的 tool: bash, write, read, edit，类似 pi 的基本工具已经有了。下一步，为了从实践中了解，到底要优化哪些功能，新增哪些功能，可以真正完成 harness 层面的一些能力提升，需要跑一些公开的 benchmark，目标就是参考主流模型发布时评测的 benchmark，这样可以在相同模型下，对比不同 harness 的能力，以此找到优化目标。 + +首先，还是先做了调研，参考 [code_agent_benchmark](../../research/zh-CN/code_agent_benchmark.md)，从调研结果看，不管要评测什么 benchmark，首先要实现的，就是非交互模式的 code agent 调用，只有这样，才能通过程序化的方式去运行 agent，端到端地执行任务，再根据任务结果去评测。然后，又根据首先推荐的几个 benchmark，整理了方便运行 benchmark 的非交互模式接口，参考 [benchmark_headless_interface](../../research/zh-CN/benchmark_headless_interface.md.md)。 + +第一个功能，先实现 From d74074b70396eb9d2cde337b554e31a1b0c52596 Mon Sep 17 00:00:00 2001 From: minixalpha Date: Wed, 19 Aug 2026 22:55:25 +0800 Subject: [PATCH 2/6] docs: complete the 0.8.x dev note with the headless CLI contract Fix two broken links, add the release-date placeholder to the version heading, and write up the first feature the benchmark surveys point at: a headless CLI with harness-compatible exit codes. --- docs/dev_notes/zh-CN/0.8.x.md | 49 ++++++++++++++++++++++++++++++++--- 1 file changed, 45 insertions(+), 4 deletions(-) diff --git a/docs/dev_notes/zh-CN/0.8.x.md b/docs/dev_notes/zh-CN/0.8.x.md index ccbee8f..b33f926 100644 --- a/docs/dev_notes/zh-CN/0.8.x.md +++ b/docs/dev_notes/zh-CN/0.8.x.md @@ -1,11 +1,52 @@ # 开发笔记 — 0.8.x -> 本文件为**手写中文源文件**(source of truth);英文版 [`../en/0.8.x.md`](../en/0.7.x.md) 由其生成。 +> 本文件为**手写中文源文件**(source of truth);英文版 [`../en/0.8.x.md`](../en/0.8.x.md) 由其生成。 -## 0.8.0 +## 0.8.0 - YYYY.MM.DD 目前已经开发了一些基本的 tool: bash, write, read, edit，类似 pi 的基本工具已经有了。下一步，为了从实践中了解，到底要优化哪些功能，新增哪些功能，可以真正完成 harness 层面的一些能力提升，需要跑一些公开的 benchmark，目标就是参考主流模型发布时评测的 benchmark，这样可以在相同模型下，对比不同 harness 的能力，以此找到优化目标。 -首先，还是先做了调研，参考 [code_agent_benchmark](../../research/zh-CN/code_agent_benchmark.md)，从调研结果看，不管要评测什么 benchmark，首先要实现的，就是非交互模式的 code agent 调用，只有这样，才能通过程序化的方式去运行 agent，端到端地执行任务，再根据任务结果去评测。然后，又根据首先推荐的几个 benchmark，整理了方便运行 benchmark 的非交互模式接口，参考 [benchmark_headless_interface](../../research/zh-CN/benchmark_headless_interface.md.md)。 +首先，还是先做了调研，参考 [code_agent_benchmark](../../research/zh-CN/code_agent_benchmark.md)，从调研结果看，不管要评测什么 benchmark，首先要实现的，就是非交互模式的 code agent 调用，只有这样，才能通过程序化的方式去运行 agent，端到端地执行任务，再根据任务结果去评测。然后，又根据首先推荐的几个 benchmark，整理了方便运行 benchmark 的非交互模式接口，参考 [benchmark_headless_interface](../../research/zh-CN/benchmark_headless_interface.md)。 -第一个功能，先实现 +第一个功能，先实现 **headless 模式的 CLI**，也就是调研里「最小可行路径」的第一步。它是唯一的硬阻塞项：三个 benchmark 的最小公共契约的头三条——一条命令拿到任务文本跑完退出、在进程当前 cwd 里干活、不交互不提问不等确认——全部卡在这里。当前唯一入口是 `agent.py` 的 `input("You> ")` 循环，容器里 stdin 是 EOF，进程会立刻打印 `Bye!` 退出，一件事都没干。这一步不通，后面任何 harness 层面的改动都无法被程序化地验证，也就无从谈优化目标。 + +### headless CLI 实现契约 + +**命令行形态**（跟着调研的 §5 走）： + +```text +nanoPyCodeAgent [-p/--prompt "<任务>" | --prompt-file | (stdin 管道)] + [--max-turns N] + [--version] +``` + +`--version` 不是凑数的：Harbor 会用适配类的 `get_version_command()` / `parse_version()` 去探测并记录 agent 版本（best-effort，失败不报错）。`_package_version()` 已经写好了，只差挂到 CLI 上。 + +**headless 判定**：给了 `-p` 或 `--prompt-file` 就是 headless；两者都没给时，若 `sys.stdin.isatty()` 为 False，就把整段 stdin 读进来当任务；只有连着 tty 且没给任务，才进现有的 REPL。三种投递方式都要吃下，不是为了形式完整——Harbor 的两个官方范例正好各占一种：Claude Code 用 `printf … | claude --print` 走 stdin 管道，避开 shell 转义和命令行长度限制；mini-swe-agent 用 `--task=` 走参数，并显式把 stdin 接到 `/dev/null`。顺带也就修掉了「容器里 stdin 是 EOF 就立刻 `Bye!` 退出」这个行为。 + +**退出码**：判据只有一条——**失败的是任务，还是 harness**。 + +| 退出码 | 场景 | +| :-: | --- | +| 0 | 模型声明完成；轮数用尽；任务没做成 | +| 非 0 | 无 API 凭证；参数错误 | + +「没做完也 exit 0」是最反直觉、也最容易做反的一条：Harbor 把 agent 命令包在 `set -o pipefail` 下执行，非零退出码直接抛 `NonZeroAgentExitCodeError`，整个 trial 判成 agent 失败，还可能触发重试白烧钱。任务做没做成该由 verifier 从 `/logs/verifier/` 的 reward 文件判，不该由 agent 的退出码判。 + +这里还牵出一个现存 bug：没有 API 凭证时，`run()` 是 `print` 完直接 `return`，`main()` 又没有返回码的概念，最终退出码是 0——在 harness 眼里这等于「跑完了，只是没做出来」，一整批任务会静悄悄地全判 0 分，而且看不出是配置问题。所以 `main()` 要改成返回 int（console script 会拿返回值当退出码）。 + +**轮数上限**：内层的 tool_use 循环现在没有任何上限。交互模式下有人看着，模型卡在「反复试同一条命令」时可以 Ctrl-C；headless 下没人看着，会一路烧到 API 报错为止。`--max-turns` 只是一个计数器的成本，却是 headless 能不能无人值守跑的前提，所以和 CLI 一起做。轮数用尽按上表算 exit 0。 + +**headless 的系统提示词**：当前这份是写给对话助手的。headless 下模型没有人可问——它礼貌地问一句「要我继续吗」，这一轮就结束了，这题直接 0 分。所以 headless 用一份单独的系统提示词，显式要求：不要向用户提问、不要停下来等确认、自己决策到底、完成后明确声明结束。 + +**API 错误原样打出来**：Harbor 用正则扫 agent 的 stdout/stderr 做错误分类（rate limit、usage limit、Overloaded、上下文超限、未登录、网络中断……），分类结果配合 `--max-retries 3 --retry-include ApiRateLimitError` 决定要不要重试。所以错误信息宁可原文吐出去，也不要吞掉或改写——重试的活儿因此可以少干一半。 + +**验收**：一行命令就能说明这一步是否真的通了—— + +```bash +printf "%s" "create hello.py that prints hi" | nanoPyCodeAgent; echo $? +``` + +任务从管道进去、文件真的建出来、退出码是 0，这一步就算完成。此后 nanoPyCodeAgent 才第一次具备「被脚本调用」的能力，也才谈得上接 benchmark。 + +上述完成后，后续要做的：`--output-format stream-json` 与 `--trajectory`（属最小可行路径第 4 步，等失败需要归因时再做）、API 重试与退避（第 2 步）、上下文压缩（第 5 步）、`--workdir`（调研已把它从 P0 降级：三家 benchmark 的工作目录都是靠容器默认 WORKDIR 传递的，agent 在进程当前 cwd 里干活即可）、`--timeout`（墙钟超时由 harness 侧管，Harbor 在 `task.toml` 的 `[agent].timeout_sec` 里，agent 自己的超时是保险而非接入前提）。 From 7140f3f95cfffed09a9dab81160f8ad113ae7233 Mon Sep 17 00:00:00 2001 From: minixalpha Date: Wed, 19 Aug 2026 23:04:05 +0800 Subject: [PATCH 3/6] feat: add a headless CLI for running one task and exiting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The only way in was an input() loop, so a container — where stdin is at EOF — made the agent print "Bye!" and exit having done nothing. Nothing about the harness could be measured, because nothing could drive it end to end. A task can now arrive three ways: -p/--prompt, --prompt-file, or piped on stdin (taken as the task whenever stdin is not a terminal). With no task and a terminal attached, the interactive session opens as before. The run happens in the process's working directory, and --max-turns caps how many model replies an unattended run may spend. Exit codes answer one question: did the harness fail, or did the task? Everything that is merely a bad outcome — the model gave up, the turn budget ran out, the work is half done — exits 0 and leaves the verdict to whatever scores the result, because a benchmark reads a non-zero code as a broken agent and may pay to retry the trial. Only a run that could not happen exits non-zero: 1 for missing credentials or an API that kept refusing, 2 for a misused command line. API errors are printed verbatim, since a harness classifies a failed run by pattern-matching that text. A headless run also gets its own system prompt. With no user at the other end, a clarifying question or a pause for approval ends the run with the task untouched, which scores exactly like a wrong answer. --- README.md | 20 +++ README.zh-CN.md | 18 +++ src/nanopycodeagent/__init__.py | 6 +- src/nanopycodeagent/agent.py | 249 ++++++++++++++++++++++++-------- src/nanopycodeagent/cli.py | 107 ++++++++++++++ tests/helpers.py | 14 +- tests/test_agent.py | 11 +- tests/test_cli.py | 210 +++++++++++++++++++++++++++ 8 files changed, 561 insertions(+), 74 deletions(-) create mode 100644 src/nanopycodeagent/cli.py create mode 100644 tests/test_cli.py diff --git a/README.md b/README.md index f44b895..0b6515f 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,26 @@ uv tool install nanoPyCodeAgent # or: pipx install nanoPyCodeAgent nanoPyCodeAgent ``` +#### Run one task and exit + +Give it a task and it works through it on its own, with no prompt and nothing +to confirm — the shape a script or a benchmark harness needs: + +```bash +nanoPyCodeAgent -p "add a --version flag and run the tests" +nanoPyCodeAgent --prompt-file task.md +printf "%s" "$TASK" | nanoPyCodeAgent +``` + +The task is carried out in the current directory. `--max-turns N` caps how +many model replies one run may spend (50 by default). + +A run like this exits `0` whenever the agent actually ran — including when it +gave up or ran out of turns with the task unfinished, which is for whatever +checks the result to judge. A non-zero exit means the run could not happen at +all: `1` for missing credentials or an API that kept refusing, `2` for a +misused command line. + #### Run a branch or tagged version Run an unreleased branch or a specific release tag straight from GitHub: diff --git a/README.zh-CN.md b/README.zh-CN.md index 79197f8..f263cac 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -31,6 +31,24 @@ uv tool install nanoPyCodeAgent # 或: pipx install nanoPyCodeAgent nanoPyCodeAgent ``` +#### 一次性任务(非交互) + +给它一个任务,它会自己做完退出——不给提示符,也不会停下来等确认,这正是脚本或 +benchmark harness 需要的形态: + +```bash +nanoPyCodeAgent -p "add a --version flag and run the tests" +nanoPyCodeAgent --prompt-file task.md +printf "%s" "$TASK" | nanoPyCodeAgent +``` + +任务在当前目录下执行。`--max-turns N` 限制一次运行最多花费多少轮模型回复(默认 +50 轮)。 + +只要 agent 真的跑起来了,退出码就是 `0`——包括它放弃了、或者轮数用尽而任务没做 +完,那该由检查结果的一方去判定。非零退出码表示这次运行根本没能进行:`1` 是缺少 +凭据或 API 持续失败,`2` 是命令行用错了。 + #### 运行某个分支或标签版本 直接从 GitHub 运行未发布的分支,或某个具体的发布标签: diff --git a/src/nanopycodeagent/__init__.py b/src/nanopycodeagent/__init__.py index 624ad8f..7cca11e 100644 --- a/src/nanopycodeagent/__init__.py +++ b/src/nanopycodeagent/__init__.py @@ -1,5 +1,3 @@ -from nanopycodeagent.agent import run +from nanopycodeagent.cli import main - -def main() -> None: - run() +__all__ = ["main"] diff --git a/src/nanopycodeagent/agent.py b/src/nanopycodeagent/agent.py index 528c4ca..83b8607 100644 --- a/src/nanopycodeagent/agent.py +++ b/src/nanopycodeagent/agent.py @@ -1,19 +1,26 @@ """A minimal agent loop built on the Anthropic Python SDK. -Run the program, type a message, and Agent replies. The full conversation is -kept in memory so each turn has context. The model can call a ``read`` tool -to view files, a ``write`` tool to create or overwrite them, an ``edit`` -tool to replace part of one, and a ``bash`` tool to run shell commands; -every call and its output are echoed to the terminal as they happen. Type -``/exit`` to quit. - -The loop handles only the happy path: anything unexpected — a network error, -a Ctrl-C mid-turn — crashes the session, and restarting it is the recovery. -That trade keeps the core flow readable; the hardened variant it replaced is -preserved at the ``hardened-agent-loop`` tag. +There are two ways in, and both land in the same loop. Interactively, run +the program, type a message, and Agent replies; the full conversation is +kept in memory so each turn has context, and ``/exit`` quits. Headlessly, +hand the program one task up front (see ``cli.py``) and it works that task +to completion and exits, with no prompt and nobody to ask. + +The model can call a ``read`` tool to view files, a ``write`` tool to create +or overwrite them, an ``edit`` tool to replace part of one, and a ``bash`` +tool to run shell commands; every call and its output are echoed to the +terminal as they happen. + +The interactive loop handles only the happy path: anything unexpected — a +network error, a Ctrl-C mid-turn — crashes the session, and restarting it is +the recovery. That trade keeps the core flow readable; the hardened variant +it replaced is preserved at the ``hardened-agent-loop`` tag. A headless run +has no one to restart it, so it catches API errors, reports them verbatim, +and turns them into an exit code. """ import os +import sys from importlib.metadata import PackageNotFoundError, version try: @@ -40,8 +47,17 @@ # the config file. DEFAULT_MODEL = "claude-sonnet-4-6" MAX_TOKENS = 8192 -SYSTEM_PROMPT = ( - "You are nanoPyCodeAgent, a concise and helpful coding assistant. " + +# How many model replies one headless task may spend before the run stops on +# its own. The interactive loop needs no such cap — a human watching the +# transcript can interrupt a model that keeps retrying the same command — +# but an unattended run would keep paying for that loop until the API +# refuses it. +DEFAULT_MAX_TURNS = 50 + +# Shared by both system prompts: which tool to reach for is the same question +# whoever is asking. +_TOOL_GUIDANCE = ( "Prefer the read tool for viewing files, the edit tool for changing " "part of an existing file, and the write tool for creating files or " "rewriting them whole. Use the bash tool to run commands, search with " @@ -49,6 +65,24 @@ "guessing." ) +SYSTEM_PROMPT = ( + "You are nanoPyCodeAgent, a concise and helpful coding assistant. " +) + _TOOL_GUIDANCE + +# The headless variant. None of this is a matter of tone: with no user at the +# other end, a clarifying question or a pause for approval ends the run with +# the task untouched, and a benchmark scores that exactly like a wrong answer. +HEADLESS_SYSTEM_PROMPT = ( + "You are nanoPyCodeAgent, a coding agent running non-interactively on a " + "task handed to you up front. There is no user to reply to you: never " + "ask a clarifying question, never stop to wait for confirmation, and " + "never present a plan for approval — decide on your own and carry it " + "out. Work the task through to the end, then check the result with the " + "tools instead of assuming it worked. When it is done, answer with a " + "short summary and no further tool calls: that reply is what ends the " + "run. " +) + _TOOL_GUIDANCE + # Every tool offered to the model on each request. TOOLS = [READ_TOOL, WRITE_TOOL, EDIT_TOOL, BASH_TOOL] @@ -108,28 +142,106 @@ def _run_one_tool(block: ToolUseBlock) -> ToolResultBlockParam: } -def run() -> None: - """Start the read → ask → answer loop until the user types ``/exit``. +def _create_client() -> anthropic.Anthropic | None: + """Build the SDK client, or explain on stderr why it cannot be built. - A reply may include tool calls; they are executed and their results - fed back to the model until it finishes the turn without tool use. + Any unset ``ANTHROPIC_*`` key is filled from the config file first + (environment variables take precedence), then the SDK reads credentials + from ``os.environ``. Missing credentials are a configuration failure, not + a task failure, so the explanation goes to stderr and the caller turns it + into a non-zero exit code. """ - # Fill any unset ANTHROPIC_* keys from the config file (environment - # variables take precedence), then let the SDK read credentials from - # os.environ. load_settings_env() client = anthropic.Anthropic() if client.api_key is None and client.auth_token is None: print( - "No API credentials found. Set the ANTHROPIC_API_KEY environment variable." + "No API credentials found. Set the ANTHROPIC_API_KEY environment variable.", + file=sys.stderr, ) print( "If you use a third-party / proxy service, also set ANTHROPIC_BASE_URL " - "to point at its endpoint." + "to point at its endpoint.", + file=sys.stderr, ) - return + return None + return client + + +def _resolve_model() -> str: + """The configured model, or the default when nothing usable is set.""" + return os.environ.get("ANTHROPIC_MODEL", "").strip() or DEFAULT_MODEL + + +def _run_exchange( + client: anthropic.Anthropic, + model: str, + messages: list[MessageParam], + system: str, + *, + max_turns: int | None = None, + reply_prefix: str = "\nAgent> ", +) -> bool: + """Reply to the conversation so far, running tools until the model stops. + + Appends every assistant reply and tool result to ``messages`` in place. + Returns True when the model ended a reply without asking for tools, and + False when ``max_turns`` replies were spent while it was still calling + them — the caller decides what an exhausted budget means. + """ + turns = 0 + while True: + # A spinner marks the wait for the reply; the first streamed + # token replaces it with the reply prefix. A tool-only reply + # streams no text, so the prefix is skipped for it entirely. + replied = False + # Stream the reply so text shows up as it is generated, then grab + # the accumulated message for the conversation history. + with Spinner() as spinner, client.messages.stream( + model=model, + max_tokens=MAX_TOKENS, + system=system, + tools=TOOLS, + messages=messages, + ) as stream: + for text in stream.text_stream: + if not replied: + spinner.stop() + if reply_prefix: + print(reply_prefix, end="", flush=True) + replied = True + print(text, end="", flush=True) + message = stream.get_final_message() + if replied: + print() - model = os.environ.get("ANTHROPIC_MODEL", "").strip() or DEFAULT_MODEL + turns += 1 + messages.append({"role": "assistant", "content": message.content}) + if message.stop_reason != "tool_use": + return True + if max_turns is not None and turns >= max_turns: + # Stop before running the tools: their results would only be + # useful to a reply this budget can no longer pay for. + return False + # Every tool_use block needs a matching tool_result in the next + # user message, or the API rejects the request. + results = [ + _run_one_tool(block) for block in message.content if block.type == "tool_use" + ] + messages.append({"role": "user", "content": results}) + + +def run() -> int: + """Start the read → ask → answer loop until the user types ``/exit``. + + A reply may include tool calls; they are executed and their results + fed back to the model until it finishes the turn without tool use. + Returns the process exit code. + """ + client = _create_client() + if client is None: + return 1 + + model = _resolve_model() print( f"nanoPyCodeAgent v{_package_version()} — model {model} " "(set ANTHROPIC_MODEL to override)." @@ -153,42 +265,57 @@ def run() -> None: break messages.append({"role": "user", "content": user_input}) - # The model may ask to run tools; keep streaming replies and feeding - # results back until it finishes a reply without tool calls. - while True: - # A spinner marks the wait for the reply; the first streamed - # token replaces it with the Agent> prompt. A tool-only reply - # streams no text, so the prompt is skipped for it entirely. - replied = False - # Stream the reply so text shows up as it is generated, then grab - # the accumulated message for the conversation history. - with Spinner() as spinner, client.messages.stream( - model=model, - max_tokens=MAX_TOKENS, - system=SYSTEM_PROMPT, - tools=TOOLS, - messages=messages, - ) as stream: - for text in stream.text_stream: - if not replied: - spinner.stop() - print("\nAgent> ", end="", flush=True) - replied = True - print(text, end="", flush=True) - message = stream.get_final_message() - if replied: - print() - - messages.append({"role": "assistant", "content": message.content}) - if message.stop_reason != "tool_use": - break - # Every tool_use block needs a matching tool_result in the next - # user message, or the API rejects the request. - results = [ - _run_one_tool(block) - for block in message.content - if block.type == "tool_use" - ] - messages.append({"role": "user", "content": results}) + _run_exchange(client, model, messages, SYSTEM_PROMPT) print("Bye!") + return 0 + + +def run_headless(task: str, *, max_turns: int = DEFAULT_MAX_TURNS) -> int: + """Work ``task`` to completion without a user, and return the exit code. + + The exit code answers one question — did the *harness* fail, or did the + *task*? A benchmark reads a non-zero code as "this agent broke", drops + the trial, and may pay to retry it, so everything that is merely a bad + outcome for the task (the model gave up, the turn budget ran out, the + work is half done) still exits 0 and leaves the verdict to whatever + scores the result. Only a run that could not happen at all — no + credentials, an API that keeps refusing — exits non-zero. + """ + client = _create_client() + if client is None: + return 1 + + model = _resolve_model() + # The banner goes to stderr so stdout carries the run itself: the + # model's prose and the echoed tool calls, nothing else. + print( + f"nanoPyCodeAgent v{_package_version()} — model {model}, " + f"max turns {max_turns}", + file=sys.stderr, + ) + + messages: list[MessageParam] = [{"role": "user", "content": task}] + try: + finished = _run_exchange( + client, + model, + messages, + HEADLESS_SYSTEM_PROMPT, + max_turns=max_turns, + reply_prefix="", + ) + except anthropic.APIError as exc: + # Printed verbatim on purpose: a harness classifies a failed run by + # pattern-matching this text (rate limit, overloaded, context length, + # …) to decide whether retrying is worth anything. Rewording it, or + # swallowing it, throws that away. + print(f"API error: {exc}", file=sys.stderr) + return 1 + if not finished: + turns = "turn" if max_turns == 1 else "turns" + print( + f"[stopped after {max_turns} {turns} without finishing the task]", + file=sys.stderr, + ) + return 0 diff --git a/src/nanopycodeagent/cli.py b/src/nanopycodeagent/cli.py new file mode 100644 index 0000000..1cc7582 --- /dev/null +++ b/src/nanopycodeagent/cli.py @@ -0,0 +1,107 @@ +"""The command-line entry point: an interactive session or a headless run. + +One command serves both. With a task — ``-p``, ``--prompt-file``, or piped +on stdin — the agent works that task and exits; with none, and a terminal +attached, it opens the usual prompt. + +Three ways to hand over the task look redundant until you watch a benchmark +harness start an agent: some pipe the instruction on stdin to sidestep shell +quoting and command-length limits, some pass it as an argument and wire +stdin to /dev/null, and a task read from a file is what makes a long +instruction bearable to type. Accepting all three costs a few lines and +removes a reason for the agent to be unusable in someone's harness. +""" + +import argparse +import sys +from pathlib import Path + +from .agent import DEFAULT_MAX_TURNS, _package_version, run, run_headless + +# Reserved by argparse for a misuse of the command line itself, and used here +# for the same: a task that cannot be read is a mistake in how the agent was +# invoked, not an outcome of running it. +EXIT_USAGE = 2 + + +def _build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="nanoPyCodeAgent", + description=( + "A nano code agent. Give it a task to run once and exit, or run " + "it with no task in a terminal for an interactive session." + ), + ) + task = parser.add_mutually_exclusive_group() + task.add_argument( + "-p", + "--prompt", + metavar="TEXT", + help="the task to carry out, then exit", + ) + task.add_argument( + "--prompt-file", + metavar="PATH", + help="read the task from this file, carry it out, then exit", + ) + parser.add_argument( + "--max-turns", + type=int, + default=DEFAULT_MAX_TURNS, + metavar="N", + help=( + "stop a headless run after this many model replies " + f"(default: {DEFAULT_MAX_TURNS})" + ), + ) + parser.add_argument( + "--version", + action="version", + version=f"nanoPyCodeAgent {_package_version()}", + ) + return parser + + +def _read_task(args: argparse.Namespace, parser: argparse.ArgumentParser) -> str | None: + """The task to run, or None to open an interactive session. + + Stdin counts as a task only when it is not a terminal, which is exactly + the case where something piped one in. Anything that arrives — from an + argument, a file, or a pipe — has to be a real instruction: an empty one + means the caller believes it sent a task and did not, and silently + starting a session (or a run with nothing to do) hides that. + """ + if args.prompt is not None: + task = args.prompt + elif args.prompt_file is not None: + try: + task = Path(args.prompt_file).expanduser().read_text(encoding="utf-8") + except OSError as exc: + parser.error(f"cannot read --prompt-file: {exc}") + except UnicodeDecodeError: + parser.error(f"--prompt-file is not valid UTF-8 text: {args.prompt_file}") + elif sys.stdin is not None and not sys.stdin.isatty(): + task = sys.stdin.read() + else: + return None + + task = task.strip() + if not task: + parser.error( + "the task is empty; pass it with -p/--prompt or --prompt-file, " + "or pipe it on stdin" + ) + return task + + +def main(argv: list[str] | None = None) -> int: + """Parse the command line and run, returning the process exit code.""" + parser = _build_parser() + args = parser.parse_args(argv) + if args.max_turns < 1: + parser.error("--max-turns must be at least 1") + + task = _read_task(args, parser) + if task is None: + return run() + return run_headless(task, max_turns=args.max_turns) diff --git a/tests/helpers.py b/tests/helpers.py index 246939b..5e6bade 100644 --- a/tests/helpers.py +++ b/tests/helpers.py @@ -108,16 +108,22 @@ def __init__(self, messages, *, api_key="sk-test", auth_token=None): self.auth_token = auth_token -def patch_client_and_input(monkeypatch, *, client, inputs): - """Wire up a fake Anthropic client and scripted input(). +def patch_client(monkeypatch, client): + """Wire up a fake Anthropic client in place of the real SDK one. The config file is isolated by the autouse ``_isolate_config`` fixture in - conftest.py, so ``run()`` sees no config file unless a test writes one to + conftest.py, so the agent sees no config file unless a test writes one to ``settings.SETTINGS_PATH``. + """ + monkeypatch.setattr(anthropic, "Anthropic", lambda *a, **k: client) + + +def patch_client_and_input(monkeypatch, *, client, inputs): + """Wire up a fake Anthropic client and scripted input(). Returns the list that records the prompt of each ``input()`` call. """ - monkeypatch.setattr(anthropic, "Anthropic", lambda *a, **k: client) + patch_client(monkeypatch, client) answers = iter(inputs) prompts = [] diff --git a/tests/test_agent.py b/tests/test_agent.py index 092cc6c..222add9 100644 --- a/tests/test_agent.py +++ b/tests/test_agent.py @@ -27,13 +27,14 @@ def test_missing_credentials_exits_early(monkeypatch, capsys): client = FakeClient(messages, api_key=None, auth_token=None) patch_client_and_input(monkeypatch, client=client, inputs=[]) - agent.run() + assert agent.run() == 1 # a configuration failure, not a task failure - out = capsys.readouterr().out - assert "No API credentials found." in out + captured = capsys.readouterr() + # The explanation goes to stderr, keeping stdout to the run itself. + assert "No API credentials found." in captured.err # Third-party / proxy users are told how to point the SDK at their endpoint. - assert "ANTHROPIC_BASE_URL" in out - assert "Bye!" not in out # returned before entering the loop + assert "ANTHROPIC_BASE_URL" in captured.err + assert "Bye!" not in captured.out # returned before entering the loop assert messages.calls == [] diff --git a/tests/test_cli.py b/tests/test_cli.py new file mode 100644 index 0000000..fe318f5 --- /dev/null +++ b/tests/test_cli.py @@ -0,0 +1,210 @@ +"""Tests for the command line: how a task gets in, and what comes back out. + +Every Anthropic API access is mocked, so these tests never hit the network and +never spend tokens. The exit codes are the point of most of them: a harness +reads a non-zero code as "this agent broke" and may pay to retry the trial, so +a task that merely went badly has to come back as 0. +""" + +import io + +import anthropic +import httpx +import pytest + +from nanopycodeagent import agent, cli + +from helpers import ( + FakeClient, + FakeMessages, + FakeStream, + patch_client, + patch_client_and_input, + text_block, + tool_use_block, + write_tool_use_block, +) + + +class TtyStdin(io.StringIO): + """A stdin that claims to be a terminal, i.e. nothing was piped in.""" + + def isatty(self): + return True + + +def test_package_entry_point_is_the_cli(): + # pyproject points the console script at nanopycodeagent:main, and the + # wrapper it generates exits with whatever that returns. + from nanopycodeagent import main + + assert main is cli.main + + +def test_prompt_argument_runs_the_task_and_exits_zero(monkeypatch, capsys): + messages = FakeMessages([[text_block("done")]]) + patch_client(monkeypatch, FakeClient(messages)) + + assert cli.main(["-p", "say hi"]) == 0 + + assert messages.calls[0] == [{"role": "user", "content": "say hi"}] + # The headless prompt goes out, not the conversational one. + assert messages.kwargs[0]["system"] == agent.HEADLESS_SYSTEM_PROMPT + out = capsys.readouterr().out + assert "done" in out + # stdout is the run itself: no chat prompts wrapped around the reply. + assert "Agent>" not in out + assert "You>" not in out + + +def test_stdin_pipe_is_taken_as_the_task(monkeypatch, capsys): + # printf "%s" "$TASK" | nanoPyCodeAgent — how a harness avoids shell + # quoting and command-length limits. + monkeypatch.setattr(cli.sys, "stdin", io.StringIO("fix the bug\n")) + messages = FakeMessages([[text_block("fixed")]]) + patch_client(monkeypatch, FakeClient(messages)) + + assert cli.main([]) == 0 + + assert messages.calls[0] == [{"role": "user", "content": "fix the bug"}] + + +def test_prompt_file_is_read_as_the_task(monkeypatch, tmp_path, capsys): + task_file = tmp_path / "instruction.md" + task_file.write_text("port the parser\n", encoding="utf-8") + messages = FakeMessages([[text_block("ported")]]) + patch_client(monkeypatch, FakeClient(messages)) + + assert cli.main(["--prompt-file", str(task_file)]) == 0 + + assert messages.calls[0] == [{"role": "user", "content": "port the parser"}] + + +def test_banner_stays_off_stdout_in_a_headless_run(monkeypatch, capsys): + messages = FakeMessages([[text_block("done")]]) + patch_client(monkeypatch, FakeClient(messages)) + + cli.main(["-p", "say hi"]) + + captured = capsys.readouterr() + assert "nanoPyCodeAgent v" not in captured.out # stdout carries the run + assert "nanoPyCodeAgent v" in captured.err # the banner still lands in logs + + +def test_no_task_on_a_terminal_starts_an_interactive_session(monkeypatch, capsys): + monkeypatch.setattr(cli.sys, "stdin", TtyStdin()) + messages = FakeMessages([[text_block("hello there")]]) + client = FakeClient(messages) + patch_client_and_input(monkeypatch, client=client, inputs=["hi", "/exit"]) + + assert cli.main([]) == 0 + + out = capsys.readouterr().out + assert "Bye!" in out + # The conversational prompt is the one an interactive session sends. + assert messages.kwargs[0]["system"] == agent.SYSTEM_PROMPT + + +def test_missing_credentials_exit_non_zero(monkeypatch, capsys): + messages = FakeMessages([]) + patch_client(monkeypatch, FakeClient(messages, api_key=None, auth_token=None)) + + # A run that never happened is the harness's problem, not the task's. + assert cli.main(["-p", "say hi"]) == 1 + + assert "No API credentials found." in capsys.readouterr().err + assert messages.calls == [] + + +def test_exhausted_turn_budget_still_exits_zero(monkeypatch, capsys, tmp_path): + # Two replies are all the budget allows; the second one's tool call is + # never run, because no reply is left to read its result. + marker = tmp_path / "second_turn.txt" + first = FakeStream([tool_use_block("tu_1", "echo one")], stop_reason="tool_use") + second = FakeStream( + [write_tool_use_block("tu_2", path=str(marker), content="x")], + stop_reason="tool_use", + ) + messages = FakeMessages([first, second]) + patch_client(monkeypatch, FakeClient(messages)) + + assert cli.main(["-p", "loop forever", "--max-turns", "2"]) == 0 + + assert len(messages.calls) == 2 + assert not marker.exists() + captured = capsys.readouterr() + assert "[bash]$ echo one" in captured.out # tool activity lands in the log + assert "stopped after 2 turns" in captured.err + + +def test_api_error_is_reported_verbatim_and_exits_non_zero(monkeypatch, capsys): + # A harness classifies a failed run by pattern-matching this text, so it + # goes out unedited. + class ExplodingMessages: + calls: list = [] + + def stream(self, **kwargs): + raise anthropic.APIConnectionError( + message="Overloaded (529)", + request=httpx.Request("POST", "https://api.anthropic.com/v1/messages"), + ) + + patch_client(monkeypatch, FakeClient(ExplodingMessages())) + + assert cli.main(["-p", "say hi"]) == 1 + + assert "Overloaded (529)" in capsys.readouterr().err + + +def test_empty_task_is_a_usage_error(monkeypatch): + patch_client(monkeypatch, FakeClient(FakeMessages([]))) + + with pytest.raises(SystemExit) as excinfo: + cli.main(["-p", " "]) + + assert excinfo.value.code == cli.EXIT_USAGE + + +def test_empty_stdin_is_a_usage_error(monkeypatch): + # The container case: stdin is redirected but carries nothing. Starting a + # session here is what used to make the agent print "Bye!" and do nothing. + monkeypatch.setattr(cli.sys, "stdin", io.StringIO("")) + patch_client(monkeypatch, FakeClient(FakeMessages([]))) + + with pytest.raises(SystemExit) as excinfo: + cli.main([]) + + assert excinfo.value.code == cli.EXIT_USAGE + + +def test_unreadable_prompt_file_is_a_usage_error(monkeypatch, tmp_path, capsys): + with pytest.raises(SystemExit) as excinfo: + cli.main(["--prompt-file", str(tmp_path / "missing.md")]) + + assert excinfo.value.code == cli.EXIT_USAGE + assert "cannot read --prompt-file" in capsys.readouterr().err + + +def test_prompt_and_prompt_file_are_mutually_exclusive(tmp_path): + with pytest.raises(SystemExit) as excinfo: + cli.main(["-p", "a task", "--prompt-file", str(tmp_path / "task.md")]) + + assert excinfo.value.code == cli.EXIT_USAGE + + +def test_max_turns_must_be_positive(): + with pytest.raises(SystemExit) as excinfo: + cli.main(["-p", "say hi", "--max-turns", "0"]) + + assert excinfo.value.code == cli.EXIT_USAGE + + +def test_version_flag_prints_the_version(capsys): + # Harbor probes an installed agent's version with its own --version call. + with pytest.raises(SystemExit) as excinfo: + cli.main(["--version"]) + + assert excinfo.value.code == 0 + out = capsys.readouterr().out + assert f"nanoPyCodeAgent {agent._package_version()}" in out + assert "unknown" not in out # metadata is present in the test environment From 0fab0aca746dd9a0dca305d57602bfb32b0b5965 Mon Sep 17 00:00:00 2001 From: minixalpha Date: Thu, 20 Aug 2026 22:29:32 +0800 Subject: [PATCH 4/6] fix(cli): handle streaming transport errors --- src/nanopycodeagent/agent.py | 3 ++- tests/test_cli.py | 27 +++++++++++++++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/src/nanopycodeagent/agent.py b/src/nanopycodeagent/agent.py index 83b8607..d7d772d 100644 --- a/src/nanopycodeagent/agent.py +++ b/src/nanopycodeagent/agent.py @@ -34,6 +34,7 @@ pass import anthropic +import httpx from anthropic.types import MessageParam, ToolResultBlockParam, ToolUseBlock from .bash_tool import BASH_TOOL, run_bash @@ -305,7 +306,7 @@ def run_headless(task: str, *, max_turns: int = DEFAULT_MAX_TURNS) -> int: max_turns=max_turns, reply_prefix="", ) - except anthropic.APIError as exc: + except (anthropic.APIError, httpx.HTTPError) as exc: # Printed verbatim on purpose: a harness classifies a failed run by # pattern-matching this text (rate limit, overloaded, context length, # …) to decide whether retrying is worth anything. Rewording it, or diff --git a/tests/test_cli.py b/tests/test_cli.py index fe318f5..7e7734a 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -156,6 +156,33 @@ def stream(self, **kwargs): assert "Overloaded (529)" in capsys.readouterr().err +def test_stream_transport_error_is_reported_verbatim_and_exits_non_zero( + monkeypatch, capsys +): + class DisconnectingStream(FakeStream): + @property + def text_stream(self): + def _gen(): + yield "partial reply" + raise httpx.ReadError( + "peer disconnected", + request=httpx.Request( + "POST", "https://api.anthropic.com/v1/messages" + ), + ) + + return _gen() + + messages = FakeMessages([DisconnectingStream([])]) + patch_client(monkeypatch, FakeClient(messages)) + + assert cli.main(["-p", "say hi"]) == 1 + + captured = capsys.readouterr() + assert "partial reply" in captured.out + assert "API error: peer disconnected" in captured.err + + def test_empty_task_is_a_usage_error(monkeypatch): patch_client(monkeypatch, FakeClient(FakeMessages([]))) From 9b115a02ce31066583720b24c3f27cc943af552c Mon Sep 17 00:00:00 2001 From: minixalpha Date: Thu, 20 Aug 2026 22:48:03 +0800 Subject: [PATCH 5/6] docs: refine headless CLI development notes --- docs/dev_notes/zh-CN/0.8.x.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/dev_notes/zh-CN/0.8.x.md b/docs/dev_notes/zh-CN/0.8.x.md index b33f926..456083e 100644 --- a/docs/dev_notes/zh-CN/0.8.x.md +++ b/docs/dev_notes/zh-CN/0.8.x.md @@ -12,7 +12,7 @@ ### headless CLI 实现契约 -**命令行形态**（跟着调研的 §5 走）： +**命令行形态**： ```text nanoPyCodeAgent [-p/--prompt "<任务>" | --prompt-file | (stdin 管道)] @@ -22,16 +22,16 @@ nanoPyCodeAgent [-p/--prompt "<任务>" | --prompt-file | (stdin 管道)] `--version` 不是凑数的：Harbor 会用适配类的 `get_version_command()` / `parse_version()` 去探测并记录 agent 版本（best-effort，失败不报错）。`_package_version()` 已经写好了，只差挂到 CLI 上。 -**headless 判定**：给了 `-p` 或 `--prompt-file` 就是 headless；两者都没给时，若 `sys.stdin.isatty()` 为 False，就把整段 stdin 读进来当任务；只有连着 tty 且没给任务，才进现有的 REPL。三种投递方式都要吃下，不是为了形式完整——Harbor 的两个官方范例正好各占一种：Claude Code 用 `printf … | claude --print` 走 stdin 管道，避开 shell 转义和命令行长度限制；mini-swe-agent 用 `--task=` 走参数，并显式把 stdin 接到 `/dev/null`。顺带也就修掉了「容器里 stdin 是 EOF 就立刻 `Bye!` 退出」这个行为。 +**headless 判定**：给了 `-p` 或 `--prompt-file` 就是 headless；两者都没给时，若 `sys.stdin.isatty()` 为 False，就把整段 stdin 读进来当任务；只有连着 tty 且没给任务，才进现有的 REPL。三种方式都要实现——Harbor 的两个官方范例正好各占一种：Claude Code 用 `printf … | claude --print` 走 stdin 管道，避开 shell 转义和命令行长度限制；mini-swe-agent 用 `--task=` 走参数，并显式把 stdin 接到 `/dev/null`。顺带也就修掉了「容器里 stdin 是 EOF 就立刻 `Bye!` 退出」这个行为。 -**退出码**：判据只有一条——**失败的是任务，还是 harness**。 +**退出码**： | 退出码 | 场景 | | :-: | --- | | 0 | 模型声明完成；轮数用尽；任务没做成 | | 非 0 | 无 API 凭证；参数错误 | -「没做完也 exit 0」是最反直觉、也最容易做反的一条：Harbor 把 agent 命令包在 `set -o pipefail` 下执行，非零退出码直接抛 `NonZeroAgentExitCodeError`，整个 trial 判成 agent 失败，还可能触发重试白烧钱。任务做没做成该由 verifier 从 `/logs/verifier/` 的 reward 文件判，不该由 agent 的退出码判。 +「没做完也 exit 0」比较反直觉：Harbor 把 agent 命令包在 `set -o pipefail` 下执行，非零退出码直接抛 `NonZeroAgentExitCodeError`，整个 trial 判成 agent 失败，还可能触发重试白烧钱。任务做没做成该由 verifier 从 `/logs/verifier/` 的 reward 文件判，不该由 agent 的退出码判。 这里还牵出一个现存 bug：没有 API 凭证时，`run()` 是 `print` 完直接 `return`，`main()` 又没有返回码的概念，最终退出码是 0——在 harness 眼里这等于「跑完了，只是没做出来」，一整批任务会静悄悄地全判 0 分，而且看不出是配置问题。所以 `main()` 要改成返回 int（console script 会拿返回值当退出码）。 @@ -47,6 +47,6 @@ nanoPyCodeAgent [-p/--prompt "<任务>" | --prompt-file | (stdin 管道)] printf "%s" "create hello.py that prints hi" | nanoPyCodeAgent; echo $? ``` -任务从管道进去、文件真的建出来、退出码是 0，这一步就算完成。此后 nanoPyCodeAgent 才第一次具备「被脚本调用」的能力，也才谈得上接 benchmark。 +任务从管道输入、创建文件、退出码是 0，这一步就算完成。此后 nanoPyCodeAgent 才第一次具备「被脚本调用」的能力，也才谈得上接 benchmark。 上述完成后，后续要做的：`--output-format stream-json` 与 `--trajectory`（属最小可行路径第 4 步，等失败需要归因时再做）、API 重试与退避（第 2 步）、上下文压缩（第 5 步）、`--workdir`（调研已把它从 P0 降级：三家 benchmark 的工作目录都是靠容器默认 WORKDIR 传递的，agent 在进程当前 cwd 里干活即可）、`--timeout`（墙钟超时由 harness 侧管，Harbor 在 `task.toml` 的 `[agent].timeout_sec` 里，agent 自己的超时是保险而非接入前提）。 From 558366f79b2bfb34309d9390c4c80cf9661cc5f6 Mon Sep 17 00:00:00 2001 From: minixalpha Date: Thu, 20 Aug 2026 22:55:27 +0800 Subject: [PATCH 6/6] docs: sync changelog and dev notes for headless CLI --- docs/changelogs/0.8.x.md | 5 ++++ docs/dev_notes/en/0.8.x.md | 52 +++++++++++++++++++++++++++++++++++ docs/dev_notes/zh-CN/0.8.x.md | 2 +- 3 files changed, 58 insertions(+), 1 deletion(-) create mode 100644 docs/dev_notes/en/0.8.x.md diff --git a/docs/changelogs/0.8.x.md b/docs/changelogs/0.8.x.md index 0249eb3..b11c1b5 100644 --- a/docs/changelogs/0.8.x.md +++ b/docs/changelogs/0.8.x.md @@ -16,6 +16,11 @@ All notable changes in the **0.8.x** release series are documented here. three. Notably, a benchmark harness reads a non-zero exit code as its own failure, so an agent that runs out of turns without solving the task must still exit 0. +- A headless CLI for running one task and exiting via `-p` / `--prompt`, + `--prompt-file`, or piped stdin. Headless runs use a dedicated + non-interactive system prompt, support a configurable `--max-turns` budget + and `--version`, keep the run itself on stdout, return benchmark-safe exit + codes, and preserve API and transport error text on stderr.