diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 0000000..0a294c6 --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,45 @@ +# Code Agent Runtime + +This glossary defines the runtime artifacts produced and maintained by a code agent so that public output, diagnostics, evaluation data, and resumable state remain distinct. + +## Language + +**Agent Run**: +A bounded execution that starts with an instruction and ends when the agent stops autonomous work. +_Avoid_: Session, turn + +**Run Output**: +The public result or event protocol that one agent run emits to its caller. +_Avoid_: Trace, transcript + +**Run Result Object**: +A single public object that summarizes how one Agent Run ended, including its final result and optional usage metadata. +_Avoid_: Trajectory, session export + +**Public Event Stream**: +A caller-facing Run Output that exposes a stable, selected sequence of Agent Run events as they occur. +_Avoid_: Internal event bus, execution trace + +**Partial Content**: +An unfinished representation of a message, reasoning block, or tool input that is exposed before the logical content is complete. +_Avoid_: Final message, completed event + +**Content Delta**: +The incremental fragment or change relative to content already emitted for the same logical item. +_Avoid_: Partial snapshot, final message + +**Terminal Event**: +The final Public Event Stream record that explicitly states how an Agent Run ended. +_Avoid_: EOF, last message + +**Execution Trace**: +A diagnostic record of runtime activity retained to explain failures, timing, and internal behavior. +_Avoid_: Run output, trajectory + +**Trajectory**: +A task-scoped record of observations, actions, results, and outcome prepared for evaluation or offline analysis. +_Avoid_: Execution trace, session, transcript + +**Session**: +Durable agent state that can span multiple runs and supports continuing or branching prior work. +_Avoid_: Agent run, trajectory, transcript diff --git a/docs/research/en/agent_output_and_trajectory.md b/docs/research/en/agent_output_and_trajectory.md index 51ec852..81954c4 100644 --- a/docs/research/en/agent_output_and_trajectory.md +++ b/docs/research/en/agent_output_and_trajectory.md @@ -1,99 +1,264 @@ -# Output Formats and Trajectory Design in Mainstream Code Agents +# Run Output, Execution Traces, Task Trajectories, and Session Design in Mainstream Code Agents > Generated from the Chinese source [`../zh-CN/agent_output_and_trajectory.md`](../zh-CN/agent_output_and_trajectory.md). Do not edit by hand. Surveyed on 2026-08-22. -[`benchmark_headless_interface.md`](benchmark_headless_interface.md) previously proposed the following interface, but did not fully define the wire protocol for each parameter: +[`benchmark_headless_interface.md`](benchmark_headless_interface.md) previously proposed the following interface, but did not explain whether the two parameters control the same kind of artifact: ```text [--output-format text|stream-json] [--trajectory ] ``` -This document works backward from the current implementations of Pi, Claude Code, Codex, OpenCode, and Grok Build to determine how these concepts should be separated. The conclusions come first: +Both parameters appear to concern “output,” but they actually belong to different planes. Before comparing Pi, Claude Code, Codex, OpenCode, and Grok Build, four questions must be answered: -1. **`--output-format` selects the stdout representation; it is not file redirection.** Writing to a file remains the responsibility of the Shell's `>` operator or another explicit file parameter. -2. **For nanoPyCodeAgent, the most natural semantics for `--trajectory PATH` are “enable trajectory recording and write it to PATH.”** It changes neither stdout nor `--output-format`. -3. **`stream-json` should be defined as an NDJSON/JSONL event stream: every line is a complete, independently parseable JSON object.** It is not “one JSON document split into chunks,” nor does it inherently promise token-level increments. -4. **The name `json` has no uniform industry meaning.** Claude Code and Grok use it to mean “emit one object at the end,” while Pi, Codex, and OpenCode use it to mean a JSONL event stream. Interface documentation must therefore specify the wire protocol rather than merely list enum names. -5. **The stdout event stream, persistent session, debugging trace, benchmark trajectory, and telemetry are five distinct artifacts.** They can originate from the same internal event model, but should not share one ambiguous switch. +1. What does this agent invocation deliver to a person or calling program? +2. What actually happened inside the agent? +3. How should a benchmark record the path the agent took to complete the task? +4. What state will the agent use to resume context next time? + +These questions correspond to **run output, execution trace, trajectory, and session**, respectively. The document first distinguishes the four using one example run, then examines each project's implementation. --- -## 1. Research Scope and Evidence Levels +## 1. Four Artifacts from One Run -Before beginning the survey, we attempted to update the code under `references/`. The four accessible repositories were fast-forwarded to their latest remote commits. The remote for the third-party Claude Code mirror was no longer accessible, so it was not forcibly replaced. +Suppose the user asks the agent to: -| Project | Local revision | Update result | Evidence level | -| --- | --- | --- | --- | -| Grok Build | `19d42e35c07a9c9244f03f6df0c4c353f970d4f9` | Updated | Official xAI open-source repository | -| Pi | `c49906ec77788625aacbdc53ebca6fbe65bd20f5` | Updated | Public repository tracked by `references/pi` | -| OpenCode | `e00890c67261a435cee6409366a68999a93393fd` | Updated | Official OpenCode open-source repository | -| Codex | `4f39251a010a8bd7d692d25fb33832ff06f1635a` | Updated | Official OpenAI open-source repository | -| Claude Code | `a371abbe75ffa0d0a3c92290e2bbf56a7ef54367` | Remote returned `Repository not found`; snapshot retained | **Unofficial sourcemap mirror, used only to corroborate implementation ideas** | +```text +Fix the exception parser.py raises when tool arguments are empty, and run the relevant tests. +``` -The authoritative Claude Code contract is the current Anthropic [CLI reference](https://code.claude.com/docs/en/cli-usage), [headless documentation](https://code.claude.com/docs/en/headless), and [sessions documentation](https://code.claude.com/docs/en/sessions). The local `references/claude-code/README.md` also explicitly states that it is not an official Anthropic project, so this document does not treat internal fields from that snapshot as a current stable API. +A typical execution might read the failing code, search for callers, modify the code, run tests, and finally answer the user. This document calls the bounded period from accepting that prompt until the agent stops autonomous work a **run**. One run may contain multiple model requests and multiple tool calls. + +### 1.1 Run Output: Public Output Delivered to the Caller + +Run output answers: **“What should this invocation expose externally?”** The caller may be a person at a terminal, a Shell script, a benchmark harness, an SDK, or a UI. + +The same result can have different representations. For example, `text` mode may emit only the final answer: + +```text +Fixed empty tool-argument parsing and added a regression test. All 12 relevant tests pass. +``` + +At the end of the run, `json` mode may deliver a machine-readable **single result object**. It summarizes **how this run ultimately ended**, such as its final status, answer, stop reason, usage, and cost. It does not summarize the complete execution process or pack the trace, trajectory, or session into one JSON object: + +```json +{"status":"completed","result":"Fixed empty tool-argument parsing and added a regression test. All 12 relevant tests pass.","usage":{"input_tokens":1200,"output_tokens":180}} +``` + +During the run, `stream-json` mode may publish a stable public event protocol. The example below uses **JSON Lines (JSONL)**, where every non-empty line is one complete JSON object. This format is also commonly called **Newline-Delimited JSON (NDJSON)**; `ND` means `Newline-Delimited`: + +```jsonl +{"type":"tool.completed","tool":"pytest","is_error":false,"result":"12 passed"} +{"type":"assistant.message","content":"Fixed empty tool-argument parsing and added a regression test. All 12 relevant tests pass."} +{"type":"run.completed","status":"completed"} +``` + +Four output-protocol terms recur throughout the rest of this document: + +- **Single result object:** one JSON object emitted after the run ends, summarizing the run's final result. Fields vary by product, but commonly include the final answer, completed/failed status, stop reason, run/session ID, turn count, token usage, and cost. It usually does not contain step-by-step execution records. +- **Public event stream:** a stable event protocol that the agent delivers to external callers in occurrence order during a run. It exposes only selected events suitable for long-term compatibility, such as tool starts/ends, complete assistant messages, and run termination. It is not a raw dump of the internal event bus or execution trace. Every CLI event stream examined here uses JSONL/NDJSON: one complete event object per line. +- **Partial and delta:** a `partial` is an intermediate, unfinished state of a message, thinking block, or tool arguments; a `delta` is the small fragment newly added or changed relative to previously emitted content. A protocol may repeatedly emit cumulative partial snapshots, or emit only deltas for the consumer to assemble. The table's “partial / delta capability” asks one question: **does the public event stream expose fragments of logical content before that content is complete?** +- **Terminal event:** a public-stream event that explicitly declares that the run ended with a state such as completed or failed, for example `run.completed` or `result`. It conveys more than EOF: EOF says only that stdout closed, which may result from normal exit, a crash, interruption, or truncation. + +Having a public event stream and having partial/delta output are therefore independent capabilities. A stream that emits `tool.completed` only after pytest finishes and then emits a complete `assistant.message` is still real-time, but it has no partial/delta capability. If answer generation instead emits `assistant.delta: "Fixed"` and then `assistant.delta: " successfully"`, it does expose content incrementally. With or without deltas, a normal shutdown can still use a terminal event to state that the entire run has ended. + +This document calls stdout's encoding and record boundaries its **transport format (wire format)**: whether stdout contains text, one JSON object, or one JSON object per line, and how records are separated. When the discussion also covers event types, ordering, termination, and error semantics, the document uses **output protocol**. + +Even when a public stream includes `tool.completed`, it remains **run output**, because it is an intentionally supported public contract for callers rather than a raw internal trace. **`--output-format` selects how this public output is encoded and framed.** It does not enable traces, save trajectories, or persist sessions. + +### 1.2 Execution Trace: Runtime Evidence for Troubleshooting + +An execution trace answers: **“What actually happened inside the agent?”** It is intended for agent developers and observability systems. Typical contents include: + +- Provider request and response metadata, retries, and backoff. +- Model calls, tool scheduling, subprocesses, concurrent tasks, and timing. +- Complete or redacted tool input/output, stderr, and exception stacks. +- Parent-child span relationships, internal state transitions, and performance data. + +A debugging trace might contain facts like these: + +```text +inference attempt=1 status=429 retry_after_ms=800 +inference attempt=2 request_id=req_2 latency_ms=1430 +tool call_id=t1 process_id=4312 stdout_bytes=824 exit_code=0 +``` + +This information can explain latency or failure, but selecting `--output-format stream-json` should not cause all of it to enter stdout. A trace is usually more detailed, more sensitive, and more closely coupled to the current implementation. Its schema generally does not carry the same public compatibility promise as run output. + +Logs, metrics, and OpenTelemetry are ways to record or transport diagnostic signals, not additional categories of “user output” parallel to traces. An OpenTelemetry trace is itself one representation of an execution trace. + +### 1.3 Trajectory: The Execution Path Retained for Task Evaluation + +A trajectory answers: **“Through which observations and actions did the agent obtain this task result?”** It is usually scoped to one benchmark trial or task run and consumed by evaluation frameworks and offline analyzers. + +The same run might be organized into this trajectory: + +```jsonl +{"step":1,"observation":"parser.py dereferences arguments when they are empty","action":{"tool":"search","query":"parse tool arguments"}} +{"step":2,"observation":"found two callers and one missing null branch","action":{"tool":"edit","file":"parser.py"},"result":"added empty-argument handling"} +{"step":3,"observation":"code updated","action":{"tool":"pytest","target":"tests/test_parser.py"},"result":"12 passed"} +{"outcome":"completed","result":"bug fixed","usage":{"input_tokens":1200,"output_tokens":180}} +``` + +Trajectories and traces both describe execution, but make different trade-offs: + +- A trace stays close to runtime implementation and aims to reconstruct a failure scene. It may include every retry, internal queue, and raw payload. +- A trajectory stays close to task semantics and supports comparison and attribution. It retains analytical fields such as observation, action, tool result, outcome, tokens, and cost. +- A trajectory can be derived from a public event stream, trace, or session, but the converted artifact is the trajectory. Its source data does not change category merely because a trajectory was derived from it. + +A trajectory is usually fixed after the run ends. It may support visualization or offline replay, but **replay is not resume**: step records alone do not mean the agent can reconstruct the product state and continue the conversation. + +Nor can an entire session export simply be renamed a trajectory. A session may contain multiple runs, earlier tasks, branches, and compaction metadata. An adapter must first isolate the current task/trial boundary and then organize it into observations, actions, and outcomes. + +### 1.4 Session: Product State Persisted for Continued Work + +A session answers: **“From what state should the next invocation continue?”** It usually outlives a single run and may support continue, resume, fork, rewind, or compaction. + +A session may retain: + +- Session ID, working directory, model, and tool configuration. +- User messages, assistant messages, tool calls, and results. +- Stable entry/message IDs and parent relationships. +- Compaction checkpoints, branches, permission decisions, and other recovery state. + +For example, after ending one process the user might run: + +```text +agent --resume s1 -p "Also handle the case where arguments is missing." +``` + +The agent must reconstruct context from session `s1`. This is the session's defining capability and the boundary between a session and a trajectory: **whether the product can reliably continue, resume, or fork matters more than whether a file is named transcript, history, rollout, or JSONL.** + +Codex is the clearest example of how names can mislead. Its persistent session files are named `rollout-*.jsonl`, but `resume` continues them, `fork` derives new sessions from them, and `--ephemeral` disables them. This document therefore classifies a Codex rollout as a **session store**. The separate opt-in `rollout-trace` is the execution trace used for troubleshooting; the two are not the same file. --- -## 2. First Separate Five Easily Confused Concepts +## 2. Classify by Lifecycle and Purpose, Not by Filename + +The minimum boundaries among the four concepts are: -A headless agent commonly needs all five output categories below. Their data overlaps, but their lifecycles, audiences, and compatibility promises differ. +| Concept | Core question | Typical lifecycle | Primary consumer | Typical contents | Used for resume | Control plane | +| --- | --- | --- | --- | --- | :-: | --- | +| Run output | What does this invocation deliver externally? | One run | People, scripts, runners, SDKs, UIs | Final answer, public events, status, usage | No | `--output-format` | +| Execution trace | What happened internally at runtime? | One run, process, or trace tree | Developers, observability platforms | Requests/responses, retries, spans, internal tools, exceptions | No | Debug / trace / telemetry configuration | +| Trajectory | How did the agent complete this task? | One task / trial / run | Benchmarks, offline analyzers | Observations, actions, tool results, outcome, cost | Usually no | `--trajectory` or an adapter | +| Session | From what state should the next invocation continue? | Multiple runs | The agent product itself | Recoverable transcript, stable IDs, branches, compaction, configuration | Yes | Session / resume / persistence configuration | -| Plane | Primary consumer | Typical medium | Primary purpose | Must support session recovery | -| --- | --- | --- | --- | :-: | -| CLI presentation | Human or one-off script | stdout text / single JSON object | Report the result of this run | No | -| Live event protocol | Runner, SDK, UI | stdout NDJSON | Observe tool calls, messages, and usage in real time | No | -| Session store | The agent itself | JSONL, SQLite, multi-file directory | continue / resume / fork / compaction | Yes | -| Benchmark trajectory | Harbor, offline analyzer | JSONL, ATIF, etc. | Count steps, tokens, and costs, and attribute failures | Usually no | -| Diagnostics / telemetry | Developer, observability platform | stderr, logs, spans, trace files | Troubleshooting, performance analysis, operational monitoring | No | +One session can contain multiple runs. Each run produces its own output and may optionally record a trace and trajectory: -This distinction explains an apparently contradictory fact: **an agent can emit live events to stdout with `stream-json` while simultaneously writing a separate, more complete and recoverable session into its own data directory.** The former is the protocol for this subprocess invocation; the latter is product state. +```text +session s1 +├─ run r1: initial fix task +│ ├─ output o1 ───────────────> current caller +│ ├─ execution trace t1 ──────> debugger / tracing backend +│ └─ trajectory j1 ───────────> benchmark artifact +└─ run r2: follow-up after resume + ├─ output o2 + ├─ execution trace t2 + └─ trajectory j2 +``` -The recommended internal structure is one event source feeding multiple projectors: +Their data overlaps, but their compatibility promises differ. A reasonable implementation can project four artifacts from the same set of internal events. Here, “project” means selecting fields and transforming structure for each purpose, not copying every internal event: ```text ┌─ text renderer ───────────────> stdout agent loop ─> canonical ├─ final JSON reducer ──────────> stdout - events ├─ NDJSON event serializer ────> stdout - ├─ trajectory writer ───────────> requested file - ├─ session recorder ────────────> session store - └─ diagnostics / telemetry ─────> stderr / exporter + events ├─ JSONL event serializer ─────> stdout + ├─ trace recorder ──────────────> debug bundle / OTel + ├─ trajectory projector ────────> requested artifact + └─ session recorder ────────────> session store ``` -`--output-format` selects only one of the first three stdout projectors; `--trajectory` controls the fourth sink; if resume support is added later, the fifth session recorder should be designed separately. This prevents one parameter from simultaneously carrying three responsibilities: format, enablement, and path. +The first three branches may emit different numbers of records, but all are run output. `--output-format` may select among only those three; it cannot also change trace, trajectory, or session persistence. + +--- + +## 3. Research Scope and Core Conclusions + +### 3.1 Evidence Scope + +Before beginning the survey, we attempted to update the code under `references/`. The four accessible repositories were fast-forwarded to their latest remote commits. The remote for the third-party Claude Code mirror was no longer accessible, so it was not forcibly replaced. + +| Project | Local revision | Update result | Evidence level | +| --- | --- | --- | --- | +| Grok Build | `19d42e35c07a9c9244f03f6df0c4c353f970d4f9` | Updated | Official xAI open-source repository | +| Pi | `c49906ec77788625aacbdc53ebca6fbe65bd20f5` | Updated | Public repository tracked by `references/pi` | +| OpenCode | `e00890c67261a435cee6409366a68999a93393fd` | Updated | Official OpenCode open-source repository | +| Codex | `4f39251a010a8bd7d692d25fb33832ff06f1635a` | Updated | Official OpenAI open-source repository | +| Claude Code | `a371abbe75ffa0d0a3c92290e2bbf56a7ef54367` | Remote returned `Repository not found`; snapshot retained | **Unofficial sourcemap mirror, used only to corroborate implementation ideas** | + +The authoritative Claude Code contract is the current Anthropic [CLI reference](https://code.claude.com/docs/en/cli-usage), [headless documentation](https://code.claude.com/docs/en/headless), and [sessions documentation](https://code.claude.com/docs/en/sessions). The local `references/claude-code/README.md` also explicitly states that it is not an official Anthropic project, so this document does not treat internal fields from that snapshot as a current stable API. + +### 3.2 Core Conclusions + +1. **`--output-format` selects only the stdout representation of public run output.** It is not file redirection and does not control execution traces, trajectories, or sessions. +2. **`text`, `json`, and `stream-json` describe three stdout transport formats.** `text` is final text, `json` is one result object at the end, and `stream-json` is a JSONL/NDJSON event sequence published line by line during the run. +3. **`--trajectory PATH` should be an independent control plane.** When present, it creates a trajectory scoped to the current task run; `PATH` selects only the artifact location and does not change stdout. +4. **A trajectory is not a simplified session.** A trajectory explains a task path for evaluation; a session restores product state. Resume requires a separate design for stable IDs, branches, compaction, and schema migration. +5. **All five surveyed products implement sessions, but none provides a benchmark `--trajectory PATH` exactly equivalent to the interface proposed here.** A benchmark can derive a trajectory from a public event stream or session, but that does not make the source artifact a trajectory. +6. **The name `json` has no uniform industry meaning.** Claude Code and Grok use it for a single result object; Pi, Codex, and OpenCode use it for a JSONL event stream. Documentation must therefore specify stdout's actual transport format and output semantics rather than list enum names alone. --- -## 3. Interface Overview Across Five Projects +## 4. Implementation Overview Across Five Projects -The table below compares **actual wire protocols**, not each project's chosen terminology. +### 4.1 What Is Delivered to the Caller: Text, One Result Object, or an Event Stream -| Project | Human-readable text | Single aggregate JSON object | JSONL event stream | Partial / delta capability | Explicit terminal event | Persistent session | -| --- | --- | --- | --- | --- | --- | --- | -| Pi | Print mode | — | `--mode json` | `message_update` deltas | `agent_settled`; `agent_end` ends only one low-level run | JSONL by default; supports continue/resume/fork/`--no-session` | -| Claude Code | `--output-format text` | `--output-format json` | `--output-format stream-json` | Add `--include-partial-messages` | `result` | JSONL by default; supports continue/resume/fork/`--no-session-persistence` | -| Codex | `codex exec` default | — | `codex exec --json` | No public token-delta contract | `turn.completed` / `turn.failed`; interrupt is an exception | Rollout JSONL by default; supports `--ephemeral`, resume, and fork | -| OpenCode | `opencode run --format default`, possibly with multiple completed text segments | `opencode export` is a separate command, not a run output mode | `opencode run --format json` | No; only coarser completion events | None; relies on EOF + exit code | SQLite + internal event table; supports continue/session/fork | -| Grok Build | `--output-format plain`, writes text chunks as they are generated | `--output-format json` | `streaming-json`; also has a Messages-compatible stream | The native stream emits text/thought chunks by default; only the compatibility stream adds a partial flag, and some deltas remain coarse-grained | Native stream: `end` on success and `error` on failure; compatibility stream: `result` | Multi-file JSONL session by default; supports continue/resume/fork | +First consider which **run output forms** each product provides. The table compares what actually appears on stdout rather than each product's format names. `—` means the output form is not provided. -Three common patterns are visible in this table: +| Project | Human-readable text | One result JSON after the run ends | Line-delimited events during the run (JSONL/NDJSON) | +| --- | --- | --- | --- | +| Pi | Print mode | — | `--mode json` | +| Claude Code | `--output-format text` | `--output-format json` | `--output-format stream-json` | +| Codex | `codex exec` default | — | `codex exec --json` | +| OpenCode | `opencode run --format default`, possibly with multiple completed text segments | —; `opencode export` is a separate session-export command | `opencode run --format json` | +| Grok Build | `--output-format plain`, writes text chunks as they are generated | `--output-format json` | `streaming-json`; also has a Messages-compatible stream | + +Now consider only the **public event stream** in the third column. The next two properties describe its content granularity and termination behavior; they are not additional output formats. + +| Project | Emits unfinished content early (partial / delta) | Explicitly declares the end of the whole run with an in-stream terminal event | +| --- | --- | --- | +| Pi | Yes; `message_update` provides deltas and `message_end` provides the complete message | Yes; `agent_settled`. `agent_end` ends only one low-level run | +| Claude Code | Optional; add `--include-partial-messages` | Yes; `result` | +| Codex | No; there is no public token/text-delta contract | Yes; `turn.completed` / `turn.failed`, except on interruption | +| OpenCode | No; it emits only coarser completion events | No; relies on EOF + exit code | +| Grok Build | Yes; the native stream emits text/thought chunks by default, while the Messages-compatible stream has a separate partial flag | Yes; native success is `end`, native failure is `error`, and the compatibility stream uses `result` | + +Three common patterns are visible in these tables: - Human-facing modes tend to put only the final answer on stdout and send progress and diagnostics to stderr. - Real-time machine-facing modes almost universally use “one object per line” JSONL rather than a long-lived JSON array. -- Sessions are usually persisted automatically and managed by session ID; none of these projects treats `--output-format` as a session switch. +- None of the products treats `--output-format` as a switch for sessions, traces, or trajectories. There are also two differences that cannot be inferred from “industry convention”: - `json` may mean either a single object or JSONL. Claude/Grok use the former; Pi/Codex/OpenCode use the latter. -- “Streaming” may mean only **emitting events immediately as they occur**, or may additionally include text/thinking/tool-argument deltas at different granularities. Claude uses an extra flag to enable raw partials; Grok's native stream already includes text/thought chunks by default, while its extra flag changes only the framing of the Messages-compatible stream. This shows that “streaming” and “token-level” are not the same promise. +- “Streaming” may mean only **emitting events immediately as they occur**, or may additionally include text/thinking/tool-argument deltas at different granularities. Claude uses an extra flag to enable raw partials; Grok's native stream already includes text/thought chunks by default, while its extra flag changes only the framing of the Messages-compatible stream. “Streaming” and “token-level” are therefore not the same promise. + +### 4.2 Execution Traces, Trajectories, and Sessions + +The next table compares the other three planes. “No dedicated trajectory” means there is no stable artifact interface scoped to one benchmark task/run; it does not mean an adapter cannot convert product data into a trajectory. + +| Project | Execution trace | Dedicated benchmark trajectory | Session: storage and primary contents | +| --- | --- | --- | --- | +| Pi | Hidden `/debug` can write TUI render lines and the latest messages sent to the model; not a stable headless trace protocol | None; can be derived from the JSON event stream or a session export | JSONL by default; header, messages, model/thinking changes, compaction, branch/custom entries; `id`/`parentId` form a tree and support continue/resume/fork | +| Claude Code | `--debug` / `--debug-file` records diagnostic logs, with separate telemetry support; independent of stdout output format | None; runners such as Harbor can generate one after parsing `stream-json` | Transcript JSONL by default; messages, tool interactions, and recovery metadata; supports continue/resume/fork and optional disabled persistence | +| Codex | Opt-in local `rollout-trace` bundle containing a manifest, raw events, prompts/responses, tool and terminal payloads, and offline reduced state; also OpenTelemetry | None; `rollout` is a session and `rollout-trace` is a debugging trace, neither is a benchmark trajectory interface | `rollout-*.jsonl` by default; session metadata, model-visible messages/reasoning, tool calls/outputs, and other recoverable items; supports resume/fork, disabled by `--ephemeral` | +| OpenCode | Runtime logs and debug subcommands exist; no complete, stable, user-facing execution-trace artifact was found | None; `export` emits a materialized session snapshot | Session/message/part data and durable events/projections in a global SQLite database; supports continue/session/fork, with large tool output optionally stored separately | +| Grok Build | `RUST_LOG` can write diagnostics to stderr and `GROK_LOG_FILE` to a file; internal logs and session trace exports also exist, none controlled by output format | No CLI parameter equivalent to the semantics proposed here | Session directory holds authoritative updates, model chat history, summary/plan/compaction/subagent and other recovery state; supports continue/resume/fork | + +The most important point is not that “everyone uses JSONL,” but who consumes each artifact: output is a stable public protocol, a trace is troubleshooting evidence, a trajectory is the task-evaluation path, and a session is product state that can be restored and forked. --- -## 4. Designs by Project +## 5. Designs by Project + +### 5.1 Pi: `json` Is an Event Stream, While the Session Is a Separate Tree-Shaped JSONL -### 4.1 Pi: `json` Is an Event Stream, While the Session Is a Separate Tree-Shaped JSONL +**Run output.** Pi's headless interface has three modes: @@ -113,8 +278,18 @@ Here, `agent_end` must not be treated as the terminal event for the entire comma Pi also takes explicit control of stdout: protocol writes go through controlled raw stdout, while other ordinary output is redirected to stderr, and write backpressure is handled. This shows that “machine-mode stdout must not contain logs” is not merely documentation etiquette, but an implementation boundary. +**Session.** + Pi's session is a separate append-only JSONL file. It has a session header plus message, model-change, compaction, branch, and other entries carrying `id` / `parentId`; its history is therefore fundamentally a tree rather than a verbatim copy of stdout events. Switching branches merely moves the current leaf and does not delete the other branch. Compaction changes the active context sent to the model without erasing the original history. +**Execution trace.** + +Pi's hidden `/debug` primarily writes TUI render lines and the latest messages sent to the model; it is not a stable headless trace. + +**Trajectory.** + +Pi has no dedicated benchmark trajectory. An evaluator can select task steps from the JSON event stream or an exported session and convert them into its own trajectory schema. + **Lessons worth adopting:** - Deltas and final snapshots have clearly separated responsibilities in the stream. @@ -124,11 +299,13 @@ Pi's session is a separate append-only JSONL file. It has a session header plus Source entry points: [JSON event stream documentation](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/json.md), [RPC event reference](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/rpc.md), [print mode](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/src/modes/print-mode.ts), and [session format](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/session-format.md). -### 4.2 Claude Code: The Clearest Separation of text, json, and stream-json +### 5.2 Claude Code: The Clearest Separation of text, json, and stream-json + +**Run output.** Claude Code's public definitions most directly answer this document's naming question: -| Format | Wire protocol | +| Format | stdout transport format | | --- | --- | | `text` | Emit the final plain text after completion | | `json` | Emit **one** result object after completion, containing result, session ID, usage/cost, and other metadata | @@ -139,22 +316,34 @@ By default, `stream-json` means “emit a message or event as soon as it is prod It also has two concepts that are easily confused with trajectory but are in fact entirely different: - `--input-format stream-json` controls stdin; it is not inferred implicitly from the output format. -- `--replay-user-messages` is input-confirmation echoing for duplex clients, not a replay of an old session trajectory. +- `--replay-user-messages` is input-confirmation echoing for duplex clients, not a replay of an old session's execution history into output. + +**Session.** Claude Code saves sessions by default under `~/.claude/projects//.jsonl`. Continue, resume, and fork operate on this persistent state; stdout's text/json/stream-json setting affects only how the current invocation reports to its parent process. `--no-session-persistence` is also a separate switch. The local unofficial snapshot shows transcript entries with parent UUIDs, and stores the full contents of large tool results separately on disk while leaving a preview and path in the message. This further demonstrates that a session is a recoverable directed history, not a stdout event log. It also shows that both sessions and streams may contain complete tool arguments, results, and hook stdout/stderr, and must be treated as sensitive data. +**Execution trace.** + +`--debug` / `--debug-file` records diagnostic logs independently of output format. + +**Trajectory.** + +Claude Code has no dedicated trajectory-path parameter. When Harbor parses `stream-json` and generates a benchmark record, it is projecting a trajectory from run output, not renaming the stdout stream as a trajectory. + **Lessons worth adopting:** -- The three output names map one-to-one to wire protocols, minimizing ambiguity. +- The three output names map one-to-one to stdout transport formats, minimizing ambiguity. - Partial deltas are a separate capability rather than being inseparably bound to the name `stream-json`. - The terminal `result` aggregates status, final answer, session ID, turn, and usage/cost, so consumers do not need to scan the entire stream to calculate the final result. - Session persistence is fully orthogonal to stdout representation. Authoritative contracts: [CLI reference](https://code.claude.com/docs/en/cli-usage), [headless mode](https://code.claude.com/docs/en/headless), [sessions](https://code.claude.com/docs/en/sessions), and [custom session storage](https://code.claude.com/docs/en/agent-sdk/session-storage). -### 4.3 Codex: Default Text, `--json` JSONL, and Automatically Persisted Rollouts +### 5.3 Codex: Default Text, `--json` JSONL, and Automatically Persisted Rollouts + +**Run output.** `codex exec` does not have `--output-format`: @@ -176,6 +365,8 @@ error An item's `type` then distinguishes agent messages, reasoning, command execution, file changes, MCP/collaboration tools, web searches, todos, and errors. A successful run normally ends with `turn.completed`, which contains usage; a failed run ends with `turn.failed`. The current interrupted path may have no terminal JSON event, so a reliable runner should still check EOF, exit code/signal, and stderr together. +**Session.** + Ordinary sessions are written automatically to: ```text @@ -184,18 +375,28 @@ $CODEX_HOME/sessions/YYYY/MM/DD/rollout--.jsonl Each line's outer envelope carries timestamp, ordinal, type, and payload, and every line is flushed after writing. Resume continues appending to the original rollout; fork creates a new thread ID and materializes the inherited history into a new rollout. The persistence policy retains messages, reasoning, tool calls/outputs, and other data needed for recovery, but filters many transient deltas, begin events, warnings, and UI events. A rollout is therefore not a mirror of stdout JSONL either. -Codex further distinguishes a deeper, opt-in `rollout-trace`: it may record prompts, responses, tool I/O, and terminal output for local troubleshooting. It is not used for resume and is not a stable CLI trajectory interface. Beyond that there is OpenTelemetry. These three similarly named concepts have entirely different purposes. +**Execution trace.** + +Codex separately implements an opt-in `rollout-trace`. Its local bundle stores `manifest.json`, ordered raw events, prompts/responses, tool I/O, terminal output, and payload references, and can reduce them offline into a semantic graph for a debugger. It has an independent `trace_id` while referencing the observed session's `rollout_id`, directly demonstrating that trace identity and session identity should not be conflated. + +`rollout-trace` is for local troubleshooting, not resume; OpenTelemetry is yet another observability output. + +**Trajectory.** + +Codex has no dedicated benchmark trajectory. Neither the session rollout nor `rollout-trace` is a stable benchmark-trajectory interface; evaluators must create a separate projection from public JSONL, the session rollout, or a trace. **Lessons worth adopting:** - The default mode strictly enforces “results on stdout, diagnostics on stderr.” -- `--output-last-message` demonstrates that an “additional artifact sink” need not change the main output format. +- `--output-last-message` demonstrates that an additional file sink need not change the main output format. - Rollouts use ordinals, flush line by line, and can repair a torn tail that lacks a final newline. - JSONL is a public integration surface actually consumed by the SDK, but the schema has no version number, so consumers must still tolerate unknown events, item types, and newly added fields. -Source entry points: [exec CLI](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/cli.rs), [stdout contract](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/lib.rs), [exec events](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/exec_events.rs), and [rollout recorder](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout/src/recorder.rs). +Source and authoritative contract entry points: [OpenAI CLI reference](https://developers.openai.com/codex/cli/reference), [exec CLI](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/cli.rs), [stdout contract](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/lib.rs), [exec events](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/exec_events.rs), [rollout recorder](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout/src/recorder.rs), and [rollout trace](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout-trace/README.md). + +### 5.4 OpenCode: run's `json` Is Coarse-Grained JSONL, While Sessions Live in SQLite -### 4.4 OpenCode: run's `json` Is Coarse-Grained JSONL, While Sessions Live in SQLite +**Run output.** OpenCode's command is: @@ -213,19 +414,31 @@ This keeps it very lightweight, but leaves automated consumers with three costs: 2. The schema is unversioned and is projected ad hoc from internal session events. 3. The name `--format json` does not reveal that it is a stream. +**Session.** + OpenCode's persistence layer differs even more from its CLI stream. Sessions, messages, parts, and internal durable events/projections primarily live in a global SQLite database; `--continue`, `--session`, and `--fork` determine which session is loaded or copied. `opencode export [sessionID]` is a separate command: it writes a materialized session snapshot as one pretty-printed JSON object to stdout, which the user can then redirect with the Shell. It cannot be treated as a final-object mode for `run --format json`. Large tool outputs have their previews truncated while the full contents are stored in tool-output files under the data directory. Streams, sessions, and exports cannot by default be regarded as having undergone complete secret redaction; `export --sanitize` performs only limited sanitization. +**Execution trace.** + +OpenCode has conventional runtime logs and debug subcommands for configuration, LSP, files, snapshots, and related issues, but no public complete execution-trace artifact. + +**Trajectory.** + +OpenCode has no dedicated benchmark trajectory. The object emitted by `opencode export` remains a session snapshot; only after an adapter converts its task steps into an evaluation schema is the result a trajectory. + **Lessons worth adopting:** - A CLI stream can project only semantic events useful to integrations instead of exposing every internal event. -- Session storage can be implemented with SQLite while the wire protocol remains JSONL. +- Session storage can be implemented with SQLite while the external transport format remains JSONL. - The counterexample is that terminal footers and schema versions are cheap, yet substantially reduce the cost of inferring runner state. Source entry points: [run command](https://github.com/anomalyco/opencode/blob/e00890c67261a435cee6409366a68999a93393fd/packages/opencode/src/cli/cmd/run.ts), [session tables](https://github.com/anomalyco/opencode/blob/e00890c67261a435cee6409366a68999a93393fd/packages/core/src/session/sql.ts), and [export command](https://github.com/anomalyco/opencode/blob/e00890c67261a435cee6409366a68999a93393fd/packages/opencode/src/cli/cmd/export.ts). -### 4.5 Grok Build: The Most Complete Output Matrix, Plus Two Separate Session Logs +### 5.5 Grok Build: The Most Complete Output Matrix, Plus Multi-Layer Session State + +**Run output.** Grok Build provides four headless formats: @@ -241,10 +454,12 @@ The native stream has event types including text, thought, tool call/update, pla This matrix demonstrates two independent axes of extension: - **Time axis:** a final result object versus a real-time event stream. -- **Schema axis:** the agent's own semantic events versus a wire format compatible with an external ecosystem. +- **Schema axis:** the agent's own semantic events versus a transport format compatible with an external ecosystem. It also demonstrates the cost of a compatibility layer: the same internal event must maintain two public projections, some internal states cannot be mapped losslessly, and partial framing, usage, errors, and terminal results must each be defined independently. nanoPyCodeAgent does not need to duplicate this complexity before it has a concrete consumer. +**Session.** + Grok stores sessions by default under `~/.grok/sessions///`, distinguishing at least: - `updates.jsonl`: the authoritative session updates for restoring the UI/conversation. @@ -253,6 +468,14 @@ Grok stores sessions by default under `~/.grok/sessions// events.jsonl nanoPyCodeAgent -p "fix it" --output-format text --trajectory run.jsonl ``` -The JSONL produced by the second and third commands **need not be identical**: the public stream should be stable, small, and safe, while the trajectory may contain more complete attribution fields and truncation metadata. Both should be projected from the same canonical event model to avoid inconsistent facts. +The JSONL produced by the second and third commands **need not be identical**: the public stream should be stable, small, and safe, while the trajectory may contain more complete attribution fields and truncation metadata. Both should be projected from the same set of canonical internal events to avoid inconsistent facts. The path contract should also specify: @@ -357,13 +580,13 @@ The path contract should also specify: - Readers should tolerate a final incomplete line left by a crash, but must not silently ignore a malformed line in the middle. - `--trajectory -` should not be allowed, because it would make the trajectory compete with the selected stdout formatter for the same protocol channel. -If sessions become **automatically persisted by default** in the future, the semantics should be separated again: `--no-session-persistence` controls whether to save, `--session`/`--resume` control identity, and `--session-path` should be introduced only if overriding the default location is genuinely supported. Do not silently promote today's debug trajectory into tomorrow's resume format. +If sessions become **automatically persisted by default** in the future, the semantics should remain separate: `--no-session-persistence` controls whether to save, `--session`/`--resume` control identity, and `--session-path` should be introduced only if overriding the default location is genuinely supported. Do not silently promote today's benchmark trajectory into tomorrow's resume format. Execution traces should likewise be controlled by separate debug/trace configuration rather than borrowing `--trajectory`. --- -## 7. Recommended Contract for nanoPyCodeAgent +## 8. Recommended Contract for nanoPyCodeAgent -### 7.1 CLI +### 8.1 CLI The recommendation is to expand the original two formats to three: @@ -374,16 +597,17 @@ nanoPyCodeAgent [-p PROMPT | --prompt-file PATH | stdin] [--trajectory PATH] ``` -| Option | stdout contract | Typical consumer | +| `--output-format` value | stdout contract | Typical consumer | | --- | --- | --- | | `text` (default) | Final assistant text only; an empty result may produce empty stdout | Humans and the simplest benchmark runners | | `json` | Exactly one result object after run initialization; preflight failure may leave stdout empty; no interspersed logs | Shell, CI, one-off scripts | | `stream-json` | One event per line; graceful termination ends with a terminal event, otherwise EOF + nonzero exit/signal denotes an aborted stream | Harbor adapters, SDKs, real-time UIs | -| `--trajectory PATH` | Does not change stdout; separately writes incremental JSONL | Offline attribution, benchmark reports, debugging | + +`--trajectory PATH` does not belong in the output-format table. It leaves the stdout contract above unchanged and separately writes incremental JSONL for benchmarks, offline statistics, and failure attribution. Diagnostics, retry notices, tracebacks, and human-facing progress must all go to stderr. The original API error may still appear on stderr to satisfy Harbor's error-classification requirements; machine-mode stdout must remain parseable at all times. -### 7.2 Single `json` Result Object +### 8.2 Single `json` Result Object It should contain at least: @@ -408,9 +632,9 @@ The protocol's starting boundary must also be explicit: if **preflight** work su When usage/cost is missing, omit it or explicitly mark `usage_incomplete: true`; do not substitute 0 for unknown. Grok's completeness rule is better suited to benchmarks here than “always fill every cell in a numeric table.” -If `--output-schema PATH` is added in the future, it should constrain the semantic contents of `result` rather than alter the transport envelope above. Codex and Grok both separate “structured model answer” from “CLI output protocol,” which is the correct boundary. +If `--output-schema PATH` is added in the future, it should constrain the semantic contents of `result` rather than change the outer structure of the CLI result object. Codex and Grok both separate “structured model answer” from “CLI output protocol,” which is the correct boundary. -### 7.3 Minimal `stream-json` Event Set +### 8.3 Minimal `stream-json` Event Set The first version does not need to reproduce every event from all five projects. The recommended minimum set is: @@ -445,7 +669,7 @@ Protocol rules: - Oversized tool output records should include a preview, original size, and truncated flag; storing the full text separately requires an explicit path and cleanup policy. - Token deltas can be added later through `assistant.delta` or a partial flag; `assistant.message` must not mean both a delta and a final snapshot. -### 7.4 Trajectory Contents and Stability +### 8.4 Trajectory Contents and Stability The purpose of a trajectory is to “explain why this run produced this result.” At minimum, it should support reconstruction of: @@ -456,13 +680,13 @@ The purpose of a trajectory is to “explain why this run produced this result. - The occurrence of compaction/truncation and the size of omitted contents. - Final status and result. -The first version should, however, be explicitly labeled a **diagnostic/benchmark artifact, not resumable session**. Resume support additionally requires stable parent/entry IDs, branch semantics, model/tool configuration migration, post-compaction context recovery, and long-term schema migration. The session implementations in Pi, Claude, Codex, OpenCode, and Grok all show that this is far more than “read the JSONL back and continue.” +The first version should, however, be explicitly labeled: **for benchmark/analysis use, not an execution trace or resumable session.** It need not collect every debugging detail such as provider retries, internal queues, and exception stacks; those belong in an execution trace. Resume support additionally requires stable parent/entry IDs, branch semantics, model/tool configuration migration, post-compaction context recovery, and long-term schema migration. The session implementations in Pi, Claude, Codex, OpenCode, and Grok all show that this is far more than “read the JSONL back and continue.” When Harbor requires ATIF, the recommendation is to convert the native trajectory into ATIF at the adapter layer instead of making the agent loop depend directly on a benchmark schema. Only if Harbor becomes the sole primary consumer would it be worth considering ATIF directly as the persistent format. -### 7.5 Security and Data Volume +### 8.5 Security and Data Volume -The sessions/streams of all five projects may store or emit user prompts, reasoning, tool arguments, file contents, command output, environment paths, and provider metadata. Some projects sanitize recognizable secrets from commands or truncate large results, but none provides a universal guarantee that all secrets are removed. +The traces, trajectories, sessions, and public streams of all five projects may store or emit user prompts, reasoning, tool arguments, file contents, command output, environment paths, and provider metadata. Some projects sanitize recognizable secrets from commands or truncate large results, but none provides a universal guarantee that all secrets are removed. The trajectory should therefore be treated as a sensitive file: @@ -476,9 +700,9 @@ The trajectory should therefore be treated as a sensitive file: --- -## 8. Final Recommendation +## 9. Final Recommendation -The final answers to the questions at the beginning of this document are: +The final boundaries among the four concepts introduced at the beginning are: ```text --output-format text|json|stream-json @@ -504,14 +728,26 @@ stream-json **Emits NDJSON during the run: one complete event object per line; on graceful termination, the last line is a terminal result, while abnormal termination may leave only a complete, still-parseable prefix.** Its difference from `json` is “one final snapshot” versus “an incrementally consumable event sequence,” not “whether output is written to a file.” -These definitions follow the clear naming of Claude Code and Grok most closely, while incorporating Pi's delta/final separation, Codex's stdout/stderr boundary and additional artifact sink, OpenCode's semantic projection, and the common design across all projects of separating sessions from public event streams. +```text +execution trace +``` + +**Enabled through separate debug/trace configuration for troubleshooting and observability.** It may be more detailed and sensitive than public output and trajectories, and it does not promise resume support. + +```text +session +``` + +**Managed through separate persistence/session/resume interfaces for continuing, forking, and compacting work across runs.** Codex's `rollout-*.jsonl` belongs to this category; the word rollout in its filename does not make it a trajectory. + +These definitions follow the clear naming of Claude Code and Grok most closely, while incorporating Pi's delta/final separation, Codex's stdout/stderr boundary and additional file sink, OpenCode's semantic projection, and the common design across all projects of separating sessions from public event streams. --- -## 9. Reference Entry Points +## 10. Reference Entry Points - Pi: [usage](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/usage.md), [JSON event stream](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/json.md), [RPC events](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/rpc.md), [sessions](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/sessions.md) - Claude Code: [CLI reference](https://code.claude.com/docs/en/cli-usage), [headless mode](https://code.claude.com/docs/en/headless), [sessions](https://code.claude.com/docs/en/sessions), [session storage](https://code.claude.com/docs/en/agent-sdk/session-storage) -- Codex: [exec CLI](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/cli.rs), [exec events](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/exec_events.rs), [rollout recorder](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout/src/recorder.rs) +- Codex: [OpenAI CLI reference](https://developers.openai.com/codex/cli/reference), [exec CLI](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/cli.rs), [exec events](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/exec_events.rs), [rollout recorder](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout/src/recorder.rs), [rollout trace](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout-trace/README.md) - OpenCode: [run command](https://github.com/anomalyco/opencode/blob/e00890c67261a435cee6409366a68999a93393fd/packages/opencode/src/cli/cmd/run.ts), [export command](https://github.com/anomalyco/opencode/blob/e00890c67261a435cee6409366a68999a93393fd/packages/opencode/src/cli/cmd/export.ts) - Grok Build: [headless guide](https://github.com/xai-org/grok-build/blob/19d42e35c07a9c9244f03f6df0c4c353f970d4f9/crates/codegen/xai-grok-pager/docs/user-guide/14-headless-mode.md), [format enum](https://github.com/xai-org/grok-build/blob/19d42e35c07a9c9244f03f6df0c4c353f970d4f9/crates/codegen/xai-grok-pager/src/headless/cli.rs), [session export contract](https://github.com/xai-org/grok-build/blob/19d42e35c07a9c9244f03f6df0c4c353f970d4f9/crates/codegen/xai-grok-shell/src/session/export.rs) diff --git a/docs/research/zh-CN/agent_output_and_trajectory.md b/docs/research/zh-CN/agent_output_and_trajectory.md index 574e55b..1e30347 100644 --- a/docs/research/zh-CN/agent_output_and_trajectory.md +++ b/docs/research/zh-CN/agent_output_and_trajectory.md @@ -1,99 +1,264 @@ -# 主流 Code Agent 的输出格式与轨迹设计 +# 主流 Code Agent 的调用输出、执行跟踪（Trace）、任务轨迹（Trajectory）与会话设计 > 本文件为**中文源文件**（source of truth）；英文版 [`../en/agent_output_and_trajectory.md`](../en/agent_output_and_trajectory.md) 由其生成。 调研时间：2026-08-22。 -[`benchmark_headless_interface.md`](benchmark_headless_interface.md) 先提出了下面这组接口，但没有把每个参数的线协议定义完整： +[`benchmark_headless_interface.md`](benchmark_headless_interface.md) 先提出了下面这组接口，但没有说明两个参数控制的是否是同一种产物： ```text [--output-format text|stream-json] [--trajectory ] ``` -本文从 Pi、Claude Code、Codex、OpenCode 和 Grok Build 的当前实现反推这些概念应该怎样拆分。结论先放在最前面： +这两个参数表面上都与“输出”有关，实际上属于不同平面。要比较 Pi、Claude Code、Codex、OpenCode 和 Grok Build，必须先回答四个问题： -1. **`--output-format` 选择 stdout 的表示形式，不是文件重定向。** 写文件仍由 Shell 的 `>`，或另一个明确的文件参数负责。 -2. **对 nanoPyCodeAgent，`--trajectory PATH` 最自然的语义是“开启 trajectory，并把它写到 PATH”。** 它不改变 stdout，也不改变 `--output-format`。 -3. **`stream-json` 应定义为 NDJSON/JSONL 事件流：每行都是一个完整、可独立解析的 JSON 对象。** 它不是“一个 JSON 文档被拆成若干块”，也不天然承诺 token 级增量。 -4. **`json` 这个名字在业界没有统一语义。** Claude Code 和 Grok 用它表示“结束时输出一个对象”；Pi、Codex 和 OpenCode 却用它表示 JSONL 事件流。因此接口文档必须写清线协议，不能只列枚举名。 -5. **stdout 事件流、持久 session、调试 trace、benchmark trajectory 和 telemetry 是五种不同产物。** 它们可以来自同一个内部事件模型，但不应共用一个含糊的开关。 +1. agent 本次调用向人或调用程序交付了什么？ +2. agent 内部到底执行了什么？ +3. benchmark 要怎样记录 agent 完成任务的路径？ +4. agent 下次从哪里恢复上下文？ + +它们分别对应 **run output、execution trace、trajectory 和 session**。下面先用同一次运行说明四者的区别，再进入各项目实现。 --- -## 一、调研范围与证据等级 +## 一、先从一次运行看四种产物 -调研前先尝试更新 `references/` 下的代码。四个可访问仓库已 fast-forward 到远端最新提交；Claude Code 的第三方镜像远端已经不可访问，未对它做强制替换。 +假设用户要求 agent： -| 项目 | 本地 revision | 更新结果 | 证据等级 | -| --- | --- | --- | --- | -| Grok Build | `19d42e35c07a9c9244f03f6df0c4c353f970d4f9` | 已更新 | xAI 官方开源仓库 | -| Pi | `c49906ec77788625aacbdc53ebca6fbe65bd20f5` | 已更新 | `references/pi` 所跟踪的公开仓库 | -| OpenCode | `e00890c67261a435cee6409366a68999a93393fd` | 已更新 | OpenCode 官方开源仓库 | -| Codex | `4f39251a010a8bd7d692d25fb33832ff06f1635a` | 已更新 | OpenAI 官方开源仓库 | -| Claude Code | `a371abbe75ffa0d0a3c92290e2bbf56a7ef54367` | 远端返回 `Repository not found`，保留快照 | **非官方 sourcemap 镜像，只用于辅助验证实现思路** | +```text +修复 parser.py 在工具参数为空时抛出的异常，并运行相关测试。 +``` -Claude Code 的正式契约以 Anthropic 当前的 [CLI reference](https://code.claude.com/docs/en/cli-usage)、[headless 文档](https://code.claude.com/docs/en/headless)和 [sessions 文档](https://code.claude.com/docs/en/sessions)为准。本地 `references/claude-code/README.md` 自己也明确说明它不是 Anthropic 官方项目，因此本文不会把该快照中的内部字段当作当前稳定 API。 +一次典型执行可能是：读取报错位置，搜索调用方，修改代码，运行测试，最后回答用户。本文把从接收这条 prompt 到 agent 停止自动执行称为一次 **run**；一个 run 内可以有多次模型请求和多次工具调用。 + +### 1.1 Run output：本次调用交付给调用方的公开输出 + +run output 回答的是：**“这次调用要让外部看见什么？”** 调用方既可以是终端前的人，也可以是 Shell 脚本、benchmark harness、SDK 或 UI。 + +同一结果可以有不同表示形式。例如 `text` 模式只给出最终回答： + +```text +已修复空工具参数的解析，并新增回归测试。相关测试 12 项全部通过。 +``` + +`json` 模式可以在 run 结束时交付一个机器可读的**单个结果对象**。它汇总的是**本次 run 最终怎样结束**，例如最终状态、回答、stop reason、用量和成本；它不汇总完整执行过程，也不是把 trace、trajectory 或 session 全部装进一个 JSON： + +```json +{"status":"completed","result":"已修复空工具参数的解析，并新增回归测试。相关测试 12 项全部通过。","usage":{"input_tokens":1200,"output_tokens":180}} +``` + +`stream-json` 模式则可以在运行期间发布一个稳定的公开事件协议。下面采用 **JSON Lines（JSONL）**：每个非空行都是一个完整 JSON 对象。它也常称 **Newline-Delimited JSON（NDJSON，按换行分隔的 JSON）**；其中 `ND` 就是 `Newline-Delimited`： + +```jsonl +{"type":"tool.completed","tool":"pytest","is_error":false,"result":"12 passed"} +{"type":"assistant.message","content":"已修复空工具参数的解析，并新增回归测试。相关测试 12 项全部通过。"} +{"type":"run.completed","status":"completed"} +``` + +这里先固定四个后文会反复使用的 output 协议术语： + +- **单个结果对象（single result object）**：run 结束后只输出一个 JSON 对象，概括本次 run 的最终结果。不同产品包含的字段不同，常见字段有最终回答、completed/failed 状态、stop reason、run/session ID、turn 数、token usage 和 cost；它通常不包含逐步执行记录。 +- **公开事件流（public event stream）**：agent 在 run 期间按发生顺序逐条交付给外部调用方的稳定事件协议。它只公开经过筛选、适合长期兼容的事件，例如工具开始/结束、完整助手消息和 run 结束；它不是内部 event bus 或 execution trace 的原样输出。本文讨论的各家 CLI 事件流都使用 JSONL/NDJSON，即一行一个完整事件对象。 +- **partial 与 delta**：`partial` 指一条尚未完成的消息、thinking 或 tool arguments 的中间状态；`delta` 指相对于此前内容新增加或改变的那一小段。协议可以反复发送累计的 partial snapshot，也可以只发送 delta，让消费者自行拼接。表格中的“partial / delta 能力”统一询问：**公开事件流是否会在一个逻辑内容尚未完成时，就把它的片段交给调用方。** +- **终止事件（terminal event）**：公开事件流中明确宣告本次 run 以 completed、failed 等状态结束的最后一类事件，例如 `run.completed` 或 `result`。它比 EOF 表达得更多：EOF 只说明 stdout 已关闭，既可能是正常退出，也可能是进程崩溃、被中断或输出被截断。 + +因此，“有公开事件流”和“有 partial / delta”是两个独立能力。例如只在 pytest 完成后发送 `tool.completed`，再发送完整的 `assistant.message`，仍是实时事件流，但没有 partial / delta；若回答生成过程中先后发送 `assistant.delta: "已修"` 和 `assistant.delta: "复完成"`，才具备内容增量能力。无论是否发送 delta，正常收尾时都还可以用一个终止事件说明整个 run 已结束。 + +本文后文把 stdout 的编码和记录边界称为**传输格式（wire format）**：stdout 中实际出现的是一段文本、一个 JSON 对象，还是一行一个 JSON 对象，以及记录之间怎样分隔。若讨论范围还包括事件类型、顺序、终止和错误语义，本文称为**输出协议**。 + +即使公开流里出现了 `tool.completed`，它仍然是 **run output**，因为这是 agent 明确承诺给调用方的公共协议；它不是内部 trace 的原样倾倒。**`--output-format` 只选择这个公开输出怎样编码和分帧。** 它不负责开启 trace、保存 trajectory 或持久化 session。 + +### 1.2 Execution trace：为排障保留的运行时证据 + +execution trace 回答的是：**“agent 内部实际发生了什么？”** 它面向 agent 开发者和可观测系统，常见内容包括： + +- provider 请求、响应元数据、重试与退避； +- 模型调用、工具调度、子进程、并发任务和耗时； +- 完整或经脱敏的工具输入输出、stderr、异常栈； +- span 之间的父子关系、内部状态转换和性能数据。 + +一段调试 trace 可能包含下面这样的事实： + +```text +inference attempt=1 status=429 retry_after_ms=800 +inference attempt=2 request_id=req_2 latency_ms=1430 +tool call_id=t1 process_id=4312 stdout_bytes=824 exit_code=0 +``` + +这些信息有助于解释延迟或故障，却不应该因为用户选择了 `--output-format stream-json` 就全部进入 stdout。trace 往往更详细、更敏感，也更贴近当前实现；其 schema 通常不具备 run output 那样的公共兼容承诺。 + +日志、metrics 和 OpenTelemetry 是诊断信号的记录或传输方式，不是与 trace 并列的另一种“用户输出”。其中 OpenTelemetry trace 本身就是 execution trace 的一种表示。 + +### 1.3 Trajectory：为任务评测保留的执行路径 + +trajectory 回答的是：**“agent 通过怎样的观察和动作得到这个任务结果？”** 它通常以一次 benchmark trial 或一次任务 run 为边界，面向评测框架和离线分析器。 + +同一运行的 trajectory 可以被整理为： + +```jsonl +{"step":1,"observation":"parser.py 在 arguments 为空时解引用失败","action":{"tool":"search","query":"parse tool arguments"}} +{"step":2,"observation":"找到两个调用方和一个缺失的空值分支","action":{"tool":"edit","file":"parser.py"},"result":"added empty-argument handling"} +{"step":3,"observation":"代码已修改","action":{"tool":"pytest","target":"tests/test_parser.py"},"result":"12 passed"} +{"outcome":"completed","result":"bug fixed","usage":{"input_tokens":1200,"output_tokens":180}} +``` + +trajectory 与 trace 都描述执行过程，但取舍不同： + +- trace 贴近运行时实现，目标是还原故障现场，可能记录每次重试、内部队列和原始 payload； +- trajectory 贴近任务语义，目标是比较和归因，保留 observation、action、tool result、outcome、token 和 cost 等分析字段； +- trajectory 可以由公开事件流、trace 或 session 转换得到，但转换后的产物才是 trajectory，数据来源本身不会因此改变类别。 + +trajectory 通常是 run 结束后固定下来的分析产物。它可以支持可视化或离线 replay，但 **replay 不等于 resume**：只有步骤记录，不代表 agent 能恢复当时的产品状态并继续对话。 + +因此也不能把整份 session export 直接改名为 trajectory：session 可能包含多次 run、旧任务、分支和 compaction 元数据。adapter 必须先切出当前 task/trial 的边界，再整理为 observation/action/outcome。 + +### 1.4 Session：为继续工作持久化的产品状态 + +session 回答的是：**“下一次调用要从什么状态继续？”** 它通常比单次 run 活得更久，并支持 continue、resume、fork、rewind 或 compaction。 + +session 可能保存： + +- session ID、工作目录、模型和工具配置； +- 用户消息、助手消息、工具调用及结果； +- 稳定的 entry/message ID 与 parent 关系； +- compaction checkpoint、分支、权限决策和其他恢复所需状态。 + +例如用户结束进程后又执行： + +```text +agent --resume s1 -p "再让它兼容 arguments 缺失的情况" +``` + +agent 必须从 session `s1` 重建上下文。这是 session 的核心能力，也是它与 trajectory 的判定边界：**能否可靠地 continue/resume/fork，比文件叫 transcript、history、rollout 还是 JSONL 更重要。** + +Codex 是最容易因命名产生误解的例子。它把持久会话文件命名为 `rollout-*.jsonl`，但 `resume` 会继续使用它，`fork` 会从它派生新会话，`--ephemeral` 会关闭它。因此本文把 Codex rollout 归入 **session store**。另一个 opt-in 的 `rollout-trace` 才是用于排障的 execution trace；二者不是同一种文件。 --- -## 二、先把五个容易混淆的概念拆开 +## 二、用生命周期和用途判断，不要用文件名判断 + +四个概念的最小边界如下： -一个 headless agent 通常同时需要下面五类输出。它们的数据有重叠，但生命周期、受众和兼容性承诺不同。 +| 概念 | 核心问题 | 典型生命周期 | 主要消费者 | 典型内容 | 是否用于 resume | 对应控制面 | +| --- | --- | --- | --- | --- | :-: | --- | +| Run output | 本次调用向外交付什么？ | 一次 run | 人、脚本、runner、SDK、UI | 最终回答、公开事件、状态、usage | 否 | `--output-format` | +| Execution trace | 运行时内部发生了什么？ | 一次 run、进程或 trace tree | 开发者、可观测平台 | 请求/响应、重试、span、内部工具与异常 | 否 | debug / trace / telemetry 配置 | +| Trajectory | agent 怎样完成这个任务？ | 一次 task / trial / run | benchmark、离线分析器 | observation、action、tool result、outcome、cost | 通常否 | `--trajectory` 或 adapter | +| Session | 下次从什么状态继续？ | 跨多次 run | agent 产品自身 | 可恢复 transcript、稳定 ID、分支、compaction、配置 | 是 | session / resume / persistence 配置 | -| 平面 | 主要消费者 | 典型载体 | 主要用途 | 是否要求可恢复会话 | -| --- | --- | --- | --- | :-: | -| CLI presentation | 人或一次性脚本 | stdout 文本 / 单个 JSON | 给出本次运行结果 | 否 | -| Live event protocol | runner、SDK、UI | stdout NDJSON | 实时观察工具调用、消息和用量 | 否 | -| Session store | agent 自己 | JSONL、SQLite、多文件目录 | continue / resume / fork / compaction | 是 | -| Benchmark trajectory | Harbor、离线分析器 | JSONL、ATIF 等 | 统计步数、token、成本和失败归因 | 通常否 | -| Diagnostics / telemetry | 开发者、可观测平台 | stderr、日志、span、trace 文件 | 排障、性能分析、运营监控 | 否 | +同一个 session 可以包含多次 run；每次 run 又各自产生自己的 output，并可选地记录 trace 和 trajectory： -这一区分解释了一个看似矛盾的事实：**agent 完全可以一边用 `stream-json` 向 stdout 发实时事件，一边把另一份更完整、可恢复的 session 写进自己的数据目录。** 前者是本次子进程的协议，后者是产品状态。 +```text +session s1 +├─ run r1：首次修复任务 +│ ├─ output o1 ───────────────> 当前调用方 +│ ├─ execution trace t1 ──────> 调试器 / tracing backend +│ └─ trajectory j1 ───────────> benchmark 产物 +└─ run r2：resume 后的追加要求 + ├─ output o2 + ├─ execution trace t2 + └─ trajectory j2 +``` -推荐的内部结构是一个事件源、多个投影器： +它们的数据会重叠，但兼容性承诺不同。一个合理实现可以从同一组内部事件投影出四种产物；这里的“投影”是指按各自用途筛选字段并转换结构，而不是复制全部内部事件： ```text ┌─ text renderer ───────────────> stdout agent loop ─> canonical ├─ final JSON reducer ──────────> stdout - events ├─ NDJSON event serializer ────> stdout - ├─ trajectory writer ───────────> requested file - ├─ session recorder ────────────> session store - └─ diagnostics / telemetry ─────> stderr / exporter + events ├─ JSONL event serializer ─────> stdout + ├─ trace recorder ──────────────> debug bundle / OTel + ├─ trajectory projector ────────> requested artifact + └─ session recorder ────────────> session store ``` -`--output-format` 只选择前三个 stdout 投影器之一；`--trajectory` 控制第四个 sink；未来如果做 resume，再单独设计第五个 session recorder。这样同一个参数就不会同时承担格式、开关和路径三种职责。 +前三条虽然可能输出不同数量的记录，但都属于 run output。`--output-format` 只能在这三条之间选择；它不能顺便改变 trace、trajectory 或 session 的持久化策略。 + +--- + +## 三、调研范围与核心结论 + +### 3.1 证据范围 + +调研前先尝试更新 `references/` 下的代码。四个可访问仓库已 fast-forward 到远端最新提交；Claude Code 的第三方镜像远端已经不可访问，未对它做强制替换。 + +| 项目 | 本地 revision | 更新结果 | 证据等级 | +| --- | --- | --- | --- | +| Grok Build | `19d42e35c07a9c9244f03f6df0c4c353f970d4f9` | 已更新 | xAI 官方开源仓库 | +| Pi | `c49906ec77788625aacbdc53ebca6fbe65bd20f5` | 已更新 | `references/pi` 所跟踪的公开仓库 | +| OpenCode | `e00890c67261a435cee6409366a68999a93393fd` | 已更新 | OpenCode 官方开源仓库 | +| Codex | `4f39251a010a8bd7d692d25fb33832ff06f1635a` | 已更新 | OpenAI 官方开源仓库 | +| Claude Code | `a371abbe75ffa0d0a3c92290e2bbf56a7ef54367` | 远端返回 `Repository not found`，保留快照 | **非官方 sourcemap 镜像，只用于辅助验证实现思路** | + +Claude Code 的正式契约以 Anthropic 当前的 [CLI reference](https://code.claude.com/docs/en/cli-usage)、[headless 文档](https://code.claude.com/docs/en/headless)和 [sessions 文档](https://code.claude.com/docs/en/sessions)为准。本地 `references/claude-code/README.md` 自己也明确说明它不是 Anthropic 官方项目，因此本文不会把该快照中的内部字段当作当前稳定 API。 + +### 3.2 核心结论 + +1. **`--output-format` 只选择公开 run output 的 stdout 表示形式。** 它不是文件重定向，也不控制 execution trace、trajectory 或 session。 +2. **`text`、`json` 和 `stream-json` 描述的是三种 stdout 传输格式。** `text` 是最终文本，`json` 是结束时的单个结果对象，`stream-json` 是运行期间逐行发布的 JSONL/NDJSON 事件。 +3. **`--trajectory PATH` 应是独立控制面。** 参数出现时为本次 run 生成一份以当前任务为边界的 trajectory，`PATH` 只指定产物位置，不改变 stdout。 +4. **trajectory 不是简化版 session。** trajectory 为评测解释任务路径；session 为产品恢复状态。要做 resume，必须另行设计稳定 ID、分支、compaction 和 schema migration。 +5. **五个被调研产品都有 session 实现，但都没有与本文建议完全等价的 benchmark `--trajectory PATH`。** benchmark 可以从公开事件流或 session 派生 trajectory，这不等于两者本来就是同一概念。 +6. **`json` 这个名字没有行业统一语义。** Claude Code 和 Grok 用它表示单个结果对象；Pi、Codex 和 OpenCode 用它表示 JSONL 事件流。因此不能只列枚举名，必须说明 stdout 的实际传输格式和输出语义。 --- -## 三、五个项目的接口全景 +## 四、五个项目的实现全景 -下表按**实际线协议**比较，而不是按各家的命名比较。 +### 4.1 向调用方输出什么：文本、单个结果对象或事件流 -| 项目 | 人类可读文本 | 单个汇总 JSON | JSONL 事件流 | partial / delta 能力 | 明确终止事件 | 持久会话 | -| --- | --- | --- | --- | --- | --- | --- | -| Pi | print mode | — | `--mode json` | 有 `message_update` delta | `agent_settled`；`agent_end` 只结束一次 low-level run | 默认 JSONL，支持 continue/resume/fork/`--no-session` | -| Claude Code | `--output-format text` | `--output-format json` | `--output-format stream-json` | 另加 `--include-partial-messages` | `result` | 默认 JSONL，支持 continue/resume/fork/`--no-session-persistence` | -| Codex | `codex exec` 默认 | — | `codex exec --json` | 没有公开 token delta 契约 | `turn.completed` / `turn.failed`；interrupt 是例外 | 默认 rollout JSONL，可 `--ephemeral`，支持 resume/fork | -| OpenCode | `opencode run --format default`，可能有多段已完成 text | `opencode export` 是另一个命令，不是 run 输出模式 | `opencode run --format json` | 否，只发较粗的完成事件 | 无，依赖 EOF + exit code | SQLite + 内部事件表，支持 continue/session/fork | -| Grok Build | `--output-format plain`，边生成边写 text chunk | `--output-format json` | `streaming-json`；另有 Messages 兼容流 | 原生流默认发 text/thought chunk；仅兼容流另加 partial flag，部分 delta 仍是粗粒度 | 原生流成功 `end`、失败 `error`；兼容流 `result` | 默认多文件 JSONL session，支持 continue/resume/fork | +先看每个项目提供哪些 **run output 形态**。下表按 stdout 中实际出现的内容比较，而不是按各家的格式名称比较；`—` 表示没有提供这种输出形态。 -从这张表能看出三个共同模式： +| 项目 | 给人读的文本 | run 结束后的一个结果 JSON | run 期间的逐行事件（JSONL/NDJSON） | +| --- | --- | --- | --- | +| Pi | print mode | — | `--mode json` | +| Claude Code | `--output-format text` | `--output-format json` | `--output-format stream-json` | +| Codex | `codex exec` 默认 | — | `codex exec --json` | +| OpenCode | `opencode run --format default`，可能有多段已完成 text | —；`opencode export` 是另一个 session 导出命令 | `opencode run --format json` | +| Grok Build | `--output-format plain`，边生成边写 text chunk | `--output-format json` | `streaming-json`；另有 Messages 兼容流 | + +再只看第三列的**公开事件流**。下面两个属性描述事件流的内容粒度与结束方式，并不是另外两种 output format。 + +| 项目 | 是否提前输出尚未完成的内容（partial / delta） | 是否用流内终止事件明确宣告整个 run 结束 | +| --- | --- | --- | +| Pi | 是；`message_update` 提供 delta，`message_end` 提供完整消息 | 是；`agent_settled`。`agent_end` 只结束一次 low-level run | +| Claude Code | 可选；另加 `--include-partial-messages` | 是；`result` | +| Codex | 否；没有公开的 token/text delta 契约 | 是；`turn.completed` / `turn.failed`，但中断是例外 | +| OpenCode | 否；只发较粗的完成事件 | 否；依赖 EOF + exit code | +| Grok Build | 是；原生流默认发 text/thought chunk，Messages 兼容流另有 partial flag | 是；原生流成功为 `end`、失败为 `error`，兼容流为 `result` | + +从这两张表能看出三个共同模式： - 给人看的模式倾向于只把最终答案放 stdout，把进度和诊断放 stderr。 - 给程序看的实时模式几乎都采用“一行一个对象”的 JSONL，而不是一个长寿命 JSON array。 -- session 通常自动持久化并按 session ID 管理；没有一家把 `--output-format` 当成 session 开关。 +- 没有一家把 `--output-format` 当成 session、trace 或 trajectory 的开关。 同时也有两个不能靠“行业惯例”猜出来的差异： - `json` 既可能是单个对象，也可能是 JSONL。Claude/Grok 属于前者，Pi/Codex/OpenCode 属于后者。 - “流式”既可能只表示**事件发生时立即发出**，也可能进一步包含不同粒度的 text/thinking/tool-argument delta。Claude 用额外 flag 开启 raw partial；Grok 的原生流默认已有 text/thought chunk，而额外 flag 只改变 Messages 兼容流的 framing。可见“流式”和“token 级”不是同一个承诺。 +### 4.2 Execution trace、trajectory 与 session + +下表比较另外三个平面。“没有专用 trajectory”表示没有面向 benchmark、以一次 task/run 为边界的稳定产物接口；不表示这些产品的数据不能被 adapter 转换成 trajectory。 + +| 项目 | Execution trace | 专用 benchmark trajectory | Session：存储与主要内容 | +| --- | --- | --- | --- | +| Pi | 隐藏的 `/debug` 可写 TUI 渲染行和最近发给模型的消息；不是稳定的 headless trace 协议 | 无；可从 JSON 事件流或 session export 派生 | 默认 JSONL；header，message，model/thinking change，compaction，branch/custom entry；`id`/`parentId` 形成树，支持 continue/resume/fork | +| Claude Code | `--debug` / `--debug-file` 记录诊断日志，另支持 telemetry；与 stdout output format 独立 | 无；Harbor 一类 runner 可解析 `stream-json` 后生成 | 默认 transcript JSONL；消息、工具交互及恢复元数据，支持 continue/resume/fork，可关闭 persistence | +| Codex | opt-in `rollout-trace` 本地 bundle：manifest、原始事件、prompt/response、工具与终端 payload，以及离线归约状态；另有 OpenTelemetry | 无；`rollout` 是 session，`rollout-trace` 是 debug trace，都不是 benchmark trajectory 接口 | 默认 `rollout-*.jsonl`；session metadata、model-visible message/reasoning、tool call/output 等可恢复项；支持 resume/fork，`--ephemeral` 关闭 | +| OpenCode | 有运行日志和 debug 子命令；未发现面向用户的完整、稳定 execution-trace 产物 | 无；`export` 导出的是 materialized session snapshot | 全局 SQLite 中的 session/message/part 及 durable event/projection；支持 continue/session/fork，大工具输出可单独落盘 | +| Grok Build | `RUST_LOG` 可把诊断写 stderr，`GROK_LOG_FILE` 可写文件；另有内部日志与 session trace export，均不属于 output format | 无与本文语义等价的 CLI 参数 | session 目录保存权威 updates、模型 chat history、summary/plan/compaction/subagent 等恢复状态；支持 continue/resume/fork | + +最值得记住的不是“各家都用了 JSONL”，而是每份数据被谁消费：公开稳定协议是 output，排障证据是 trace，任务评测路径是 trajectory，能恢复和分叉的产品状态是 session。 + --- -## 四、逐项目设计 +## 五、逐项目设计 + +### 5.1 Pi：`json` 就是事件流，session 是另一份树形 JSONL -### 4.1 Pi：`json` 就是事件流，session 是另一份树形 JSONL +**Run output。** Pi 的 headless 接口分三种模式： @@ -113,7 +278,17 @@ Pi 的 headless 接口分三种模式： Pi 还专门接管 stdout：协议写入走受控的 raw stdout，其他普通输出被导向 stderr，并处理写入背压。这说明“机器模式 stdout 不得混入日志”不是文档礼仪，而是实现层的边界。 -Pi 的 session 则是另一份 append-only JSONL。它有 session header，以及带 `id` / `parentId` 的 message、model change、compaction、branch 等 entry，因此历史本质上是一棵树，而不是 stdout 事件的逐字复制。切换分支只是移动当前 leaf，不会删除另一条分支；compaction 会改变送给模型的活动上下文，但不会抹掉原历史。 +**Session。** + +Pi 的 session 是另一份 append-only JSONL。它有 session header，以及带 `id` / `parentId` 的 message、model change、compaction、branch 等 entry，因此历史本质上是一棵树，而不是 stdout 事件的逐字复制。切换分支只是移动当前 leaf，不会删除另一条分支；compaction 会改变送给模型的活动上下文，但不会抹掉原历史。 + +**Execution trace。** + +Pi 的隐藏 `/debug` 主要写 TUI 渲染行和最近送给模型的消息，不是稳定的 headless trace。 + +**Trajectory。** + +Pi 没有专用 benchmark trajectory；评测方可以从 JSON 事件流或导出的 session 中选择 task steps，再转换成自己的 trajectory schema。 **可借鉴点：** @@ -124,11 +299,13 @@ Pi 的 session 则是另一份 append-only JSONL。它有 session header，以 源码入口：[JSON event stream 文档](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/json.md)、[RPC event reference](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/rpc.md)、[print mode](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/src/modes/print-mode.ts)、[session format](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/session-format.md)。 -### 4.2 Claude Code：最清楚地区分 text、json 与 stream-json +### 5.2 Claude Code：最清楚地区分 text、json 与 stream-json + +**Run output。** Claude Code 的公开定义最适合直接回答本文的命名问题： -| 格式 | 线协议 | +| 格式 | stdout 传输格式 | | --- | --- | | `text` | 完成后输出最终纯文本 | | `json` | 完成后输出**一个**结果对象，包含 result、session ID、用量/成本等元数据 | @@ -139,22 +316,34 @@ Claude Code 的公开定义最适合直接回答本文的命名问题： 它还有两个容易与 trajectory 混淆、实际完全不同的概念： - `--input-format stream-json` 控制 stdin；不会由 output format 隐式推导； -- `--replay-user-messages` 是双工客户端的输入确认回显，不是把旧 session trajectory 重放一遍。 +- `--replay-user-messages` 是双工客户端的输入确认回显，不是把旧 session 的执行历史重放到输出。 + +**Session。** Claude Code 默认把 session 保存为 `~/.claude/projects//.jsonl`。continue、resume 和 fork 操作的是这份持久状态；stdout 的 text/json/stream-json 只影响当前调用怎样向父进程报告。`--no-session-persistence` 也是独立开关。 本地非官方快照显示 transcript entry 带 parent UUID，并会把大 tool result 的完整内容单独落盘、在消息中留下 preview 和路径。这进一步说明 session 是可恢复的有向历史，不是 stdout event log；同时也说明 session 和 stream 都可能包含完整工具参数、结果、hook stdout/stderr，必须按敏感数据处理。 +**Execution trace。** + +`--debug` / `--debug-file` 记录的是诊断日志，不受 output format 控制。 + +**Trajectory。** + +Claude Code 没有专用 trajectory 路径参数；Harbor 解析 `stream-json` 后生成 benchmark 记录，是“从 run output 投影 trajectory”，不是把 stdout 流重新命名为 trajectory。 + **可借鉴点：** -- 三种输出名与线协议一一对应，歧义最小； +- 三种输出名与 stdout 传输格式一一对应，歧义最小； - partial delta 是独立能力，不绑死在 `stream-json` 名字上； - terminal `result` 汇总状态、最终答案、session ID、turn、usage/cost，消费者不必自己扫描整条流算最终结果； - session persistence 与 stdout representation 完全正交。 正式契约：[CLI reference](https://code.claude.com/docs/en/cli-usage)、[headless mode](https://code.claude.com/docs/en/headless)、[sessions](https://code.claude.com/docs/en/sessions)、[custom session storage](https://code.claude.com/docs/en/agent-sdk/session-storage)。 -### 4.3 Codex：默认文本、`--json` JSONL、rollout 自动持久化 +### 5.3 Codex：默认文本、`--json` JSONL、rollout 自动持久化 + +**Run output。** `codex exec` 没有 `--output-format`： @@ -176,6 +365,8 @@ error item 再用 `type` 区分 agent message、reasoning、command execution、file change、MCP/collab tool、web search、todo 和 error。成功运行通常以 `turn.completed` 收尾，里面带 usage；失败以 `turn.failed` 收尾。当前实现的 interrupted 分支可能没有终止 JSON 事件，因此可靠的 runner 仍应同时检查 EOF、exit code/signal 和 stderr。 +**Session。** + 普通 session 会自动写到： ```text @@ -184,18 +375,28 @@ $CODEX_HOME/sessions/YYYY/MM/DD/rollout--.jsonl 每行外层带 timestamp、ordinal、type 和 payload，写一行 flush 一次。resume 会继续 append 原 rollout；fork 创建新 thread ID，并把继承历史物化到一份新的 rollout。持久化 policy 会保留可恢复所需的 message、reasoning、tool call/output 等，但过滤许多 transient delta、begin、warning 和 UI event，所以 rollout 也不是 stdout JSONL 的镜像。 -Codex 还区分了更深的 opt-in `rollout-trace`：它可能记录 prompt、response、工具 I/O 和终端输出，用于本地排障，不用于 resume，也不是稳定的 CLI trajectory 接口。再往外还有 OpenTelemetry。这三个名字相近，目的完全不同。 +**Execution trace。** + +Codex 还区分了 opt-in `rollout-trace`。它在本地 bundle 中保存 `manifest.json`、有序原始事件、prompt/response、工具 I/O、终端输出和 payload 引用，并可离线归约成供调试器查看的语义图。它有独立的 `trace_id`，同时引用被观察 session 的 `rollout_id`，正面说明 trace identity 与 session identity 不应混为一谈。 + +`rollout-trace` 用于本地排障，不用于 resume；OpenTelemetry 又是另一条可观测出口。 + +**Trajectory。** + +Codex 没有专用 benchmark trajectory。session rollout 与 `rollout-trace` 都不是稳定的 benchmark trajectory 接口；评测方需要从公开 JSONL、session rollout 或 trace 中另做投影。 **可借鉴点：** - 默认模式严格执行“stdout 结果、stderr 诊断”； -- `--output-last-message` 展示了“额外 artifact sink”不必改变主输出格式； +- `--output-last-message` 展示了“额外文件出口”不必改变主输出格式； - rollout 使用 ordinal、逐行 flush，并能修复缺少结尾换行的 torn tail； - JSONL 是 SDK 实际消费的公开集成面，但 schema 没有版本号，消费者仍要容忍未知事件、item 类型和新增字段。 -源码入口：[exec CLI](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/cli.rs)、[stdout contract](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/lib.rs)、[exec events](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/exec_events.rs)、[rollout recorder](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout/src/recorder.rs)。 +源码与正式契约：[OpenAI CLI reference](https://developers.openai.com/codex/cli/reference)、[exec CLI](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/cli.rs)、[stdout contract](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/lib.rs)、[exec events](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/exec_events.rs)、[rollout recorder](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout/src/recorder.rs)、[rollout trace](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout-trace/README.md)。 + +### 5.4 OpenCode：run 的 `json` 是较粗的 JSONL，session 存在 SQLite -### 4.4 OpenCode：run 的 `json` 是较粗的 JSONL，session 存在 SQLite +**Run output。** OpenCode 的命令是： @@ -213,19 +414,31 @@ default 格式会把每个已完成的 assistant text part 写到 stdout；一 2. schema 未版本化，且是从内部 session event 临时投影出来的； 3. `--format json` 的名字看不出它是流。 +**Session。** + OpenCode 的持久层与 CLI 流差别更大。session/message/part，以及内部 durable event/projection，主要存在全局 SQLite 数据库；`--continue`、`--session` 和 `--fork` 决定加载或复制哪份 session。`opencode export [sessionID]` 是另一个命令，它把 materialized session snapshot 作为一个 pretty JSON 写到 stdout，用户再自行用 Shell 重定向；它不能当作 `run --format json` 的最终对象模式。 大工具输出会截断 preview，把全文放到 data 目录的 tool-output 文件中。stream、session 和 export 默认都不能被视作已经做过完整 secret redaction；`export --sanitize` 也只是有限清洗。 +**Execution trace。** + +OpenCode 有常规运行日志和一组面向配置、LSP、文件、snapshot 等问题的 debug 子命令，但没有公开的完整 execution-trace 产物。 + +**Trajectory。** + +OpenCode 没有专用 benchmark trajectory。`opencode export` 的对象仍是 session snapshot；只有 adapter 把其中 task steps 转成评测 schema 后，结果才是 trajectory。 + **可借鉴点：** - CLI 流可以只投影对集成方有用的语义事件，不必泄露内部所有 event； -- session 存储实现可以是 SQLite，wire protocol 仍然可以是 JSONL； +- session 存储实现可以是 SQLite，对外传输格式仍然可以是 JSONL； - 反面经验是：terminal footer 和 schema version 很便宜，却能显著降低 runner 的状态推断成本。 源码入口：[run command](https://github.com/anomalyco/opencode/blob/e00890c67261a435cee6409366a68999a93393fd/packages/opencode/src/cli/cmd/run.ts)、[session tables](https://github.com/anomalyco/opencode/blob/e00890c67261a435cee6409366a68999a93393fd/packages/core/src/session/sql.ts)、[export command](https://github.com/anomalyco/opencode/blob/e00890c67261a435cee6409366a68999a93393fd/packages/opencode/src/cli/cmd/export.ts)。 -### 4.5 Grok Build：最完整的输出矩阵，以及双份 session 日志 +### 5.5 Grok Build：最完整的输出矩阵，以及多层 session 状态 + +**Run output。** Grok Build 提供四种 headless 格式： @@ -241,10 +454,12 @@ Grok Build 提供四种 headless 格式： 这个矩阵展示了两条不同的扩展轴： - **时间轴：**最终结果对象，还是实时事件流； -- **schema 轴：**agent 自己的语义事件，还是某个外部生态的兼容 wire format。 +- **schema 轴：**agent 自己的语义事件，还是某个外部生态的兼容传输格式（wire format）。 它也展示了兼容层的代价：同一个内部事件要维护两套公开投影，某些内部状态无法无损映射，partial framing、usage、error 和 terminal result 都要各自定义。nanoPyCodeAgent 在有具体消费者之前不需要复制这套复杂度。 +**Session。** + Grok 的 session 默认存在 `~/.grok/sessions///`，其中至少区分： - `updates.jsonl`：恢复 UI/conversation 的权威 session updates； @@ -253,6 +468,14 @@ Grok 的 session 默认存在 `~/.grok/sessions///`， JSONL writer 使用 owner-only 目录、append 和 torn-tail 修复。continue/resume/fork 操作这份 session；output format 仍然只是当前 headless 调用的 stdout 选择。 +**Execution trace。** + +headless 模式可通过 `RUST_LOG` 把诊断日志写到 stderr，或通过 `GROK_LOG_FILE` 写文件；产品数据目录还包含内部日志和 session trace exports。这些用于排障或 session 分析，不是 output format。 + +**Trajectory。** + +Grok Build 没有形成与本文建议等价的 benchmark `--trajectory PATH`；评测方仍需从公开流、session 或 trace export 中转换。 + Grok 的原生 `json` / `streaming-json` 对 usage/cost 还有一个值得借鉴的规则：服务端没有完整上报成本时就省略 cost，或标记 incomplete，而不是把缺失值写成 0。Messages 兼容流受目标 schema 约束，有些未知值仍会回落到 0，并在文档中明确 caveat。对 nanoPyCodeAgent 自己可控的原生 benchmark 协议来说，“未知”与“免费”必须是两个状态。 **可借鉴点：** @@ -266,7 +489,7 @@ Grok 的原生 `json` / `streaming-json` 对 usage/cost 还有一个值得借鉴 --- -## 五、`stream-json` 到底是什么 +## 六、`stream-json` 到底是什么 建议把 `stream-json` 的正式定义写成： @@ -282,7 +505,7 @@ Grok 的原生 `json` / `streaming-json` 对 usage/cost 还有一个值得借鉴 {"schema_version":1,"type":"run.completed","run_id":"r1","sequence":4,"timestamp":"2026-08-22T10:00:02.010Z","status":"completed","result":"Done.","turns":1,"usage":{"input_tokens":120,"output_tokens":8}} ``` -### 5.1 它与普通 JSON 的区别 +### 6.1 它与普通 JSON 的区别 | 维度 | `json` | `stream-json` | | --- | --- | --- | @@ -293,9 +516,9 @@ Grok 的原生 `json` / `streaming-json` 对 usage/cost 还有一个值得借鉴 | 中断后产物 | 整个文档可能无效或根本没写 | 之前的完整行仍可解析，但必须结合 exit code 判断未正常结束 | | 最适合 | Shell 脚本、CI 读取一次结果 | runner、实时 UI、Harbor adapter、长任务观测 | -它通常也叫 **NDJSON** 或 **JSON Lines / JSONL**。这里推荐 CLI 枚举名用 `stream-json`，文档里明确“wire format is NDJSON”，避免用户把 `jsonl` 误解成只适用于文件。 +本文将这种格式称为 **JSON Lines（JSONL）**；它也常称 **Newline-Delimited JSON（NDJSON）**。两个名字都强调记录由换行分隔，并不表示它只能保存在文件里。这里推荐 CLI 枚举名用 `stream-json`，再在文档中明确传输格式是 JSONL/NDJSON。 -### 5.2 它不自动承诺什么 +### 6.2 它不自动承诺什么 `stream-json` 不自动意味着： @@ -310,9 +533,9 @@ Grok 的原生 `json` / `streaming-json` 对 usage/cost 还有一个值得借鉴 --- -## 六、`--trajectory PATH` 应该是什么语义 +## 七、`--trajectory PATH` 应该是什么语义 -五个项目大多没有一个同名 flag，因为它们默认拥有产品级 session store：Pi、Claude、Codex 和 Grok 自动保存 JSONL，OpenCode 自动保存 SQLite。用户通过 session ID continue/resume/fork，而不是每次指定一个 trajectory 文件。 +五个项目都没有与本文语义完全相同的 `--trajectory PATH`。这不是因为 session 可以替代 trajectory，而是因为这些产品首先解决的是交互式继续工作：Pi、Claude、Codex 和 Grok 会持久化 session 文件或目录，OpenCode 使用 SQLite；用户用 session ID continue/resume/fork。需要 benchmark trajectory 时，集成方通常再从公开事件流或 session 中投影。 nanoPyCodeAgent 当前还没有这样的 session 系统。在这个前提下，建议： @@ -322,8 +545,8 @@ nanoPyCodeAgent 当前还没有这样的 session 系统。在这个前提下， 同时表达两件紧密相关、不会互相冲突的事： -1. **presence enables：**出现该参数才开启本次运行的 trajectory artifact； -2. **value chooses destination：**`PATH` 是该 artifact 的文件路径。 +1. **presence enables：**出现该参数才生成本次运行的 trajectory； +2. **value chooses destination：**`PATH` 是该 trajectory 的文件路径。 它**不应该**： @@ -346,7 +569,7 @@ nanoPyCodeAgent -p "fix it" --output-format stream-json > events.jsonl nanoPyCodeAgent -p "fix it" --output-format text --trajectory run.jsonl ``` -第二条和第三条产生的 JSONL **不必相同**：公开 stream 追求稳定、小而安全；trajectory 可以有更完整的归因字段和截断元数据。两者应从同一个 canonical event model 投影，避免事实不一致。 +第二条和第三条产生的 JSONL **不必相同**：公开 stream 追求稳定、小而安全；trajectory 可以有更完整的归因字段和截断元数据。两者应从同一组内部标准事件投影，避免事实不一致。 路径契约还应明确： @@ -357,13 +580,13 @@ nanoPyCodeAgent -p "fix it" --output-format text --trajectory run.jsonl - reader 应容忍 crash 留下的最后一个不完整行，但不能悄悄忽略中间坏行； - 不建议允许 `--trajectory -`，否则它会与选定的 stdout formatter 争用同一个协议通道。 -如果未来改成**默认自动持久化 session**，含义应重新拆分：`--no-session-persistence` 控制是否保存，`--session`/`--resume` 控制身份，只有在确实允许覆盖默认位置时才引入 `--session-path`。不要悄悄把今天的 debug trajectory 升格为明天的 resume 格式。 +如果未来增加**默认自动持久化 session**，含义应继续分开：`--no-session-persistence` 控制是否保存，`--session`/`--resume` 控制身份，只有在确实允许覆盖默认位置时才引入 `--session-path`。不要把今天的 benchmark trajectory 悄悄升格为明天的 resume 格式。execution trace 也应由单独的 debug/trace 配置控制，不能借用 `--trajectory`。 --- -## 七、给 nanoPyCodeAgent 的建议契约 +## 八、给 nanoPyCodeAgent 的建议契约 -### 7.1 CLI +### 8.1 CLI 建议把原来的两档扩成三档： @@ -374,16 +597,17 @@ nanoPyCodeAgent [-p PROMPT | --prompt-file PATH | stdin] [--trajectory PATH] ``` -| 选项 | stdout 契约 | 典型消费者 | +| `--output-format` 值 | stdout 契约 | 典型消费者 | | --- | --- | --- | | `text`（默认） | 仅最终助手文本；空结果允许空 stdout | 人、最简单的 benchmark runner | | `json` | run 初始化后恰好一个 result object；preflight 失败可空 stdout；不夹日志 | Shell、CI、一次性脚本 | | `stream-json` | 每行一个事件；可优雅收尾时以 terminal event 结束，否则 EOF + 非零退出/信号表示 aborted stream | Harbor adapter、SDK、实时 UI | -| `--trajectory PATH` | 不改变 stdout；另写一份增量 JSONL | 离线归因、benchmark 报表、debug | + +`--trajectory PATH` 不属于这张 output-format 表。它保持上述 stdout 契约不变，另外写一份供 benchmark、离线统计和失败归因使用的增量 JSONL。 诊断、重试提示、traceback 和人类进度一律写 stderr。API 错误原文仍可出现在 stderr，满足 Harbor 的错误分类要求；机器模式 stdout 必须始终保持可解析。 -### 7.2 单个 `json` 结果对象 +### 8.2 单个 `json` 结果对象 建议至少包含： @@ -408,9 +632,9 @@ nanoPyCodeAgent [-p PROMPT | --prompt-file PATH | stdin] usage/cost 缺失时应省略或明确标记 `usage_incomplete: true`，不能用 0 代替未知。Grok 的完整性规则在这里比“始终填满一张数字表”更适合 benchmark。 -如果未来增加 `--output-schema PATH`，它应该约束 `result` 的语义内容，而不是改变上述 transport envelope。Codex 和 Grok 都把“模型结构化回答”与“CLI 输出协议”分开，这是正确边界。 +如果未来增加 `--output-schema PATH`，它应该约束 `result` 的语义内容，而不是改变 CLI 结果对象的外层结构。Codex 和 Grok 都把“模型结构化回答”与“CLI 输出协议”分开，这是正确边界。 -### 7.3 `stream-json` 最小事件集 +### 8.3 `stream-json` 最小事件集 第一版不必复制五家全部事件。建议最小集是： @@ -445,7 +669,7 @@ run.failed - tool output 过大时记录 preview、原始大小、truncated 标志；是否另存全文要有明确路径和清理策略； - token delta 以后用 `assistant.delta` 或 partial flag 增加，不能让 `assistant.message` 同时表示 delta 和 final snapshot。 -### 7.4 trajectory 内容与稳定性 +### 8.4 trajectory 内容与稳定性 trajectory 的目标是“解释这次运行为什么得到这个结果”，至少应能重建： @@ -456,13 +680,13 @@ trajectory 的目标是“解释这次运行为什么得到这个结果”，至 - compaction/truncation 的发生和被省略内容的大小； - 最终 status 与 result。 -但第一版应明确标记为 **diagnostic/benchmark artifact, not resumable session**。要支持 resume，还需要稳定 parent/entry ID、分支语义、模型/工具配置迁移、compaction 后上下文恢复和长期 schema migration；Pi、Claude、Codex、OpenCode、Grok 的 session 实现都证明这远不只是“读回 JSONL 再继续”。 +但第一版应明确标记为：**供 benchmark/analysis 使用，不是 execution trace，也不是 resumable session。** 它不需要收集 provider 重试、内部队列和异常栈等全部调试细节；这些属于 execution trace。要支持 resume，还需要稳定 parent/entry ID、分支语义、模型/工具配置迁移、compaction 后上下文恢复和长期 schema migration；Pi、Claude、Codex、OpenCode、Grok 的 session 实现都证明这远不只是“读回 JSONL 再继续”。 Harbor 需要 ATIF 时，建议在 adapter 层把 native trajectory 转成 ATIF，而不是让 agent loop 直接依赖 benchmark schema。只有当 Harbor 成为唯一主要消费者时，才值得考虑把 ATIF 直接作为持久格式。 -### 7.5 安全与数据量 +### 8.5 安全与数据量 -五个项目的 session/stream 都可能保存或输出 user prompt、reasoning、tool arguments、文件内容、命令输出、环境路径和 provider metadata。部分项目会清洗命令中的可识别 secret 或截断大结果，但没有一个通用保证能把所有秘密洗掉。 +五个项目的 trace、trajectory、session 和公开 stream 都可能保存或输出 user prompt、reasoning、tool arguments、文件内容、命令输出、环境路径和 provider metadata。部分项目会清洗命令中的可识别 secret 或截断大结果，但没有一个通用保证能把所有秘密洗掉。 因此应把 trajectory 当作敏感文件： @@ -476,9 +700,9 @@ Harbor 需要 ATIF 时，建议在 adapter 层把 native trajectory 转成 ATIF --- -## 八、最终建议 +## 九、最终建议 -对本文开头三个问题，最终答案是： +对本文开头四个概念，最终边界是： ```text --output-format text|json|stream-json @@ -504,14 +728,26 @@ stream-json **运行期间输出 NDJSON：一行一个完整事件对象；能优雅收尾时最后一行是 terminal result，异常中止时则可能只有一个仍可解析的完整前缀。** 它与 `json` 的差别是“单个最终快照”对“可增量消费的事件序列”，而不是“是否写文件”。 -这套定义最接近 Claude Code 和 Grok 的清晰命名，同时吸收 Pi 的 delta/final 分工、Codex 的 stdout/stderr 边界和额外 artifact sink、OpenCode 的语义投影，以及各家 session 与公开事件流分离的共同设计。 +```text +execution trace +``` + +**由独立的 debug/trace 配置开启，服务排障和可观测性。** 它可以比公开 output 和 trajectory 更详细、更敏感，也不承诺可用于 resume。 + +```text +session +``` + +**由独立的 persistence/session/resume 接口管理，服务跨 run 的继续、分叉和压缩。** Codex 的 `rollout-*.jsonl` 属于这一类；文件名中的 rollout 不会把它变成 trajectory。 + +这套定义最接近 Claude Code 和 Grok 的清晰命名，同时吸收 Pi 的 delta/final 分工、Codex 的 stdout/stderr 边界和额外文件出口、OpenCode 的语义投影，以及各家 session 与公开事件流分离的共同设计。 --- -## 九、参考入口 +## 十、参考入口 - Pi：[usage](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/usage.md)、[JSON event stream](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/json.md)、[RPC events](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/rpc.md)、[sessions](https://github.com/earendil-works/pi/blob/c49906ec77788625aacbdc53ebca6fbe65bd20f5/packages/coding-agent/docs/sessions.md) - Claude Code：[CLI reference](https://code.claude.com/docs/en/cli-usage)、[headless mode](https://code.claude.com/docs/en/headless)、[sessions](https://code.claude.com/docs/en/sessions)、[session storage](https://code.claude.com/docs/en/agent-sdk/session-storage) -- Codex：[exec CLI](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/cli.rs)、[exec events](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/exec_events.rs)、[rollout recorder](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout/src/recorder.rs) +- Codex：[OpenAI CLI reference](https://developers.openai.com/codex/cli/reference)、[exec CLI](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/cli.rs)、[exec events](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/exec/src/exec_events.rs)、[rollout recorder](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout/src/recorder.rs)、[rollout trace](https://github.com/openai/codex/blob/4f39251a010a8bd7d692d25fb33832ff06f1635a/codex-rs/rollout-trace/README.md) - OpenCode：[run command](https://github.com/anomalyco/opencode/blob/e00890c67261a435cee6409366a68999a93393fd/packages/opencode/src/cli/cmd/run.ts)、[export command](https://github.com/anomalyco/opencode/blob/e00890c67261a435cee6409366a68999a93393fd/packages/opencode/src/cli/cmd/export.ts) - Grok Build：[headless guide](https://github.com/xai-org/grok-build/blob/19d42e35c07a9c9244f03f6df0c4c353f970d4f9/crates/codegen/xai-grok-pager/docs/user-guide/14-headless-mode.md)、[format enum](https://github.com/xai-org/grok-build/blob/19d42e35c07a9c9244f03f6df0c4c353f970d4f9/crates/codegen/xai-grok-pager/src/headless/cli.rs)、[session export contract](https://github.com/xai-org/grok-build/blob/19d42e35c07a9c9244f03f6df0c4c353f970d4f9/crates/codegen/xai-grok-shell/src/session/export.rs)