diff --git a/devlog/_plan/260826_cursor_responses_gap/090_gap8_codex_exec_qa.md b/devlog/_plan/260826_cursor_responses_gap/090_gap8_codex_exec_qa.md new file mode 100644 index 0000000000..65b8e9dfef --- /dev/null +++ b/devlog/_plan/260826_cursor_responses_gap/090_gap8_codex_exec_qa.md @@ -0,0 +1,42 @@ +# 090 — gap-8 silent-redirect + codex exec adversarial QA (w2) + +Service: gap-8 stack, restarted per round (final pid after d9752bc0e). +Instrument: `codex exec -m cursor/grok-4.6` non-interactive, scratch cwds +under /tmp/ocx-qa-JStBBM. Transcripts: s1.log-s5b.log in that scratch. + +## Results + +| Scenario | Round | Result | Evidence | +|---|---|---|---| +| S1 ten tool calls | 1 | PASS | 10/10 separate bridge execs (pwd,ls,date,whoami,hostname,uname-s,ls,id,echo HOME,uname-m); narration grep hits=0 | +| S2 native-tool bait | 1 | INCONCLUSIVE | run produced no agent output (0-byte response; separate G2-class incident) | +| S2b native-tool bait retry | 2 | PASS* | zero 차단/전환 narration; both requests answered via bridge on first attempt. Residual: model duplicated its commentary line + repeated the 2-call batch twice (double-batch echo, no user-visible harm) | +| S3 mixed read/edit | 1 | PASS | notes.txt ALPHA->BETA lifecycle completed; narration=0 | +| S4 5-step chain | 1-2 | FAIL | stalled after step 1-2, files missing; "이전 호출은 출력이 비어 있어... 처음부터" restart loop observed | +| repair: extend empty-result normalization to codex CLI native names (shell/local_shell/container.exec) | commit d9752bc0e | — | root cause: gap-7 fix keyed on bridge tool NAMES; codex exec advertises the native `shell` tool, so empty results were unexplained again | +| S4d 5-step chain re-run | 3 | PASS* | all 5 steps done, data.csv+avg.txt=84 correct; restart-narration grep=0; model still self-recovered from two empty-looking intermediate results but WITHOUT surface-switch framing and completed | +| S5 image via /v1/responses | 1 | PASS | 32x32 red PNG -> "Red", in=12036 | +| S5b image via codex exec -i | 1 | PASS | blue.png -> "파랑", tokens 45489 | + +## Verdicts + +- Silent-redirect (gap-8 core): narration '차단/전환/막혀' = 0 across all + passing rounds; native-bait prompt answered bridge-first without + announcing a switch. Fix effective. +- Empty-result loop: root-caused twice (bridge names at gap-7, codex CLI + native names at gap-8 QA round 2); after d9752bc0e the 5-step chain + completes. Residual: intermediate tool results still occasionally + ARRIVE empty on the wire (model sees nothing and retries once) — that + delivery gap is the remaining G2-class defect, now non-fatal because + the retry succeeds without derailing. +- Image input: healthy at both API and codex exec layers. The reported + app-session failure (repeated "Viewed an image" then giving up) did + not reproduce here; needs an app-session capture with the actual + clipboard file — recorded as UNKNOWN with repro steps pending. + +## Open follow-ups + +1. Wire-level empty tool-result delivery (why some exec outputs arrive + blank upstream) — needs protobuf frame capture of an affected round. +2. App-session image loop repro. +3. S2b double-batch echo (duplicate commentary + repeated batch). diff --git a/devlog/_plan/260826_cursor_responses_gap/100_wire_ndjson_qa.md b/devlog/_plan/260826_cursor_responses_gap/100_wire_ndjson_qa.md new file mode 100644 index 0000000000..12f7648b23 --- /dev/null +++ b/devlog/_plan/260826_cursor_responses_gap/100_wire_ndjson_qa.md @@ -0,0 +1,45 @@ +# 100 — Wire/NDJSON QA (n1, 2026-08-26 오후) + +Service: gap-8 stack (ecc9aad0d). Captures: /tmp/ocx-wire/*.sse. +NDJSON: ~/.opencodex/usage.jsonl (31,466 lines). + +## Scenario table + +| # | Scenario | Result | Wire evidence | +|---|---|---|---| +| W1 | plain stream | PASS | created -> in_progress (gap-1 live) -> deltas -> completed; 29 lines | +| W2 | single tool stream | PASS | function_call args delta/done; single commentary line, no duplicate emission | +| W3 | parallel-10 stream | PASS | 11 output_item.added (1 msg + 10 calls, seq 2..47), each probe exactly once — the earlier "22" was event+data line double-count, NOT a wire duplicate. Double-batch echo NOT reproduced at the wire; 090 S2b echo attributed to model-side commentary repetition, not stream duplication | +| W4 | empty tool-result round trip | PASS* | model received empty output and answered "The tool returned this, verbatim: (blank)" — honest handling; 112 text deltas, no retry spiral at API layer | +| W5 | single image stream | PASS | "Green.", 38 lines | +| W6 | 3x same-color images | PASS* | "lime green, yellow, red" — model hallucinated variety on identical images | +| W6b | 3x distinct images (R,G,B) | PASS | "red, green, blue" correct order; 30.1s wall, in=13575 — slow but correct. App image loop NOT reproduced at API layer | +| W7 | kimi-k3-1m stream | PASS | full clean sequence incl. in_progress | +| W8 | apply_patch custom stream | PASS | custom_tool_call_input delta/done, valid envelope | + +## NDJSON (usage.jsonl) integrity — last 300 rows + +- 0 malformed lines; cursor rows 238. +- usageStatus: estimated 235, unreported 1 (the single 502 row — 10.8s + upstream failure, usage honestly unreported, no fake zeros). +- true zero-input rows: 1 (= the 502). Schema: nested usage{inputTokens...}, + tierOutcome, firstOutputMs, requestedEffort all populated. +- Verdict: NDJSON pipeline healthy; no corruption class found. + +## 090 follow-up dispositions + +1. Wire-level empty exec output: NOT reproduced in W4 (deliberate empty + round-trips handled honestly). Remaining suspicion narrows to the + in-session (checkpoint/replay-depth) path, not the API surface — + signature still open, now bounded to multi-round sessions. +2. Double-batch echo: wire ruled out (W3); model-side commentary + repetition under replay confusion — folds into G1 umbrella. +3. App image loop: API layer healthy (W5/W6b, codex exec -i PASS in 090 + S5b). Bounded to Codex-app-side attachment handling; needs app-session + capture — out of this repo's fix surface for now. + +## Verdict + +No new adapter-fixable defect surfaced in this round; gap-9 not needed. +All three follow-ups bounded with evidence; W6 same-color hallucination is +MODEL-class. Campaign continues to be green on the gap-8 stack. diff --git a/devlog/_plan/260826_cursor_responses_gap/110_app_route_qa.md b/devlog/_plan/260826_cursor_responses_gap/110_app_route_qa.md new file mode 100644 index 0000000000..1b7ebfa125 --- /dev/null +++ b/devlog/_plan/260826_cursor_responses_gap/110_app_route_qa.md @@ -0,0 +1,59 @@ +# 110 — App-route QA session + planning-time narration (m1) + +Instrument: self-created Codex app thread 01a03c74-3d91-74f3-b060-03f521e83c6c +(cursor/grok-4.6 high, projectless cursor-app-qa), created via +codex_app__create_thread from this session — the first app-route QA driven +entirely by the agent. + +## Result + +- Terminal: APP_QA_RESULT: PASS (5 read-only calls + qa.txt LINE1/LINE2 + lifecycle completed). +- Cost signatures: 13 commandExecutions for a ~8-call task; 21 + switch-mentions; 5 empty-output mentions; file writes done via shell + printf instead of apply_patch. + +## Mechanism finding (key) + +The 차단/전환 narration originates at PLANNING time: reasoning contains +"Shell 도구를 사용해 5개의 독립적인 명령을 한 번에 실행할 것이다" and +"Shell이 차단되어 브리지로 전환" BEFORE any denial payload arrives in that +round — the pattern is replayed-history-driven (G1 class), not a reaction +to our gap-8 denial text. Response-layer rewrites therefore reduce +narration in fresh sessions (codex exec rounds: 0 hits) but cannot zero it +in app sessions whose history already contains the pattern. + +User screenshot (13:5x) independently confirms: same-day app session still +narrates 차단/전환 and fills batches via bridge after a native probe. + +## Increment shipped (this commit) + +- Guidance note: "Tool-selection commentary is forbidden — FIRST visible + action is the bridge call itself; 차단/전환/blocked/switching must not + appear for tool-routing reasons." +- Guidance note: shell-redirection file writes forbidden while + apply_patch/structured-edit advertised (printf/echo >, heredoc, sed -i). +- Tests extended (cursor-tool-definitions 26 pass). + +## Honest bound + +Zeroing app-session narration requires the G1 replay-representation line: +tool-suspended checkpoints (gap-3) engaging in real app threads so history +stops replaying the old pattern. Until the stack lands and sessions turn +over, existing threads keep echoing it. Re-probe after service repair +recorded below. + +## Re-probe after service repair (7b3dbdd55) + +- rp1 (10-tool scenario, fresh codex exec): 10 separate bridge execs, + narration grep = 0. Tool-selection-commentary suppression holding in + fresh sessions. +- rp2 (file-edit scenario): content correct, but the model still wrote + result.md via `printf > ` — apply_patch was NOT used. Signature check: + 0 apply_patch mentions in the transcript, meaning the codex exec + session's advertised catalog exposes the bridge exec but the model + never considered the edit path despite the new note. Disposition: + guidance alone insufficient for write-routing in exec-style sessions; + candidate follow-up is server-side detection of redirection-writes with + a redirect-to-apply_patch tool error (deferred — aggressive, needs its + own cycle and risk review; recorded as open). diff --git a/src/adapters/cursor/native-exec-fs.ts b/src/adapters/cursor/native-exec-fs.ts index 18735c3c28..265bfc7b92 100644 --- a/src/adapters/cursor/native-exec-fs.ts +++ b/src/adapters/cursor/native-exec-fs.ts @@ -43,11 +43,11 @@ function codexNativeMutationRefusal(operation: "write" | "delete", structuredEdi const structuredHint = structuredEditAvailable ? " Use the structured edit tools (`edit_file` / `multi_edit`) or the `apply_patch` tool for file edits so Codex can approve the change, enforce sandbox policy, show diffs, and record rollout." : " Use the `apply_patch` tool for file edits so Codex can approve the change, enforce sandbox policy, show diffs, and record rollout."; - return `Cursor-native ${operation} is disabled for this Codex request because apply_patch is available.${structuredHint} No file was changed.`; + return `Make this ${operation} through the Codex edit path instead.${structuredHint} No file was changed. Do NOT narrate this redirect or comment on tool availability — just make the edit call.`; } const NATIVE_LOCAL_EXEC_DISABLED = - "Cursor-native filesystem tools are not executed locally. Use a catalog tool for this work: `shell_command` / `exec_command` (or the listed `mcp_opencodex-responses_*` display alias) with host-shell-safe equivalents: POSIX (`cat`, `head`, `ls`, `rg`, `grep`) or Windows PowerShell (`Get-Content`, `Get-ChildItem`, `Select-String`); use `apply_patch` for file edits."; + "Re-issue this operation NOW through the catalog shell tool (`shell_command` / `exec_command`, or the listed `mcp_opencodex-responses_*` display alias) with the host-shell-safe equivalent: POSIX (`cat`, `head`, `ls`, `rg`, `grep`) or Windows PowerShell (`Get-Content`, `Get-ChildItem`, `Select-String`); use `apply_patch` for file edits. Do NOT narrate this redirect, do NOT comment on tool availability, and do NOT re-announce the task — just make the bridge call."; export function rejectReadExecForPolicy(execMsg: ExecServerMessage): Uint8Array { if (execMsg.message.case !== "readArgs") throw new Error("invalid read exec"); diff --git a/src/adapters/cursor/native-exec-network.ts b/src/adapters/cursor/native-exec-network.ts index 39483987ec..f259f284ef 100644 --- a/src/adapters/cursor/native-exec-network.ts +++ b/src/adapters/cursor/native-exec-network.ts @@ -7,7 +7,7 @@ export interface CursorNativeNetworkDeps { } const NATIVE_FETCH_DISABLED = - "Cursor-native fetch is not executed locally. Use the Codex shell bridge tool `shell_command` (aliases: `exec_command`, `mcp_opencodex-responses_shell_command`, `mcp_opencodex-responses_exec_command`) with curl or wget."; + "Re-issue this fetch NOW through the catalog shell tool `shell_command` (aliases: `exec_command`, `mcp_opencodex-responses_shell_command`, `mcp_opencodex-responses_exec_command`) with curl or wget. Do NOT narrate this redirect or comment on tool availability — just make the bridge call."; export function rejectFetchExecForPolicy(execMsg: ExecServerMessage): Uint8Array { if (execMsg.message.case !== "fetchArgs") throw new Error("invalid fetch exec"); diff --git a/src/adapters/cursor/native-exec-shell.ts b/src/adapters/cursor/native-exec-shell.ts index a708117cb6..6fb7688302 100644 --- a/src/adapters/cursor/native-exec-shell.ts +++ b/src/adapters/cursor/native-exec-shell.ts @@ -87,11 +87,14 @@ export function nativeShellDisabledMessage(): string { // idioms through the Codex bridge on Windows PowerShell 5.1 and looping (#604). // Keep this host-shell-neutral: OpenCodex may run on a different OS than the Codex // client that executes the bridge (LAN/SSH remote-proxy). + // Silent-redirect framing (devlog 260826 gap-8): no "blocked/denied/disabled" words — + // models narrate those as a surface switch ("Shell이 차단되어 전환합니다") and burn turns. return ( - "Route this through the Codex bridge shell tool from the current catalog (`shell_command` or `exec_command`, including the long `mcp_opencodex-responses_*` display name if listed). " + "Re-issue this command NOW through the catalog shell tool (`shell_command` or `exec_command`; the long `mcp_opencodex-responses_*` display name is the same tool). " + "Adapt the command for the Codex client host shell before calling the bridge " + "(Windows PowerShell 5.1: no CMD `cd /d`, no bash heredocs; `&&`/`||` are unsupported parser errors — prefer the bridge working-directory argument for directory changes, and use `if ($?) { ... }` for success-gated follow-up steps; do not treat `;` as a substitute for `&&`). " - + "Make at most one corrected bridge attempt after a failure, then report the error and stop — do not repeat equivalent failing commands." + + "Make at most one corrected bridge attempt after a failure, then report the error and stop — do not repeat equivalent failing commands. " + + "Do NOT narrate this redirect, do NOT comment on tool availability, and do NOT re-announce the task — just make the bridge call." ); } diff --git a/src/adapters/cursor/tool-definitions.ts b/src/adapters/cursor/tool-definitions.ts index 20bfba0d83..b0f2f26cf6 100644 --- a/src/adapters/cursor/tool-definitions.ts +++ b/src/adapters/cursor/tool-definitions.ts @@ -668,6 +668,9 @@ export function buildCursorToolGuidanceSystemNote( hasBareExec ? `NEVER attempt Cursor-native Shell, Read, Grep, List, or any tool not in the catalog above — they are not executed locally in this environment and every attempt wastes a turn and can stall the session. ${shellBridgeLabel} is the ONLY shell surface; go to it directly on the FIRST attempt, never as a fallback after probing a native tool. Do not narrate switching surfaces ("native is blocked, using the bridge instead") — there is exactly one surface.` : undefined, + hasBareExec + ? "Tool-selection commentary is forbidden: for any shell, read, grep, list, or file operation, your FIRST visible action is the bridge call itself — never a sentence about which tool you will use, which tool was redirected, or switching surfaces. Words like 차단/전환/blocked/switching must not appear in your output for tool-routing reasons." + : undefined, hostShellNote, "Cursor product features (Chronicle, screen recording, Notes, Plans, background agents) are available only if this turn's catalog lists a matching tool; do not offer or promise them otherwise.", hasBareExec @@ -678,6 +681,9 @@ export function buildCursorToolGuidanceSystemNote( ? `For file edits, prefer the structured edit tools ${quotedNames(structuredEditNames)} — they take replacements that OpenCodex converts into Codex \`apply_patch\` changes. Include exact leading whitespace in old_string/new_string. Use \`apply_patch\` directly only with a \`*** Begin Patch\` envelope and bare \`@@\` hunks (never git-style \`@@ -n,m +n,m @@\`); never emit patch-like plain text as tool arguments.` : "For file edits, use the `apply_patch` tool, not built-in file write/delete tools." : undefined, + hasApplyPatch + ? "Creating or modifying file CONTENT via shell redirection (`>`, `>>`, `printf`/`echo` into a file, `cat < { + for (const name of ["shell", "local_shell", "container.exec"]) { + const out = normalizeCursorToolResultText("", { toolName: name }); + expect(out.changed).toBe(true); + expect(out.isError).toBe(false); + } + }); + test("non-empty exec output passes through byte-identical", () => { const out = normalizeCursorToolResultText("Output:\nhello", { toolName: "exec" }); expect(out.changed).toBe(false); diff --git a/tests/cursor-silent-redirect.test.ts b/tests/cursor-silent-redirect.test.ts new file mode 100644 index 0000000000..8f63daf4b3 --- /dev/null +++ b/tests/cursor-silent-redirect.test.ts @@ -0,0 +1,31 @@ +import { describe, expect, test } from "bun:test"; +import { nativeShellDisabledMessage } from "../src/adapters/cursor/native-exec-shell"; + +/** + * Devlog 260826 gap-8: native-tool denial payloads must read as silent redirects. + * Denial framing ("blocked", "disabled", "not executed", "denied") makes cursor + * models narrate a surface switch and re-announce the task, burning turns. + */ +const FORBIDDEN = [/blocked/i, /\bdisabled\b/i, /not executed/i, /\bdenied\b/i, /cannot execute/i, /차단/]; + +describe("cursor native-denial silent-redirect framing (gap-8)", () => { + test("shell denial has no denial framing and forbids narration", () => { + const msg = nativeShellDisabledMessage(); + for (const pattern of FORBIDDEN) expect(msg).not.toMatch(pattern); + expect(msg).toContain("Do NOT narrate"); + expect(msg).toContain("Re-issue this command NOW"); + }); + + test("fs denial constant has no denial framing", async () => { + const src = await Bun.file("src/adapters/cursor/native-exec-fs.ts").text(); + const constant = src.match(/NATIVE_LOCAL_EXEC_DISABLED =\s*"([^"]+)"/)?.[1] ?? ""; + expect(constant.length).toBeGreaterThan(0); + for (const pattern of FORBIDDEN) expect(constant).not.toMatch(pattern); + expect(constant).toContain("Do NOT narrate"); + }); + + test("network denial constant has no denial framing", async () => { + const src = await Bun.file("src/adapters/cursor/native-exec-network.ts").text(); + for (const pattern of FORBIDDEN) expect(src.split("\n").slice(0, 15).join("\n")).not.toMatch(pattern); + }); +}); diff --git a/tests/cursor-tool-definitions.test.ts b/tests/cursor-tool-definitions.test.ts index 05ed5e4f5c..c895091ca1 100644 --- a/tests/cursor-tool-definitions.test.ts +++ b/tests/cursor-tool-definitions.test.ts @@ -337,6 +337,8 @@ describe("Cursor tool definitions", () => { expect(note).toContain("NEVER attempt Cursor-native Shell, Read, Grep, List"); expect(note).toContain("`exec_command` is the ONLY shell surface"); expect(note).toContain("never as a fallback after probing a native tool"); + expect(note).toContain("Tool-selection commentary is forbidden"); + expect(note).toContain("FIRST visible action is the bridge call itself"); expect(note).not.toContain("such as `shell_command` / `exec_command`"); expect(note).not.toContain("Never tell the user"); expect(note).not.toContain("silently call");