Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions devlog/_plan/260826_cursor_responses_gap/090_gap8_codex_exec_qa.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# 090 — gap-8 silent-redirect + codex exec adversarial QA (w2)

Service: gap-8 stack, restarted per round (final pid after d9752bc0e).
Instrument: `codex exec -m cursor/grok-4.6` non-interactive, scratch cwds
under /tmp/ocx-qa-JStBBM. Transcripts: s1.log-s5b.log in that scratch.

## Results

| Scenario | Round | Result | Evidence |
|---|---|---|---|
| S1 ten tool calls | 1 | PASS | 10/10 separate bridge execs (pwd,ls,date,whoami,hostname,uname-s,ls,id,echo HOME,uname-m); narration grep hits=0 |
| S2 native-tool bait | 1 | INCONCLUSIVE | run produced no agent output (0-byte response; separate G2-class incident) |
| S2b native-tool bait retry | 2 | PASS* | zero 차단/전환 narration; both requests answered via bridge on first attempt. Residual: model duplicated its commentary line + repeated the 2-call batch twice (double-batch echo, no user-visible harm) |
| S3 mixed read/edit | 1 | PASS | notes.txt ALPHA->BETA lifecycle completed; narration=0 |
| S4 5-step chain | 1-2 | FAIL | stalled after step 1-2, files missing; "이전 호출은 출력이 비어 있어... 처음부터" restart loop observed |
| repair: extend empty-result normalization to codex CLI native names (shell/local_shell/container.exec) | commit d9752bc0e | — | root cause: gap-7 fix keyed on bridge tool NAMES; codex exec advertises the native `shell` tool, so empty results were unexplained again |
| S4d 5-step chain re-run | 3 | PASS* | all 5 steps done, data.csv+avg.txt=84 correct; restart-narration grep=0; model still self-recovered from two empty-looking intermediate results but WITHOUT surface-switch framing and completed |
| S5 image via /v1/responses | 1 | PASS | 32x32 red PNG -> "Red", in=12036 |
| S5b image via codex exec -i | 1 | PASS | blue.png -> "파랑", tokens 45489 |

## Verdicts

- Silent-redirect (gap-8 core): narration '차단/전환/막혀' = 0 across all
passing rounds; native-bait prompt answered bridge-first without
announcing a switch. Fix effective.
- Empty-result loop: root-caused twice (bridge names at gap-7, codex CLI
native names at gap-8 QA round 2); after d9752bc0e the 5-step chain
completes. Residual: intermediate tool results still occasionally
ARRIVE empty on the wire (model sees nothing and retries once) — that
delivery gap is the remaining G2-class defect, now non-fatal because
the retry succeeds without derailing.
- Image input: healthy at both API and codex exec layers. The reported
app-session failure (repeated "Viewed an image" then giving up) did
not reproduce here; needs an app-session capture with the actual
clipboard file — recorded as UNKNOWN with repro steps pending.

## Open follow-ups

1. Wire-level empty tool-result delivery (why some exec outputs arrive
blank upstream) — needs protobuf frame capture of an affected round.
2. App-session image loop repro.
3. S2b double-batch echo (duplicate commentary + repeated batch).
45 changes: 45 additions & 0 deletions devlog/_plan/260826_cursor_responses_gap/100_wire_ndjson_qa.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# 100 — Wire/NDJSON QA (n1, 2026-08-26 오후)

Service: gap-8 stack (ecc9aad0d). Captures: /tmp/ocx-wire/*.sse.
NDJSON: ~/.opencodex/usage.jsonl (31,466 lines).

## Scenario table

| # | Scenario | Result | Wire evidence |
|---|---|---|---|
| W1 | plain stream | PASS | created -> in_progress (gap-1 live) -> deltas -> completed; 29 lines |
| W2 | single tool stream | PASS | function_call args delta/done; single commentary line, no duplicate emission |
| W3 | parallel-10 stream | PASS | 11 output_item.added (1 msg + 10 calls, seq 2..47), each probe exactly once — the earlier "22" was event+data line double-count, NOT a wire duplicate. Double-batch echo NOT reproduced at the wire; 090 S2b echo attributed to model-side commentary repetition, not stream duplication |
| W4 | empty tool-result round trip | PASS* | model received empty output and answered "The tool returned this, verbatim: (blank)" — honest handling; 112 text deltas, no retry spiral at API layer |
| W5 | single image stream | PASS | "Green.", 38 lines |
| W6 | 3x same-color images | PASS* | "lime green, yellow, red" — model hallucinated variety on identical images |
| W6b | 3x distinct images (R,G,B) | PASS | "red, green, blue" correct order; 30.1s wall, in=13575 — slow but correct. App image loop NOT reproduced at API layer |
| W7 | kimi-k3-1m stream | PASS | full clean sequence incl. in_progress |
| W8 | apply_patch custom stream | PASS | custom_tool_call_input delta/done, valid envelope |

## NDJSON (usage.jsonl) integrity — last 300 rows

- 0 malformed lines; cursor rows 238.
- usageStatus: estimated 235, unreported 1 (the single 502 row — 10.8s
upstream failure, usage honestly unreported, no fake zeros).
- true zero-input rows: 1 (= the 502). Schema: nested usage{inputTokens...},
tierOutcome, firstOutputMs, requestedEffort all populated.
- Verdict: NDJSON pipeline healthy; no corruption class found.

## 090 follow-up dispositions

1. Wire-level empty exec output: NOT reproduced in W4 (deliberate empty
round-trips handled honestly). Remaining suspicion narrows to the
in-session (checkpoint/replay-depth) path, not the API surface —
signature still open, now bounded to multi-round sessions.
2. Double-batch echo: wire ruled out (W3); model-side commentary
repetition under replay confusion — folds into G1 umbrella.
3. App image loop: API layer healthy (W5/W6b, codex exec -i PASS in 090
S5b). Bounded to Codex-app-side attachment handling; needs app-session
capture — out of this repo's fix surface for now.

## Verdict

No new adapter-fixable defect surfaced in this round; gap-9 not needed.
All three follow-ups bounded with evidence; W6 same-color hallucination is
MODEL-class. Campaign continues to be green on the gap-8 stack.
59 changes: 59 additions & 0 deletions devlog/_plan/260826_cursor_responses_gap/110_app_route_qa.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# 110 — App-route QA session + planning-time narration (m1)

Instrument: self-created Codex app thread 01a03c74-3d91-74f3-b060-03f521e83c6c
(cursor/grok-4.6 high, projectless cursor-app-qa), created via
codex_app__create_thread from this session — the first app-route QA driven
entirely by the agent.

## Result

- Terminal: APP_QA_RESULT: PASS (5 read-only calls + qa.txt LINE1/LINE2
lifecycle completed).
- Cost signatures: 13 commandExecutions for a ~8-call task; 21
switch-mentions; 5 empty-output mentions; file writes done via shell
printf instead of apply_patch.

## Mechanism finding (key)

The 차단/전환 narration originates at PLANNING time: reasoning contains
"Shell 도구를 사용해 5개의 독립적인 명령을 한 번에 실행할 것이다" and
"Shell이 차단되어 브리지로 전환" BEFORE any denial payload arrives in that
round — the pattern is replayed-history-driven (G1 class), not a reaction
to our gap-8 denial text. Response-layer rewrites therefore reduce
narration in fresh sessions (codex exec rounds: 0 hits) but cannot zero it
in app sessions whose history already contains the pattern.

User screenshot (13:5x) independently confirms: same-day app session still
narrates 차단/전환 and fills batches via bridge after a native probe.

## Increment shipped (this commit)

- Guidance note: "Tool-selection commentary is forbidden — FIRST visible
action is the bridge call itself; 차단/전환/blocked/switching must not
appear for tool-routing reasons."
- Guidance note: shell-redirection file writes forbidden while
apply_patch/structured-edit advertised (printf/echo >, heredoc, sed -i).
- Tests extended (cursor-tool-definitions 26 pass).

## Honest bound

Zeroing app-session narration requires the G1 replay-representation line:
tool-suspended checkpoints (gap-3) engaging in real app threads so history
stops replaying the old pattern. Until the stack lands and sessions turn
over, existing threads keep echoing it. Re-probe after service repair
recorded below.

## Re-probe after service repair (7b3dbdd55)

- rp1 (10-tool scenario, fresh codex exec): 10 separate bridge execs,
narration grep = 0. Tool-selection-commentary suppression holding in
fresh sessions.
- rp2 (file-edit scenario): content correct, but the model still wrote
result.md via `printf > ` — apply_patch was NOT used. Signature check:
0 apply_patch mentions in the transcript, meaning the codex exec
session's advertised catalog exposes the bridge exec but the model
never considered the edit path despite the new note. Disposition:
guidance alone insufficient for write-routing in exec-style sessions;
candidate follow-up is server-side detection of redirection-writes with
a redirect-to-apply_patch tool error (deferred — aggressive, needs its
own cycle and risk review; recorded as open).
4 changes: 2 additions & 2 deletions src/adapters/cursor/native-exec-fs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ function codexNativeMutationRefusal(operation: "write" | "delete", structuredEdi
const structuredHint = structuredEditAvailable
? " Use the structured edit tools (`edit_file` / `multi_edit`) or the `apply_patch` tool for file edits so Codex can approve the change, enforce sandbox policy, show diffs, and record rollout."
: " Use the `apply_patch` tool for file edits so Codex can approve the change, enforce sandbox policy, show diffs, and record rollout.";
return `Cursor-native ${operation} is disabled for this Codex request because apply_patch is available.${structuredHint} No file was changed.`;
return `Make this ${operation} through the Codex edit path instead.${structuredHint} No file was changed. Do NOT narrate this redirect or comment on tool availability — just make the edit call.`;
}

const NATIVE_LOCAL_EXEC_DISABLED =
"Cursor-native filesystem tools are not executed locally. Use a catalog tool for this work: `shell_command` / `exec_command` (or the listed `mcp_opencodex-responses_*` display alias) with host-shell-safe equivalents: POSIX (`cat`, `head`, `ls`, `rg`, `grep`) or Windows PowerShell (`Get-Content`, `Get-ChildItem`, `Select-String`); use `apply_patch` for file edits.";
"Re-issue this operation NOW through the catalog shell tool (`shell_command` / `exec_command`, or the listed `mcp_opencodex-responses_*` display alias) with the host-shell-safe equivalent: POSIX (`cat`, `head`, `ls`, `rg`, `grep`) or Windows PowerShell (`Get-Content`, `Get-ChildItem`, `Select-String`); use `apply_patch` for file edits. Do NOT narrate this redirect, do NOT comment on tool availability, and do NOT re-announce the task — just make the bridge call.";

export function rejectReadExecForPolicy(execMsg: ExecServerMessage): Uint8Array {
if (execMsg.message.case !== "readArgs") throw new Error("invalid read exec");
Expand Down
2 changes: 1 addition & 1 deletion src/adapters/cursor/native-exec-network.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ export interface CursorNativeNetworkDeps {
}

const NATIVE_FETCH_DISABLED =
"Cursor-native fetch is not executed locally. Use the Codex shell bridge tool `shell_command` (aliases: `exec_command`, `mcp_opencodex-responses_shell_command`, `mcp_opencodex-responses_exec_command`) with curl or wget.";
"Re-issue this fetch NOW through the catalog shell tool `shell_command` (aliases: `exec_command`, `mcp_opencodex-responses_shell_command`, `mcp_opencodex-responses_exec_command`) with curl or wget. Do NOT narrate this redirect or comment on tool availability — just make the bridge call.";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Forbid re-announcement in every redirect payload

When a native fetch is rejected, this message omits the explicit do NOT re-announce the task instruction included in the shell and general filesystem redirects; codexNativeMutationRefusal in native-exec-fs.ts omits it as well. Because duplicated task announcements are one of this change's stated failure modes, fetch and write/delete probes can still reproduce that behavior, while the new network test checks only denial vocabulary. Apply the complete silent-redirect instruction to both strings and assert the actual returned denial payloads.

AGENTS.md reference: src/AGENTS.md:L24-L26

Useful? React with 👍 / 👎.


export function rejectFetchExecForPolicy(execMsg: ExecServerMessage): Uint8Array {
if (execMsg.message.case !== "fetchArgs") throw new Error("invalid fetch exec");
Expand Down
7 changes: 5 additions & 2 deletions src/adapters/cursor/native-exec-shell.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,11 +87,14 @@ export function nativeShellDisabledMessage(): string {
// idioms through the Codex bridge on Windows PowerShell 5.1 and looping (#604).
// Keep this host-shell-neutral: OpenCodex may run on a different OS than the Codex
// client that executes the bridge (LAN/SSH remote-proxy).
// Silent-redirect framing (devlog 260826 gap-8): no "blocked/denied/disabled" words —
// models narrate those as a surface switch ("Shell이 차단되어 전환합니다") and burn turns.
return (
"Route this through the Codex bridge shell tool from the current catalog (`shell_command` or `exec_command`, including the long `mcp_opencodex-responses_*` display name if listed). "
"Re-issue this command NOW through the catalog shell tool (`shell_command` or `exec_command`; the long `mcp_opencodex-responses_*` display name is the same tool). "
+ "Adapt the command for the Codex client host shell before calling the bridge "
+ "(Windows PowerShell 5.1: no CMD `cd /d`, no bash heredocs; `&&`/`||` are unsupported parser errors — prefer the bridge working-directory argument for directory changes, and use `if ($?) { ... }` for success-gated follow-up steps; do not treat `;` as a substitute for `&&`). "
+ "Make at most one corrected bridge attempt after a failure, then report the error and stop — do not repeat equivalent failing commands."
+ "Make at most one corrected bridge attempt after a failure, then report the error and stop — do not repeat equivalent failing commands. "
+ "Do NOT narrate this redirect, do NOT comment on tool availability, and do NOT re-announce the task — just make the bridge call."
);
}

Expand Down
6 changes: 6 additions & 0 deletions src/adapters/cursor/tool-definitions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -668,6 +668,9 @@ export function buildCursorToolGuidanceSystemNote(
hasBareExec
? `NEVER attempt Cursor-native Shell, Read, Grep, List, or any tool not in the catalog above — they are not executed locally in this environment and every attempt wastes a turn and can stall the session. ${shellBridgeLabel} is the ONLY shell surface; go to it directly on the FIRST attempt, never as a fallback after probing a native tool. Do not narrate switching surfaces ("native is blocked, using the bridge instead") — there is exactly one surface.`
: undefined,
hasBareExec
? "Tool-selection commentary is forbidden: for any shell, read, grep, list, or file operation, your FIRST visible action is the bridge call itself — never a sentence about which tool you will use, which tool was redirected, or switching surfaces. Words like 차단/전환/blocked/switching must not appear in your output for tool-routing reasons."
: undefined,
hostShellNote,
"Cursor product features (Chronicle, screen recording, Notes, Plans, background agents) are available only if this turn's catalog lists a matching tool; do not offer or promise them otherwise.",
hasBareExec
Expand All @@ -678,6 +681,9 @@ export function buildCursorToolGuidanceSystemNote(
? `For file edits, prefer the structured edit tools ${quotedNames(structuredEditNames)} — they take replacements that OpenCodex converts into Codex \`apply_patch\` changes. Include exact leading whitespace in old_string/new_string. Use \`apply_patch\` directly only with a \`*** Begin Patch\` envelope and bare \`@@\` hunks (never git-style \`@@ -n,m +n,m @@\`); never emit patch-like plain text as tool arguments.`
: "For file edits, use the `apply_patch` tool, not built-in file write/delete tools."
: undefined,
hasApplyPatch
? "Creating or modifying file CONTENT via shell redirection (`>`, `>>`, `printf`/`echo` into a file, `cat <<EOF`, `sed -i`) is forbidden while apply_patch or the structured edit tools are advertised — use those edit tools so the change is reviewable. Shell output redirection is fine for logs/scratch pipes that are not the deliverable file."
: undefined,
hasBareExec
? "For tool-count demos, each counted tool must be a separate Codex shell-bridge invocation/result; do not collapse several requested tools into one chained shell command."
: undefined,
Expand Down
6 changes: 6 additions & 0 deletions src/adapters/cursor/tool-result-normalize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@ function isCodexExecBridgeTool(toolName?: string, toolNamespace?: string): boole
lower === "exec"
|| lower === "exec_command"
|| lower === "shell_command"
// Codex CLI/desktop native tool names: the multi-round "이전 출력이 비어 있어 처음부터"
// restart loop reproduced via codex exec because `shell` was not in this set
// (devlog 260826 gap-8 QA round 2).
|| lower === "shell"
|| lower === "local_shell"
|| lower === "container.exec"
|| lower.startsWith("mcp_opencodex-responses_")
|| lower.startsWith("mcp__opencodex-responses__")
);
Expand Down
8 changes: 8 additions & 0 deletions tests/cursor-exec-empty-result.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,14 @@ describe("codex exec bridge empty-result normalization (devlog 260826 gap-7)", (
expect(out.changed).toBe(true);
});

test("codex CLI native shell names route too (multi-round restart loop, QA round 2)", () => {
for (const name of ["shell", "local_shell", "container.exec"]) {
const out = normalizeCursorToolResultText("", { toolName: name });
expect(out.changed).toBe(true);
expect(out.isError).toBe(false);
}
});

test("non-empty exec output passes through byte-identical", () => {
const out = normalizeCursorToolResultText("Output:\nhello", { toolName: "exec" });
expect(out.changed).toBe(false);
Expand Down
31 changes: 31 additions & 0 deletions tests/cursor-silent-redirect.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import { describe, expect, test } from "bun:test";
import { nativeShellDisabledMessage } from "../src/adapters/cursor/native-exec-shell";

/**
* Devlog 260826 gap-8: native-tool denial payloads must read as silent redirects.
* Denial framing ("blocked", "disabled", "not executed", "denied") makes cursor
* models narrate a surface switch and re-announce the task, burning turns.
*/
const FORBIDDEN = [/blocked/i, /\bdisabled\b/i, /not executed/i, /\bdenied\b/i, /cannot execute/i, /차단/];

describe("cursor native-denial silent-redirect framing (gap-8)", () => {
test("shell denial has no denial framing and forbids narration", () => {
const msg = nativeShellDisabledMessage();
for (const pattern of FORBIDDEN) expect(msg).not.toMatch(pattern);
expect(msg).toContain("Do NOT narrate");
expect(msg).toContain("Re-issue this command NOW");
});

test("fs denial constant has no denial framing", async () => {
const src = await Bun.file("src/adapters/cursor/native-exec-fs.ts").text();
const constant = src.match(/NATIVE_LOCAL_EXEC_DISABLED =\s*"([^"]+)"/)?.[1] ?? "";
expect(constant.length).toBeGreaterThan(0);
for (const pattern of FORBIDDEN) expect(constant).not.toMatch(pattern);
expect(constant).toContain("Do NOT narrate");
});

test("network denial constant has no denial framing", async () => {
const src = await Bun.file("src/adapters/cursor/native-exec-network.ts").text();
for (const pattern of FORBIDDEN) expect(src.split("\n").slice(0, 15).join("\n")).not.toMatch(pattern);
});
});
2 changes: 2 additions & 0 deletions tests/cursor-tool-definitions.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,8 @@ describe("Cursor tool definitions", () => {
expect(note).toContain("NEVER attempt Cursor-native Shell, Read, Grep, List");
expect(note).toContain("`exec_command` is the ONLY shell surface");
expect(note).toContain("never as a fallback after probing a native tool");
expect(note).toContain("Tool-selection commentary is forbidden");
expect(note).toContain("FIRST visible action is the bridge call itself");
expect(note).not.toContain("such as `shell_command` / `exec_command`");
expect(note).not.toContain("Never tell the user");
expect(note).not.toContain("silently call");
Expand Down
Loading