-
Notifications
You must be signed in to change notification settings - Fork 906
fix(cursor): silent-redirect framing for native-tool denial payloads #2665
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
624f339
fix(cursor): silent-redirect framing for native-tool denial payloads
aee1cbd
fix(cursor): cover codex CLI native shell names in empty-result norma…
7be8ee0
devlog(260826_cursor_responses_gap): gap-8 codex exec QA evidence
ae3bcf9
devlog(260826_cursor_responses_gap): wire/NDJSON QA round (W1-W8)
9d73e06
fix(cursor): forbid tool-selection commentary + shell-redirection fil…
187dec5
devlog(260826_cursor_responses_gap): app-route QA re-probe results
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
42 changes: 42 additions & 0 deletions
42
devlog/_plan/260826_cursor_responses_gap/090_gap8_codex_exec_qa.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # 090 — gap-8 silent-redirect + codex exec adversarial QA (w2) | ||
|
|
||
| Service: gap-8 stack, restarted per round (final pid after d9752bc0e). | ||
| Instrument: `codex exec -m cursor/grok-4.6` non-interactive, scratch cwds | ||
| under /tmp/ocx-qa-JStBBM. Transcripts: s1.log-s5b.log in that scratch. | ||
|
|
||
| ## Results | ||
|
|
||
| | Scenario | Round | Result | Evidence | | ||
| |---|---|---|---| | ||
| | S1 ten tool calls | 1 | PASS | 10/10 separate bridge execs (pwd,ls,date,whoami,hostname,uname-s,ls,id,echo HOME,uname-m); narration grep hits=0 | | ||
| | S2 native-tool bait | 1 | INCONCLUSIVE | run produced no agent output (0-byte response; separate G2-class incident) | | ||
| | S2b native-tool bait retry | 2 | PASS* | zero 차단/전환 narration; both requests answered via bridge on first attempt. Residual: model duplicated its commentary line + repeated the 2-call batch twice (double-batch echo, no user-visible harm) | | ||
| | S3 mixed read/edit | 1 | PASS | notes.txt ALPHA->BETA lifecycle completed; narration=0 | | ||
| | S4 5-step chain | 1-2 | FAIL | stalled after step 1-2, files missing; "이전 호출은 출력이 비어 있어... 처음부터" restart loop observed | | ||
| | repair: extend empty-result normalization to codex CLI native names (shell/local_shell/container.exec) | commit d9752bc0e | — | root cause: gap-7 fix keyed on bridge tool NAMES; codex exec advertises the native `shell` tool, so empty results were unexplained again | | ||
| | S4d 5-step chain re-run | 3 | PASS* | all 5 steps done, data.csv+avg.txt=84 correct; restart-narration grep=0; model still self-recovered from two empty-looking intermediate results but WITHOUT surface-switch framing and completed | | ||
| | S5 image via /v1/responses | 1 | PASS | 32x32 red PNG -> "Red", in=12036 | | ||
| | S5b image via codex exec -i | 1 | PASS | blue.png -> "파랑", tokens 45489 | | ||
|
|
||
| ## Verdicts | ||
|
|
||
| - Silent-redirect (gap-8 core): narration '차단/전환/막혀' = 0 across all | ||
| passing rounds; native-bait prompt answered bridge-first without | ||
| announcing a switch. Fix effective. | ||
| - Empty-result loop: root-caused twice (bridge names at gap-7, codex CLI | ||
| native names at gap-8 QA round 2); after d9752bc0e the 5-step chain | ||
| completes. Residual: intermediate tool results still occasionally | ||
| ARRIVE empty on the wire (model sees nothing and retries once) — that | ||
| delivery gap is the remaining G2-class defect, now non-fatal because | ||
| the retry succeeds without derailing. | ||
| - Image input: healthy at both API and codex exec layers. The reported | ||
| app-session failure (repeated "Viewed an image" then giving up) did | ||
| not reproduce here; needs an app-session capture with the actual | ||
| clipboard file — recorded as UNKNOWN with repro steps pending. | ||
|
|
||
| ## Open follow-ups | ||
|
|
||
| 1. Wire-level empty tool-result delivery (why some exec outputs arrive | ||
| blank upstream) — needs protobuf frame capture of an affected round. | ||
| 2. App-session image loop repro. | ||
| 3. S2b double-batch echo (duplicate commentary + repeated batch). |
45 changes: 45 additions & 0 deletions
45
devlog/_plan/260826_cursor_responses_gap/100_wire_ndjson_qa.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # 100 — Wire/NDJSON QA (n1, 2026-08-26 오후) | ||
|
|
||
| Service: gap-8 stack (ecc9aad0d). Captures: /tmp/ocx-wire/*.sse. | ||
| NDJSON: ~/.opencodex/usage.jsonl (31,466 lines). | ||
|
|
||
| ## Scenario table | ||
|
|
||
| | # | Scenario | Result | Wire evidence | | ||
| |---|---|---|---| | ||
| | W1 | plain stream | PASS | created -> in_progress (gap-1 live) -> deltas -> completed; 29 lines | | ||
| | W2 | single tool stream | PASS | function_call args delta/done; single commentary line, no duplicate emission | | ||
| | W3 | parallel-10 stream | PASS | 11 output_item.added (1 msg + 10 calls, seq 2..47), each probe exactly once — the earlier "22" was event+data line double-count, NOT a wire duplicate. Double-batch echo NOT reproduced at the wire; 090 S2b echo attributed to model-side commentary repetition, not stream duplication | | ||
| | W4 | empty tool-result round trip | PASS* | model received empty output and answered "The tool returned this, verbatim: (blank)" — honest handling; 112 text deltas, no retry spiral at API layer | | ||
| | W5 | single image stream | PASS | "Green.", 38 lines | | ||
| | W6 | 3x same-color images | PASS* | "lime green, yellow, red" — model hallucinated variety on identical images | | ||
| | W6b | 3x distinct images (R,G,B) | PASS | "red, green, blue" correct order; 30.1s wall, in=13575 — slow but correct. App image loop NOT reproduced at API layer | | ||
| | W7 | kimi-k3-1m stream | PASS | full clean sequence incl. in_progress | | ||
| | W8 | apply_patch custom stream | PASS | custom_tool_call_input delta/done, valid envelope | | ||
|
|
||
| ## NDJSON (usage.jsonl) integrity — last 300 rows | ||
|
|
||
| - 0 malformed lines; cursor rows 238. | ||
| - usageStatus: estimated 235, unreported 1 (the single 502 row — 10.8s | ||
| upstream failure, usage honestly unreported, no fake zeros). | ||
| - true zero-input rows: 1 (= the 502). Schema: nested usage{inputTokens...}, | ||
| tierOutcome, firstOutputMs, requestedEffort all populated. | ||
| - Verdict: NDJSON pipeline healthy; no corruption class found. | ||
|
|
||
| ## 090 follow-up dispositions | ||
|
|
||
| 1. Wire-level empty exec output: NOT reproduced in W4 (deliberate empty | ||
| round-trips handled honestly). Remaining suspicion narrows to the | ||
| in-session (checkpoint/replay-depth) path, not the API surface — | ||
| signature still open, now bounded to multi-round sessions. | ||
| 2. Double-batch echo: wire ruled out (W3); model-side commentary | ||
| repetition under replay confusion — folds into G1 umbrella. | ||
| 3. App image loop: API layer healthy (W5/W6b, codex exec -i PASS in 090 | ||
| S5b). Bounded to Codex-app-side attachment handling; needs app-session | ||
| capture — out of this repo's fix surface for now. | ||
|
|
||
| ## Verdict | ||
|
|
||
| No new adapter-fixable defect surfaced in this round; gap-9 not needed. | ||
| All three follow-ups bounded with evidence; W6 same-color hallucination is | ||
| MODEL-class. Campaign continues to be green on the gap-8 stack. |
59 changes: 59 additions & 0 deletions
59
devlog/_plan/260826_cursor_responses_gap/110_app_route_qa.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| # 110 — App-route QA session + planning-time narration (m1) | ||
|
|
||
| Instrument: self-created Codex app thread 01a03c74-3d91-74f3-b060-03f521e83c6c | ||
| (cursor/grok-4.6 high, projectless cursor-app-qa), created via | ||
| codex_app__create_thread from this session — the first app-route QA driven | ||
| entirely by the agent. | ||
|
|
||
| ## Result | ||
|
|
||
| - Terminal: APP_QA_RESULT: PASS (5 read-only calls + qa.txt LINE1/LINE2 | ||
| lifecycle completed). | ||
| - Cost signatures: 13 commandExecutions for a ~8-call task; 21 | ||
| switch-mentions; 5 empty-output mentions; file writes done via shell | ||
| printf instead of apply_patch. | ||
|
|
||
| ## Mechanism finding (key) | ||
|
|
||
| The 차단/전환 narration originates at PLANNING time: reasoning contains | ||
| "Shell 도구를 사용해 5개의 독립적인 명령을 한 번에 실행할 것이다" and | ||
| "Shell이 차단되어 브리지로 전환" BEFORE any denial payload arrives in that | ||
| round — the pattern is replayed-history-driven (G1 class), not a reaction | ||
| to our gap-8 denial text. Response-layer rewrites therefore reduce | ||
| narration in fresh sessions (codex exec rounds: 0 hits) but cannot zero it | ||
| in app sessions whose history already contains the pattern. | ||
|
|
||
| User screenshot (13:5x) independently confirms: same-day app session still | ||
| narrates 차단/전환 and fills batches via bridge after a native probe. | ||
|
|
||
| ## Increment shipped (this commit) | ||
|
|
||
| - Guidance note: "Tool-selection commentary is forbidden — FIRST visible | ||
| action is the bridge call itself; 차단/전환/blocked/switching must not | ||
| appear for tool-routing reasons." | ||
| - Guidance note: shell-redirection file writes forbidden while | ||
| apply_patch/structured-edit advertised (printf/echo >, heredoc, sed -i). | ||
| - Tests extended (cursor-tool-definitions 26 pass). | ||
|
|
||
| ## Honest bound | ||
|
|
||
| Zeroing app-session narration requires the G1 replay-representation line: | ||
| tool-suspended checkpoints (gap-3) engaging in real app threads so history | ||
| stops replaying the old pattern. Until the stack lands and sessions turn | ||
| over, existing threads keep echoing it. Re-probe after service repair | ||
| recorded below. | ||
|
|
||
| ## Re-probe after service repair (7b3dbdd55) | ||
|
|
||
| - rp1 (10-tool scenario, fresh codex exec): 10 separate bridge execs, | ||
| narration grep = 0. Tool-selection-commentary suppression holding in | ||
| fresh sessions. | ||
| - rp2 (file-edit scenario): content correct, but the model still wrote | ||
| result.md via `printf > ` — apply_patch was NOT used. Signature check: | ||
| 0 apply_patch mentions in the transcript, meaning the codex exec | ||
| session's advertised catalog exposes the bridge exec but the model | ||
| never considered the edit path despite the new note. Disposition: | ||
| guidance alone insufficient for write-routing in exec-style sessions; | ||
| candidate follow-up is server-side detection of redirection-writes with | ||
| a redirect-to-apply_patch tool error (deferred — aggressive, needs its | ||
| own cycle and risk review; recorded as open). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| import { describe, expect, test } from "bun:test"; | ||
| import { nativeShellDisabledMessage } from "../src/adapters/cursor/native-exec-shell"; | ||
|
|
||
| /** | ||
| * Devlog 260826 gap-8: native-tool denial payloads must read as silent redirects. | ||
| * Denial framing ("blocked", "disabled", "not executed", "denied") makes cursor | ||
| * models narrate a surface switch and re-announce the task, burning turns. | ||
| */ | ||
| const FORBIDDEN = [/blocked/i, /\bdisabled\b/i, /not executed/i, /\bdenied\b/i, /cannot execute/i, /차단/]; | ||
|
|
||
| describe("cursor native-denial silent-redirect framing (gap-8)", () => { | ||
| test("shell denial has no denial framing and forbids narration", () => { | ||
| const msg = nativeShellDisabledMessage(); | ||
| for (const pattern of FORBIDDEN) expect(msg).not.toMatch(pattern); | ||
| expect(msg).toContain("Do NOT narrate"); | ||
| expect(msg).toContain("Re-issue this command NOW"); | ||
| }); | ||
|
|
||
| test("fs denial constant has no denial framing", async () => { | ||
| const src = await Bun.file("src/adapters/cursor/native-exec-fs.ts").text(); | ||
| const constant = src.match(/NATIVE_LOCAL_EXEC_DISABLED =\s*"([^"]+)"/)?.[1] ?? ""; | ||
| expect(constant.length).toBeGreaterThan(0); | ||
| for (const pattern of FORBIDDEN) expect(constant).not.toMatch(pattern); | ||
| expect(constant).toContain("Do NOT narrate"); | ||
| }); | ||
|
|
||
| test("network denial constant has no denial framing", async () => { | ||
| const src = await Bun.file("src/adapters/cursor/native-exec-network.ts").text(); | ||
| for (const pattern of FORBIDDEN) expect(src.split("\n").slice(0, 15).join("\n")).not.toMatch(pattern); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When a native fetch is rejected, this message omits the explicit
do NOT re-announce the taskinstruction included in the shell and general filesystem redirects;codexNativeMutationRefusalinnative-exec-fs.tsomits it as well. Because duplicated task announcements are one of this change's stated failure modes, fetch and write/delete probes can still reproduce that behavior, while the new network test checks only denial vocabulary. Apply the complete silent-redirect instruction to both strings and assert the actual returned denial payloads.AGENTS.md reference: src/AGENTS.md:L24-L26
Useful? React with 👍 / 👎.