feat(desktop): codex app-server driver, and the input shapes codex rejects (L4c) - #571
Merged
Conversation
…jects (L4c)
L4a chose the rung and L4b opened the byte path; neither was imported by
main.ts. This is the driver that makes the lane user-visible: pick `codex`
in the Companion's local picker and the dock drives a real app-server
thread — streaming as it writes, folding tool calls the way a claude
session does, taking approvals inline, and reattaching after a restart.
Notifications go through the L2 interpreter with the hub's OWN codex frame
profile, which already ships to the desktop in agent_families.generated.json,
so a local codex transcript and a hub-driven one are the same rows.
Measured against a live codex-cli 0.147.0, not read off documentation — and
two of the measurements contradict what we ship:
* `{type:"input_image", image_url:…}` and `{type:"input_file", file_data:…}`
are answered `-32600 unknown variant`. They are OpenAI responses-API
content types, not app-server ones. The hub's AppServerDriver has been
sending both since W4.3, so E4's image passthrough has never reached a
codex M2 agent, and a PDF was worse than a no-op: an unknown variant
fails the whole turn/start, losing the director's message with it.
Images now use `{type:"image", url:"data:…"}` (confirmed end to end —
the model described the picture); PDFs are stripped with a system row,
the strip-and-warn shape driver_exec_resume.go already uses.
`codex.prompt_pdf.M2` in the registry was `true` and is now `false`:
0.147.0's UserInput union has no file variant at all.
* The plan's "resume argv from the N1 table" names a rung this driver
does not use, and N1's own table says so — the codex family's mechanism
is `appserver_thread_resume`. `service.rebind` now routes by mechanism:
argv families get a splice, the rest hand their handle to the driver.
Why a green suite missed the first one: the Go test's fake app-server
accepts any params, so it pinned the shipped shape rather than the protocol.
Hence an opt-in live e2e (TERMIPOD_CODEX_DRIVER_E2E=1) that runs real turns
against a real app-server — it caught a wrong assertion of my own on the
first run.
Also here:
* Posture is measured, not named. `read_local` lowers to
`{sandbox:"read-only", approvalPolicy:"never"}`, probed by asking codex
to write a file: it tried, failed, said so, and nothing landed on disk.
`converse` cannot be kept exactly (codex has no tool-disable switch), so
the lifecycle row carries a posture_note instead of implying a boundary
that is not there.
* Approvals become `approval_request` events in R1's own shapes, answered
by the cards that already ship — which is what R1's comment predicted.
Four kinds are refused AT ONCE rather than parked, because a card nobody
can answer holds the engine open forever; the one worth naming is a
question codex marks `isSecret`, since service.input() records every
input into the durable on-disk transcript before sending it.
* Config seeding rides `thread/start.config`, so configuring a session
never leaves a .codex/config.toml in the director's project.
* The engine-neutral half of claudewire.ts moves to driver.ts: with two
engines, a type named after claude that codex must import is a name that
lies to the next reader.
Verify: 716 electron tests (0 fail, 6 skipped — the two live e2e skip
without their env var), go build/vet/test ./... clean, CI's full lint set
clean, and both live e2e tests pass against a real codex app-server.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…unrace the image e2e
The rebind guard in the approval request id never guarded: `#epoch` was
bumped once per driver instance (there is no reconnect path inside one), so
every id was `codex-1-<n>` — and a rebind is a NEW instance counting from
the same start. The server's JSON-RPC id counter restarts with every
connection, and `#onClose` leaves parked asks unanswered, so the collision
lands exactly where it matters: a card left dangling by a dead engine could
answer whatever request reached the same number after the rebind — approving
a command the director never saw. The prefix is now a per-instance random
token (transcripts outlive the process, so a counter shared across instances
would still collide across app restarts); a stale click lands as
`codex_answer_unmatched`, pinned by a two-instance test.
The image e2e asserted /green/ on the FIRST completed text, but a turn is
free to open with a preamble ("i'll inspect the image…") before the answer —
it did, on this box. The wait now looks for a completed text that names the
colour, timing out with every collected text in hand — the same lesson the
e2e's other two assertions already record.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Vision-parity L4c — the codex driver. L4a chose the rung, L4b opened the byte path, and neither was imported by
main.ts. This is what makes the lane user-visible: pickcodexin the Companion's local picker and the dock drives a realapp-serverthread.Continues #568 (L4a) and #569 (L4b). Plan:
docs/plans/desktop-companion-vision-parity.md, lane L.What ships
localagent/codexwire.ts— pure JSON-RPC shapes: thread params, turn input, server-request classification, per-method response shapes.localagent/codexdriver.ts— the session: handshake, queued input, turn/interrupt, delta throttling, parked approvals, resume.localagent/driver.ts— the engine-neutral vocabulary (posture, input kinds,DriverEvent,LocalDriver), moved out ofclaudewire.ts. With two engines, a type named after claude that codex has to import is a name that lies to the next reader.service.tspicks a driver per family and routes rebind by resume mechanism.Notifications go through the L2 interpreter with the hub's own codex frame profile, already shipped in
agent_families.generated.json— so a local codex transcript and a hub-driven one are the same rows. No second vocabulary.★★ Two of the shapes we ship are rejected by codex
Measured against a live codex-cli 0.147.0:
{type:"input_image", image_url:"data:…"}-32600 Invalid request: unknown variant 'input_image', expected one of 'text', 'image', 'localImage', 'audio', 'localAudio', 'skill', 'mention'{type:"input_file", file_data:"data:…"}{type:"image", url:"data:…"}Those are OpenAI responses-API content types, not app-server ones. This is not only the new driver's problem: the hub's
AppServerDriver.startTurnhas sent both since W4.3, so E4's image passthrough has never reached a codex M2 agent, and a PDF was worse than a no-op — an unknown variant fails the wholeturn/start, taking the director's message with it. Fixed here, with the strip-and-warn shapedriver_exec_resume.goalready uses for what gemini cannot carry.The registry asserted it too:
codex.prompt_pdf.M2wastrue(what the F3 composer gate reads) and is nowfalse— 0.147.0'sUserInputunion has no file variant at all.prompt_image.M2staystrue; only the shape was wrong.Why a green suite missed it: the Go test's fake app-server accepts any params, so it pinned the shipped shape rather than the protocol — a fixture that cannot disprove the rule it tests. Hence an opt-in live e2e.
The plan line that was wrong
"resume argv from the N1 table" names a rung this driver does not use — and N1's own table says so: the codex family row reads
mechanism: appserver_thread_resume, with a note that thecodex resume <id>CLI recipe is a different rung.service.rebindnow routes by mechanism: argv families get a splice, the rest hand their handle to a driver that knows what to do with it.Other measurements the driver is built on
thread/resumerestores memory and emits no replay — the only notifications after it areconfigWarning,remoteControl/status/changed,thread/status/changed,thread/tokenUsage/updated,thread/goal/cleared. Same shape as claude's--resume, so L3b's durable log restores the view here too. (Its response does carrythread.turns[], unlike claude's — recorded, not used.)sandbox: read-only+approvalPolicy: neveris a real write barrier. Asked to create a file in its cwd, codex tried, failed, said "the environment is read-only", and nothing appeared on disk. The response echoes{"type":"readOnly","networkAccess":false}.conversecannot be kept exactly (codex has no tool-disable switch), so the lifecycle row carries aposture_noterather than implying a boundary that is not there.Approvals, with no attention table
Server requests become
approval_requestevents in R1's own shapes, answered by the cards that already ship — which is exactly what R1's comment predicted. Four kinds are refused at once rather than parked, because a card nobody can answer holds the engine open forever: a form-mode elicitation, aurlelicitation,item/permissions/requestApproval(its response is a granted profile, not a verdict), and — the one worth naming — a question codex marksisSecret, sinceservice.input()records every input into the durable on-disk transcript before sending it. Each refusal uses that method's own response shape; an empty{}fails to deserialize on every one of them.Config seeding rides
thread/start.config, so configuring a session never leaves a.codex/config.tomlin the director's project.Verification
go build ./...,go vet ./...,go test ./...— all clean.lint-openapi.shandlint-desktop-tokens.sh.TERMIPOD_CODEX_DRIVER_E2E=1): a real turn round-trips with streamed partials,thread/resumefinds the thread and the engine remembers, and an image reaches the model.The e2e earned itself on its first run by failing two of my assertions: one asserted a streamed partial on a reply too short to produce one (a race), and the other asserted the wrong colour — the widely-copied 1×1 "red dot" PNG is
RGBA(255,0,0,127), and a half-transparent pixel gets described differently depending on what it is composited against. It now uses an opaque pure-green pixel, since red and blue are what a model guesses when it cannot see the image at all.Left in the lane, named rather than skipped
No posture maps to
approvalPolicy: on-request, so codex's own command/file-change gates never fire from a Companion session today. Adding one is a posture decision, not a driver change — the approval path is built and tested, and MCP elicitations already exercise it.🤖 Generated with Claude Code