Rollouts persist each command's stdout 3x per item_completed record (stdout/aggregated_output/formatted_output) — 60% of session bytes; thread_history stores a 4th copy

[shaswatshah]

What version of Codex CLI is running?

codex-cli 0.150.1 (also Codex Desktop app-server 151.0.7922.170 on the same ~/.codex)

What subscription do you have?

Pro

Which model were you using?

gpt-5.6-sol

What platform is your computer?

Darwin 25.5.0 arm64 arm

What terminal emulator and version are you using (if applicable)?

Orca (terminal multiplexer/agent launcher) driving codex TUI sessions; Codex Desktop app-server also running against the same CODEX_HOME.

Codex doctor report

See "Additional information" (full JSON attached at the bottom; nothing redacted was needed).

What issue are you seeing?

~/.codex reached 14 GB on a machine where Codex is the secondary agent (Claude Code is primary). After removing rollouts older than 30 days, the last 30 days alone are 6.7 GB across 5,271 rollout files. Sizing what is inside them shows the dominant cost is not model traffic but the same command stdout being persisted four times — three copies inside a single rollout record, plus one more in thread_history_1.sqlite.

1. event_msg/item_completed for CommandExecution stores the full stdout three times in one record.

Each completed command is appended to the rollout as an event_msg whose item has stdout, stderr, aggregated_output, and formatted_output. Measured on the largest rollout (114 MB, 1,359 CommandExecution items):

field	bytes	note
stdout	27.9 MB
aggregated_output	27.9 MB	byte-identical to stdout in 1,359 / 1,359 items
formatted_output	16.8 MB	byte-identical to aggregated_output in 1,186 / 1,359 items
stderr	~0

That one record type is 75 MB of the 114 MB file (68%). The response_item/function_call_output / custom_tool_call_output records — the (truncated) output the model actually saw — total only 22 MB in the same file. So the rollout carries ~3.4× more untruncated stdout than model-visible output.

Across the 200 largest rollouts in ~/.codex/sessions (2.0 GB total):

record type	bytes	share
event_msg/item_completed (CommandExecution)	1,199 MB	59.9%
response_item/custom_tool_call_output	311 MB	15.5%
response_item/function_call_output	128 MB	6.4%
response_item/reasoning	110 MB	5.5%
everything else	~250 MB	~12%

Inside those CommandExecution items: stdout 461 MB + aggregated_output 461 MB + formatted_output 236 MB.

2. thread_history_1.sqlite stores a fourth copy.

The projection DB is 1.68 GB; thread_items rows with item_type = 'commandExecution' account for 1,486 MB of it, and json_extract(item_json, '$.aggregatedOutput') alone sums to 1,416 MB (individual items cap at ~1 MB). Every thread in it still has its rollout on disk, so this is a straight duplicate of data already in sessions/.

3. There is no age-based retention for any of this, so it only grows. (Tracked separately in #6015 / #28187 / #20230 — noted here only because it turns the duplication into a permanent cost.) logs_2.sqlite separately sat at 2.63 GB with 1.76 GB of freelist pages; that is #35823 and VACUUM fixed it.

Why this is distinct from existing reports. #36557 / #39469 / #31198 / #29531 are about parent history or compacted.replacement_history being replayed into child/resumed rollouts; #24948 and #34061 attribute growth to raw function_call_output volume and event counts. In my data the response_item outputs are the small part. The cost is the three redundant fields on item_completed events plus the projection copy — it applies to every command in every session, subagent or not, and stacks multiplicatively on top of the issues above. Workload context: 77% of my rollouts (4,631 of 5,271; 5.1 GB) are subagent threads from a multi-agent plugin whose workers run many read-heavy shell commands (cat, grep, git show), so each byte of stdout costs ~4 bytes on disk, forever.

What steps can reproduce the bug?

1. Run any session that executes a shell command with non-trivial stdout, e.g. codex exec "run: cat some-200KB-file" (or any TUI session doing normal work).
2. Open the rollout under ~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl and find the event_msg record with payload.type == "item_completed" and payload.item.type == "CommandExecution".
3. Observe item.stdout == item.aggregated_output == item.formatted_output (the last differs only when the formatted view truncates).
4. Observe the same output again, truncated, in the preceding response_item (custom_tool_call_output / function_call_output).
5. Observe it a fourth time: sqlite3 ~/.codex/thread_history_1.sqlite "select length(json_extract(item_json,'$.aggregatedOutput')) from thread_items where item_type='commandExecution' order by 1 desc limit 5".

Measurement script used for the numbers above (Python 3, stdlib only):

import json, sys, collections
by = collections.Counter(); fld = collections.Counter(); tot = 0
for path in sys.argv[1:]:
    for line in open(path):
        tot += len(line)
        r = json.loads(line); p = r.get("payload", {})
        t = r.get("type", "?") + "/" + str(p.get("type", ""))
        if t == "event_msg/item_completed":
            t += "/" + str(p.get("item", {}).get("type"))
        by[t] += len(line)
        if t == "event_msg/item_completed/CommandExecution":
            it = p["item"]
            for k in ("stdout", "stderr", "aggregated_output", "formatted_output"):
                fld[k] += len(it.get(k) or "")
print(f"total {tot/2**20:.0f} MB")
for t, b in by.most_common(8): print(f"{b/2**20:7.0f} MB {100*b/tot:5.1f}%  {t}")
for k, v in fld.items(): print(f"{v/2**20:7.0f} MB  {k}")

Example thread id with the 114 MB rollout: 01a020c7-c50c-7850-b187-da14ba3adc88.

What is the expected behavior?

• A command's output is persisted once per rollout. item_completed should either omit the output fields (the response_item already carries what the model saw, and the raw output could be referenced by offset) or carry a single field; stdout / aggregated_output / formatted_output should not be three literal copies.
• thread_history_1.sqlite should reference rollout offsets (it already stores rollout_byte_offset per turn) rather than re-embedding aggregatedOutput.
• Some size cap or retention on persisted raw output, so a cat of a large file does not become a permanent multi-megabyte on-disk cost.

Additional information

Disk state when this was measured (macOS, single user, Codex used a few hours a day):

• ~/.codex/sessions: 6.7 GB for 30 days (5,271 rollouts; 4,631 subagent, 626 interactive). Before pruning >30-day files it was 8.5 GB.
• ~/.codex/thread_history_1.sqlite: 1.68 GB (2,943 threads, all from the current month; 1,486 MB is commandExecution items).
• ~/.codex/logs_2.sqlite: 2.63 GB → 717 MB after VACUUM (logs_2.sqlite never reclaims freed pages: auto_vacuum=INCREMENTAL is set but never run, so the file grows monotonically despite working 10-day retention #35823).
• Related: Codex session logs grow to 700MB-2GB from repeated compaction history and raw tool output #24948, Subagent spawns duplicate the full parent session history into each child rollout file (quadratic disk growth) #36557, Codex Desktop subagent session logs grow to 145GiB from repeated compacted replacement_history #31198, Multi-agent fan-out duplicates full parent context into every subagent rollout file - 22 files x 9.4 GB = 208 GiB; codex.exe reaches 14 GB working set #39469, Insane Codex Disk Usage from Subagents #34061 (growth from other duplication paths); logs_2.sqlite never reclaims freed pages: auto_vacuum=INCREMENTAL is set but never run, so the file grows monotonically despite working 10-day retention #35823 (logs freelist); Customizable retention period for sessions #6015, Clarify tracking for Codex local session deletion, retention, and cleanup UX #28187, Add a first-class workflow for reviewing and clearing old CLI sessions #20230 (retention/cleanup).

codex doctor --json

{
  "schemaVersion": 1,
  "generatedAt": "1787893861s since unix epoch",
  "overallStatus": "warning",
  "codexVersion": "0.150.1",
  "checks": {
    "app_server.status": {
      "id": "app_server.status",
      "category": "app-server",
      "status": "ok",
      "summary": "background server is not running",
      "details": {
        "control socket": "/Users/shaswat/.codex/app-server-control/app-server-control.sock",
        "daemon state dir": "/Users/shaswat/.codex/app-server-daemon",
        "mode": "ephemeral",
        "pid file": "/Users/shaswat/.codex/app-server-daemon/app-server.pid (missing)",
        "settings": "/Users/shaswat/.codex/app-server-daemon/settings.json (missing)",
        "status": "not running",
        "update-loop pid file": "/Users/shaswat/.codex/app-server-daemon/app-server-updater.pid (missing)"
      },
      "remediation": null,
      "durationMs": 0
    },
    "auth.credentials": {
      "id": "auth.credentials",
      "category": "auth",
      "status": "ok",
      "summary": "auth is configured",
      "details": {
        "auth env vars present": "OPENAI_API_KEY",
        "auth file": "/Users/shaswat/.codex/auth.json",
        "auth storage mode": "File",
        "stored API key": "false",
        "stored ChatGPT tokens": "true",
        "stored agent identity": "false",
        "stored auth mode": "chatgpt"
      },
      "remediation": null,
      "durationMs": 0
    },
    "config.load": {
      "id": "config.load",
      "category": "config",
      "status": "ok",
      "summary": "config loaded",
      "details": {
        "CODEX_HOME": "/Users/shaswat/.codex",
        "config.toml": "/Users/shaswat/.codex/config.toml",
        "config.toml parse": "ok",
        "cwd": "/Users/shaswat/.codex",
        "enabled feature flags": "shell_tool, view_image, unified_exec, unified_exec_zsh_fork, shell_snapshot, code_mode_host, terminal_resize_reflow, sqlite, hooks, enable_request_compression, unbounded_connection_retries, multi_agent, multi_agent_v2, tool_search_always_defer_mcp_tools, tool_suggest, plugins, in_app_browser, in_app_chat, in_app_dictation, in_app_local_automation, in_app_updates, browser_use, browser_use_full_cdp_access, browser_use_external, computer_use, remote_plugin, plugin_sharing, image_generation, resize_all_images, item_ids, skill_mcp_dependency_install, skill_search, mentions_v2, steer, guardian_approval, goals, collaboration_modes, tool_call_mcp_elicitation, auth_elicitation, personality, fast_mode, tui_app_server, remote_compaction_v2, compaction_image_budget, workspace_dependencies",
        "feature flag overrides": "multi_agent_v2=true, apps=false",
        "feature flags enabled": "45",
        "log dir": "/Users/shaswat/.codex/log",
        "mcp servers": "3",
        "model": "gpt-5.6-sol",
        "model provider": "openai",
        "sqlite home": "/Users/shaswat/.codex"
      },
      "remediation": null,
      "durationMs": 0
    },
    "desktop.app.version": {
      "id": "desktop.app.version",
      "category": "desktop",
      "status": "ok",
      "summary": "the desktop application is installed",
      "details": {
        "log directory": "$HOME/Library/Logs/com.openai.codex",
        "running": "true",
        "version": "26.818.61809"
      },
      "remediation": null,
      "durationMs": 0
    },
    "desktop.app_server.handshake": {
      "id": "desktop.app_server.handshake",
      "category": "desktop",
      "status": "ok",
      "summary": "no desktop app-server handshake was recorded",
      "details": {},
      "remediation": null,
      "durationMs": 0
    },
    "desktop.security.enforcement": {
      "id": "desktop.security.enforcement",
      "category": "desktop",
      "status": "ok",
      "summary": "the desktop application passed available macos security assessments",
      "details": {
        "gatekeeper": "accepted"
      },
      "remediation": null,
      "durationMs": 0
    },
    "git.environment": {
      "id": "git.environment",
      "category": "git",
      "status": "ok",
      "summary": "git version 2.50.1 (Apple Git-155)",
      "details": {
        "PATH git #1": "/usr/bin/git",
        "PATH git entries": "1",
        "git build options": "git version 2.50.1 (Apple Git-155); cpu: arm64; no commit associated with this build; sizeof-long: 8; sizeof-size_t: 8; shell-path: /bin/sh; feature: fsmonitor--daemon; libcurl: 8.7.1; zlib: 1.2.12; SHA-1: SHA1_DC; SHA-256: SHA256_BLK",
        "git exec path": "/Library/Developer/CommandLineTools/usr/libexec/git-core",
        "git version": "git version 2.50.1 (Apple Git-155)",
        "repo detected": "false",
        "selected git": "/usr/bin/git"
      },
      "remediation": null,
      "durationMs": 146
    },
    "installation": {
      "id": "installation",
      "category": "install",
      "status": "ok",
      "summary": "installation looks consistent",
      "details": {
        "PATH codex #1": "/opt/homebrew/bin/codex",
        "current executable": "/opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/bin/codex",
        "install context": "npm (package /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin, bin /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/bin, resources /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/codex-resources, path /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/codex-path)",
        "managed by bun": "false",
        "managed by npm": "true",
        "managed by pnpm": "false",
        "managed package root": "/opt/homebrew/lib/node_modules/@openai/codex",
        "npm update target": "/opt/homebrew/lib/node_modules/@openai/codex"
      },
      "remediation": null,
      "durationMs": 191
    },
    "mcp.config": {
      "id": "mcp.config",
      "category": "mcp",
      "status": "ok",
      "summary": "MCP configuration is locally consistent",
      "details": {
        "configured servers": "3",
        "disabled servers": "1",
        "stdio servers": "2",
        "streamable_http servers": "1"
      },
      "remediation": null,
      "durationMs": 250
    },
    "network.env": {
      "id": "network.env",
      "category": "network",
      "status": "ok",
      "summary": "network-related environment looks readable",
      "details": {
        "managed proxy": "not configured",
        "proxy env vars": "none",
        "respect system proxy": "disabled",
        "system proxy": "direct"
      },
      "remediation": null,
      "durationMs": 1
    },
    "network.provider_reachability": {
      "id": "network.provider_reachability",
      "category": "reachability",
      "status": "ok",
      "summary": "active provider endpoints are reachable over HTTP",
      "details": {
        "ChatGPT inference URL": "https://chatgpt.com/backend-api/<redacted> reachable (HTTP 405)",
        "desktop assets CDN": "https://chatgpt.com/backend-api/<redacted> reachable (HTTP 200)",
        "reachability mode": "ChatGPT auth"
      },
      "remediation": null,
      "durationMs": 274
    },
    "network.websocket_reachability": {
      "id": "network.websocket_reachability",
      "category": "websocket",
      "status": "ok",
      "summary": "Responses WebSocket handshake succeeded",
      "details": {
        "DNS": "2 IPv4, 2 IPv6, first IPv6",
        "auth mode": "chatgpt",
        "connect timeout": "15000 ms",
        "endpoint": "wss://chatgpt.com/backend-api/<redacted>",
        "handshake result": "HTTP 101 Switching Protocols",
        "model provider": "openai",
        "provider name": "OpenAI",
        "proxy env vars": "none",
        "reasoning header": "false",
        "server model present": "false",
        "supports websockets": "true",
        "wire API": "responses"
      },
      "remediation": null,
      "durationMs": 1130
    },
    "runtime.provenance": {
      "id": "runtime.provenance",
      "category": "runtime",
      "status": "ok",
      "summary": "running npm on macos-aarch64",
      "details": {
        "commit": "unknown",
        "current executable": "/opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/bin/codex",
        "install method": "npm (package /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin, bin /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/bin, resources /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/codex-resources, path /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/codex-path)",
        "platform": "macos-aarch64",
        "version": "0.150.1"
      },
      "remediation": null,
      "durationMs": 0
    },
    "runtime.search": {
      "id": "runtime.search",
      "category": "search",
      "status": "ok",
      "summary": "search is OK (bundled)",
      "details": {
        "search command": "/opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/codex-path/rg",
        "search command readiness": "file exists",
        "search provider": "bundled"
      },
      "remediation": null,
      "durationMs": 0
    },
    "sandbox.helpers": {
      "id": "sandbox.helpers",
      "category": "sandbox",
      "status": "ok",
      "summary": "sandbox configuration is readable",
      "details": {
        "approval policy": "OnRequest",
        "codex-linux-sandbox helper": "none",
        "execve wrapper helper": "/Users/shaswat/.codex/tmp/arg0/codex-arg06QuFb4/codex-execve-wrapper",
        "filesystem sandbox": "restricted",
        "network sandbox": "enabled"
      },
      "remediation": null,
      "durationMs": 0
    },
    "security.endpoint": {
      "id": "security.endpoint",
      "category": "security",
      "status": "ok",
      "summary": "no supported endpoint protection detected",
      "details": {
        "endpoint products": "none detected"
      },
      "remediation": null,
      "durationMs": 92
    },
    "state.paths": {
      "id": "state.paths",
      "category": "state",
      "status": "ok",
      "summary": "state paths and databases are inspectable",
      "details": {
        "CODEX_HOME": "/Users/shaswat/.codex (dir)",
        "active rollout files": "5271 files, 7021023063 total bytes, 1332009 average bytes",
        "archived rollout files": "0 files, 0 total bytes, 0 average bytes",
        "goals DB": "/Users/shaswat/.codex/goals_1.sqlite (file)",
        "goals DB integrity": "ok",
        "log DB": "/Users/shaswat/.codex/logs_2.sqlite (file)",
        "log DB integrity": "ok",
        "log dir": "/Users/shaswat/.codex/log (dir)",
        "memories DB": "/Users/shaswat/.codex/memories_1.sqlite (file)",
        "memories DB integrity": "ok",
        "queue DB": "/Users/shaswat/.codex/queue_1.sqlite (file)",
        "queue DB integrity": "ok",
        "sqlite home": "/Users/shaswat/.codex (dir)",
        "state DB": "/Users/shaswat/.codex/state_5.sqlite (file)",
        "state DB integrity": "ok",
        "thread history DB": "/Users/shaswat/.codex/thread_history_1.sqlite (file)",
        "thread history DB integrity": "ok"
      },
      "remediation": null,
      "durationMs": 7875
    },
    "state.rollout_db_parity": {
      "id": "state.rollout_db_parity",
      "category": "threads",
      "status": "warning",
      "summary": "rollout files and state DB thread inventory differ",
      "details": {
        "default model provider": "openai",
        "rollout DB active files": "5271",
        "rollout DB active rows": "8220",
        "rollout DB archive mismatches": "0",
        "rollout DB archived files": "0",
        "rollout DB archived rows": "0",
        "rollout DB duplicate DB paths": "0",
        "rollout DB duplicate rollout thread ids": "0",
        "rollout DB malformed file names": "0",
        "rollout DB missing active rows": "0",
        "rollout DB missing archived rows": "0",
        "rollout DB model providers": "openai=8220",
        "rollout DB rows": "8220",
        "rollout DB scan cap reached": "false",
        "rollout DB scan errors": "0",
        "rollout DB sources": "subagent:thread_spawn=6838, cli=1300, exec=58, subagent:review=17, vscode=7",
        "rollout DB stale row sample": [
          "/Users/shaswat/.codex/sessions/2025/09/23/rollout-2025-09-23T12-24-04-c4679dde-dc1d-42f2-9613-ab0f2524fe4c.jsonl",
          "/Users/shaswat/.codex/sessions/2025/09/23/rollout-2025-09-23T12-24-31-01997808-e0ca-7a80-8e7a-813cdd935e16.jsonl",
          "/Users/shaswat/.codex/sessions/2025/09/23/rollout-2025-09-23T12-25-33-01997809-d148-7883-82c9-283b5cc83c66.jsonl",
          "/Users/shaswat/.codex/sessions/2025/09/24/rollout-2025-09-24T11-38-02-01997d04-ad27-7d40-9c5e-a27779e946e1.jsonl",
          "/Users/shaswat/.codex/sessions/2025/09/24/rollout-2025-09-24T18-55-39-01997e95-5414-76d1-acaf-79d52ac01763.jsonl"
        ],
        "rollout DB stale rows": "2949"
      },
      "issues": [
        {
          "severity": "warning",
          "cause": "state DB rows point at missing or unusable rollout files",
          "measured": "2949 stale rows",
          "expected": "every state DB rollout path is a file on disk",
          "remedy": null,
          "fields": []
        }
      ],
      "remediation": null,
      "durationMs": 3420
    },
    "system.disk": {
      "id": "system.disk",
      "category": "disk",
      "status": "ok",
      "summary": "sufficient free disk space (618.7 GiB)",
      "details": {
        "CODEX_HOME available": "618.7 GiB",
        "failure threshold": "1.0 GiB",
        "warning threshold": "5.0 GiB",
        "worktree available": "618.7 GiB"
      },
      "remediation": null,
      "durationMs": 0
    },
    "system.environment": {
      "id": "system.environment",
      "category": "system",
      "status": "ok",
      "summary": "OS language en-US",
      "details": {
        "EDITOR": "set",
        "LANG": "en_US.UTF-8",
        "LESS": "set",
        "PAGER": "set",
        "VISUAL": "set",
        "os": "Mac OS 26.5.1 [64-bit]",
        "os language": "en-US",
        "os type": "Mac OS",
        "os version": "26.5.1"
      },
      "remediation": null,
      "durationMs": 6
    },
    "terminal.env": {
      "id": "terminal.env",
      "category": "terminal",
      "status": "ok",
      "summary": "terminal metadata was detected",
      "details": {
        "COLORTERM": "truecolor",
        "TERM_PROGRAM": "Orca",
        "color output": "disabled (stdout is not a terminal)",
        "effective locale": "en_US.UTF-8",
        "stderr is terminal": "false",
        "stdin is terminal": "false",
        "stdout is terminal": "false",
        "terminal": "unknown",
        "terminal size": "80x24",
        "terminal version": "1.4.190"
      },
      "remediation": null,
      "durationMs": 8
    },
    "terminal.title": {
      "id": "terminal.title",
      "category": "title",
      "status": "ok",
      "summary": "terminal title default",
      "details": {
        "terminal title activity": "true",
        "terminal title items": "activity, project-name",
        "terminal title project source": "cwd",
        "terminal title project value": ".codex",
        "terminal title source": "default"
      },
      "remediation": null,
      "durationMs": 0
    },
    "updates.status": {
      "id": "updates.status",
      "category": "updates",
      "status": "ok",
      "summary": "update configuration is locally consistent",
      "details": {
        "cached latest version": "0.150.1",
        "check for update on startup": "true",
        "last checked at": "2026-08-27T17:26:50.054342Z",
        "latest version": "0.150.1",
        "latest version status": "current version is not older",
        "npm update target": "/opt/homebrew/lib/node_modules/@openai/codex",
        "update action": "npm install -g @openai/codex",
        "version cache": "/Users/shaswat/.codex/version.json"
      },
      "remediation": null,
      "durationMs": 286
    }
  }
}

[shaswatshah]

Follow-up, since 4 copies is an awkward number and the goal is clearly 10. Fastest routes using shipped mechanisms only:

1. Resume the thread once (Codex session rollout files duplicate full historical context on resume, causing multi-GB .jsonl and SQLite log growth #29531). The new date-stamped rollout gets the full history. 4 → 8 in one keystroke.
2. Spawn a subagent (Subagent spawns duplicate the full parent session history into each child rollout file (quadratic disk growth) #36557). Child rollouts open with the parent's complete history. One spawn after the resume lands you past 10.
3. Compact (Codex Desktop subagent session logs grow to 145GiB from repeated compacted replacement_history #31198). replacement_history re-embeds everything in the same file, for anyone who insists the copies be co-located. 811 compactions in a single thread has been demonstrated in production; two suffice here.
4. Add raw_output and display_output to CommandExecution. Matches the existing naming, one-line struct changes, passes review on the first pass.
5. Attach a screenshot (Browser tool auto-attaches a screenshot to nearly every Code Mode result, permanently inflating history and app-server memory #40175, Codex Desktop: screenshots re-persisted in full on every compaction and inherited by subagent forks - ~/.codex/sessions reached ~165 GiB (95% base64 images) #35458). Base64 PNGs re-persist on every compaction and are inherited by every fork, so this multiplies 2 and 3 instead of adding to them.
6. Log the item at TRACE (logs_2.sqlite: rmcp::service TRACE rows (~1.3 MB ListToolsResult dumps) bypass the noisy-target filter — 50-68 MB/day with a large MCP catalog #34291). logs_2.sqlite already accepts 1.3 MB ListToolsResult rows, and per logs_2.sqlite never reclaims freed pages: auto_vacuum=INCREMENTAL is set but never run, so the file grows monotonically despite working 10-day retention #35823 the pages are never reclaimed, making this the only copy that outlives its own retention policy.

Resume, then spawn: three actions, ten copies, zero code changes. That's how you 10x.

[argszero]

I traced the full persistence path on current main (94311d4475) and can confirm the duplication precisely — plus, importantly, which copies are actually consumed. Summary: only aggregated_output is ever read back from a persisted item; the other three strings (stdout, stderr, formatted_output) are write-only today, which makes a safe, backward-compatible dedup fairly easy.

Where the copies live

Each completed command produces a TurnItem::CommandExecution (codex-rs/protocol/src/items.rs:225) carrying four output strings:

• stdout (items.rs:246), stderr (249), aggregated_output (252), formatted_output (261)

constructed at codex-rs/core/src/tasks/user_shell.rs (completed branch ~line 298) and codex-rs/core/src/tools/events.rs:588 (emit_exec_end). This item is what lands in the rollout JSONL as the ItemCompleted record — so each item_completed record carries 3+ text copies as you measured.

The thread-history copy is real too: thread history is a separately materialized store that re-serializes items derived from the same ItemCompleted records (codex-rs/app-server-protocol/src/protocol/thread_history_projection.rs:21 → ThreadItem::from(CoreTurnItem::CommandExecution) at v2/item.rs:901, storing aggregated_output at :917). The legacy-migration path duplicates all four fields again (codex-rs/thread-store/src/local/rollout_migration/legacy_event.rs:213).

The redundancy is mostly structural, not informational

1. aggregated_output ⊇ stdout in all cases. aggregate_output (core/src/exec.rs:825) concatenates stdout then stderr, and both channels are individually truncated to the same retained_bytes_cap (exec.rs:711–728). So the aggregated string always contains the entire stdout field (identical prefix when stdout hits the cap). stdout is fully redundant given aggregated_output.

2. formatted_output is a pure function of the others. format_exec_output_str (core/src/tools/mod.rs:121) is formatted_truncate_text(build_content_with_timeout(aggregated_output, duration, timed_out)) — fully derivable at read time.

3. stderr is the only partially-redundant field: under byte-cap contention (stdout large), the stderr tail can be cut from aggregated_output while preserved in stderr — so it's the one field with actual information not contained elsewhere.

Who reads what (exhaustive grep over the workspace)

Consumer	Field used
TUI live rendering (tui/src/exec_cell/model.rs:23)	aggregated_output
TUI resume/replay (tui/src/chatwidget/command_lifecycle.rs:340,422; thread_transcript.rs:227)	aggregated_output
exec-server sync (exec/src/event_processor_with_jsonl_output.rs:169)	aggregated_output
wire thread history (v2/item.rs:917)	aggregated_output
search index (thread-store/.../search.rs:338)	excluded entirely
stdout / stderr / formatted_output on persisted items	no read consumers anywhere

The wire ThreadItem type (v2/item.rs:286) never carried stdout/formatted_output at all — the CLI/TUI have always been fed aggregated_output only.

Proposed minimal dedup (backward-compatible)

Since every consumer uses aggregated_output as the canonical text:

1. Stop populating stdout and formatted_output on new CommandExecutionItems (keep stderr for the truncation edge case, keep aggregated_output as the canonical stream).
2. The struct fields already have #[serde(default, skip_serializing_if = "Option::is_none")] (items.rs:247–261), so old records keep reading fine and new records degrade gracefully on any reader that does touch them later.
3. This removes ~2 of the 3 duplicated strings per item — the largest part of the 60% you quantified.

The thread-history copy is a separate, bigger question (it's a second store materializing from the same records, so it re-serializes aggregated_output again). Fully removing it would need thread history to reference the canonical rollout item instead of re-serializing — a structural change worth its own discussion; I'd scope the item-level fix first since it's small, safe, and captures most of the byte win.

Happy to implement the item-level change if that direction sounds right.

[argszero]

Implemented the first dedup step on my fork — formatted_output is no longer persisted for new commands. Branch: fix/rollout-dedup-command-output (commit 8c4cddb0ce on top of current main dc2ccc6843).

Scope: formatted_output only — and why not stdout too

While implementing, I found one consumer I'd missed in my analysis: the legacy event projection. Every ItemCompleted also derives a legacy ExecCommandEnd event at emit time (as_legacy_request_event, codex-rs/protocol/src/legacy_events.rs:211), which copies stdout from the item (stdout: self.stdout.clone().unwrap_or_default()). That legacy event feeds the wire (event_mapping.rs:457 → build_command_execution_end_item) and is asserted by the CLI/suite tests (e.g. suite/user_shell_cmd.rs checks ExecCommandEnd.stdout contains the command output). So stdout on the item is not write-only — it's the source for the legacy event stream. It has to stay for now; dropping it cleanly would require reworking the legacy projection, which is a larger change than this issue warrants on its own.

formatted_output, by contrast, has zero read consumers in production (the wire ThreadItem never carried it; the trace reducer consumes a different dispatch payload). It's also a pure function of aggregated_output + duration/timed_out/truncation policy (format_exec_output_str), so nothing is lost.

The change

Stop populating formatted_output on new CommandExecutionItems:

• core/src/tasks/user_shell.rs — all three completion branches (cancelled / success / error)
• core/src/tools/events.rs — emit_exec_end (removed the field from ExecCommandResult and dropped the format_exec_output_str call there)

The struct field stays (#[serde(default, skip_serializing_if = "Option::is_none")]), so old rollouts still deserialize and new records simply omit the field. Also removes the per-command truncation computation at emit time.

Verification

• codex-core unit: 2358/2358 pass
• codex-protocol unit: 332/332 pass
• Suite: user_shell_cmd 8/8, unified_exec + shell_snapshot 67/67 pass
• Full codex-core suite (1490 tests): failure set is identical to clean-main baseline (stash-diff) — only the known env-dependent groups fail locally (code-mode needs the V8 codex-code-mode-host, cli_stream/hooks need their harness env), zero new failures
• cargo clippy --tests -p codex-core: clean for the touched files

This removes one of the three per-item copies (the largest part for verbose commands, since formatted_output was the truncated-but-still-large view). The remaining stdout duplication (legacy-event contract) and the thread-history 4th copy (separate materialized store) are follow-ups — happy to take them on separately if this direction looks right.