bug: Snapshot.diffFull serialises full patch text into message rows, causing unbounded DB growth and web UI hang

[Samyak2406]

Summary

opencode web becomes completely unresponsive when the project working directory contains an untracked runtime-generated directory (e.g. a Docker bind-mount for a database). Two structural bugs in the Snapshot.diffFull → SessionSummary.summarize pipeline cause unbounded SQLite row growth.

Reproduction

1. Start a Docker stack that bind-mounts a database data directory inside the repo working tree (e.g. PostgreSQL/QuestDB data — 4 300+ binary files, ~12 GB). Directory is not in .gitignore.
2. Run any opencode session with a few agent turns.
3. Open opencode web. UI hangs indefinitely.

Root Cause (v1.18.15)

SessionSummary.summarize()
  → SessionSummary.computeDiff({ messages })
    → Snapshot.diffFull(startHash, endHash)
      → git diff --numstat <h1> <h2>
      → git diff --name-status <h1> <h2>
      → git cat-file --batch  (bulk content for every changed file)
      → unified diff generated per file
    ← returns [{ file, patch, additions, deletions, status }]
  ← stored into message.info.summary.diffs
  → Session.updateMessage()  ← written to SQLite, no size guard

Bug 1 — track() commits ignored files into snapshot trees

track() runs git add --all without honouring .gitignore. Every file in the bind-mount directory is staged and committed into the snapshot bare repo, then appears in every subsequent diffFull call.

Bug 2 — Full diff text serialised into message.data with no size guard

SessionSummary.summarize writes the complete diffFull result (raw unified diff for every file) into message.info.summary.diffs and persists it to SQLite. No byte-length check exists before the write.

Single user-turn message: 4 325 file diff entries, ~400 MB. With parallel subagent sessions: opencode.db reached 66.8 GB in hours.

Bug 3 — Binary files produce { patch: "" } entries that are still serialised

diffFull correctly detects binary files via --numstat but still pushes { file, patch: "", additions: 0, deletions: 0 } into the result, which is meaningless and persisted.

Observed Impact

Metric	Observed
opencode.db size	66.8 GB
Largest message row	413 MB
GET /session/:id/message payload	1.2 GB
opencode web CPU	371% sustained (GC loops)
Web UI	Completely unresponsive

Proposed Fixes

Fix 1 — Honour .gitignore in track()

Replace unrestricted stage-all with one that respects ignore rules:

# instead of: git add --all
git add $(git ls-files --cached --others --exclude-standard)

Or libgit2 equivalent: exclude ignored paths when building the index for snapshot commits. This is the only fix that prevents the bloat from occurring.

Fix 2 — Store snapshot hashes in message.data; compute diffs on demand

{ "summary": { "startSnapshot": "<40-char hash>", "endSnapshot": "<40-char hash>" } }

Add: GET /session/:sessionID/message/:messageID/diff — computes the diff live from snapshot hashes. This is how Git itself works: commits store tree pointers, not serialised diff text. The two hashes are already stored on the step-start/step-finish part rows.

Fix 3 — Omit binary files from the diff result

- L.push({ file, patch: c.binary ? "" : Q0(...), additions, deletions, status });
+ if (!c.binary) L.push({ file, patch: Q0(...), additions, deletions, status });

Data Migration

A startup data_migration should trim bloated existing rows: find message rows where length(data) > 50 000 and role = 'user', strip summary.diffs entries with empty patches or from ignored paths, store only { startSnapshot, endSnapshot } hashes. Snapshot commits themselves remain intact in the bare repo under ~/.local/share/opencode/snapshot/.

Environment

• opencode v1.18.15, macOS arm64
• Trigger: Docker bind-mount of PostgreSQL + QuestDB data inside the repo working directory

[github-actions]

This issue might be a duplicate of existing issues. Please check:

• OOM and ~30 GB database: huge workspace-file diffs embedded in message summaries are re-serialized in full on every message.updated event #43551: OOM and ~30 GB database: huge workspace-file diffs embedded in message summaries are re-serialized in full on every message.updated event — reports the same root cause chain (uncapped diffs in summary.diffs → massive DB growth → OOM on resume) triggered by a large continuously-written file rather than a bind-mounted directory.
• [BUG]: message.updated.1 re-serializes summary.diffs on every update, making event writes quadratic in diff size #42748: [BUG]: message.updated.1 re-serializes summary.diffs on every update, making event writes quadratic in diff size — specifically tracks the quadratic amplification of message.updated.1 events from summary.diffs, with nearly identical measured impact (51 GB DB, 13 MB peak event row, same code path in packages/opencode/src/session/summary.ts).
• Excessive storage: event table stores full message snapshots per streaming update — community tool available #41175: Excessive storage: event table stores full message snapshots per streaming update — broader tracking issue for unbounded opencode.db growth via the event table.

Bug 1 (snapshot track() staging ignored files via git add --all) appears to be a new, distinct contribution not covered by the above issues — the others all assume the diff content comes from legitimately tracked files. You may want to note that distinction when cross-referencing.

[Samyak2406]

@kitlangton @Hona PR opened in #46140 addressing the transient file staging failure.