feat(coref): co-reference-aware compaction

[amiddavid]

Implements co-reference-aware compaction — picking what to
drop at a threshold crossing by looking at back-references rather than at content or age — and
measures the substrate it depends on.

What's here

• internal/coref — the tier-1 reference index: which identifiers each tool output
  introduced, and whether any later model turn carried them forward. No bifrost, no components,
  no tokenizer dependency, because it has to stay interchangeable with deploy/harbor/coref.py's
  definition. Its fixture is the twin of coref_fixture.py, negative control included and
  asserted.
• components/offload/coref — the Offload component. The one component that mutates the
  cached prefix on purpose, so: batched cuts, a per-session rewrite_budget, latched decisions
  replayed byte-for-byte, repairLostFreeze deliberately not consulted, side-effect-free planning.
• deploy/harbor/coref.py + two converters (cc_capture.py, runlog_capture.py) — the
  measurement pass, plus the plumbing to run it on Claude Code transcripts and benchmark harness
  logs without an eval-box run.
• Docs — the proposal, measured results, a
  component reference, and a
  one-page cheat sheet for the vocabulary.

The measurement, and the finding

Run on three corpora (none of them the eval-box captures — those were unreachable). The headline is
that they disagree by a factor of three:

	Claude Code (interactive)	UltraHorizon	LOCA-bench
unreferenced	23%	78%	95%
closed	15%	8%	0%
open	60%	13%	4%
…restricted to ≥20 later turns	21%	70%	70%

Reference density is a property of the workload, not a constant. Interactive work on a coherent
codebase keeps returning to the same files and errors; benchmark tasks survey, extract, and move on.
The last row bounds the obvious tail bias and the ordering survives it.

Three more results:

• Distance is not the discriminator; repetition is. Sweeping closed_dist over a 10× range
  moves the answer 2–3 points; sweeping open_reps 2→6 moves it 18. And 44% of mass was last
  referenced 40+ messages ago while 60% is open — most referenced mass is old and still hot. A
  distance-based A/B split would confidently cut repeatedly-referenced content.
• A reference consumes a median 18.7% of what its output introduced — hypothesis A confirmed.
• Break-even is workload-dependent: median required T is 95 turns on interactive traffic
  (15/30 sessions clear it) against 17 and 14 on the benchmarks. Batching moves it from unreachable
  to comfortable-on-benchmarks, marginal-on-interactive. Steps and deferred agent-compaction remain
  the load-bearing justification.

LOCA's 0% closed is the proposal's own §8 prediction landing: it argued LOCA would be a tier-2/3
stress test where references arrive transformed past what a substring match can see.

One bug worth calling out

The first measurement said 71% referenced. The rule deciding "identifier vs English word" accepted
any token of 10+ characters, so description, transparency, efficiency and conditions scored
as references. A manufactured reference makes an output look load-bearing, so this class of bug
fails by silently declining to compact — invisible to any metric that counts only what the
component did. Corrected to require interior structure, a digit, or camelCase; every false positive
is now a regression case. The residual is bounded at ~6 points of under-reporting.

Status

Opt-in, in no preset. cut_unreferenced is on by default and justified on every corpus.
cut_closed is off: its yield ranges 0–15% by workload, which is no basis for a default.

Next, in order: re-run on capture-swe/capture-tb at the eval box → enable cut_closed there →
observe-mode expand rate as the precision inner loop → only then the scored benchmarks.

Verification

gofmt clean · go vet ./... clean · go test ./... 24 packages, 0 failures · fixture reproduces
its documented ground truth.

[amiddavid]

pls explain this sentence, perhaps add an example

[amiddavid]

Rewritten with a worked example rather than the assertion. It now walks turn 4 reads src/auth.py / turn 5 says "the bug is TOKEN_GRACE_SECONDS" / thirty turns later the agent is on tests — and shows that asked "is the turn-4 output still needed?", conversationGoal can only answer "the task is still about auth", which is true of every output and so decides nothing. The fact that settles it (the one value taken sits in turn 5, and turn 5 isn't going anywhere) is positional and backward-looking, which that signal cannot represent at all.

[amiddavid]

pls explain this paragraph in more details, I find it hard to follow, especially for a proposal document

[amiddavid]

Expanded into a bulleted walk-through of the two existing pieces and what each contributes: deterministic.go's important-key list is already an answer to "which parts of an output would a model carry forward?", and contain.go today checks a shrunken output is a subset of its original. The reusable idea is the second one run backwards — today it asks "is this compacted text contained in the original?", inverted it asks "is this span of the original contained in a later message?", and the same primitive becomes a reference detector. Same test, opposite direction: one validates a rewrite, the other measures reuse.

[amiddavid]

add an example column

[amiddavid]

Added an EXAMPLE column. Same reference at each tier: Tier 1 TOKEN_GRACE_SECONDS = 0 reappearing verbatim in an Edit argument; Tier 2 [{"ms":1200},{"ms":1800}] → "total latency is 3 seconds" (the 3 appears nowhere — it was computed); Tier 3 a directory listing → "as I saw earlier, the tests live beside the source", which is unmistakable to a reader and shares no token at all.

[amiddavid]

I'm not sure I fully agree. consider this scenario:
a tool output returned: { "name": "david", "id": 123, "address": "foobarbaz"}
, { "name": "osher", "id": 235, "address": "banana"} the agent said, I need to remember david 123 address.
the address itself wasn't coref, but the tool output is needed and cannot be removed.

[amiddavid]

i.e. doesnt this contradicts case B?

[amiddavid]

You're right, and this was the most valuable comment on the PR — it found a real bug, not just a wording problem.

I ran your exact example through the index rather than reasoning about it, and it's worse than you flagged. david, 123, foobarbaz are short lowercase words and a 3-digit number — precisely what the precision rules in §2 exclude — so the output yields zero trackable tokens. Zero novel tokens means zero references, which scored unreferenced, which is the class the default config cuts. So the shipped default would have deleted that output while the agent was still asking for the address.

Two separate defects, both now fixed in e7a2623:

1. Your conceptual point. "Any reference is a surviving copy" is too strong. The model referenced an anchor (david, 123) in order to point at a payload (foobarbaz) it never restated. An exact matcher can't distinguish an anchor reference from a payload reference — so closed can't rest on "referenced once, long ago" alone. That is now stated as the reason cut_closed ships off, rather than mere caution. It also inverts my §7 reading of used_frac: a low value is ambiguous, not evidence for case A, because "took the value, rest is chaff" and "took an anchor, still needs the payload" look identical.
2. The concrete one. refs == 0 conflated two opposite states — "introduced 200 identifiers, nobody touched one" (evidence of deadness) and "introduced nothing I can see" (absence of evidence). There's now an opaque class that is never cut at any setting. It is not a corner case: 8% of tool-output mass on interactive traffic, 20% on UltraHorizon, 40% on LOCA-bench — the last being 11 outputs averaging 22k tokens of exactly the record-dump shape you described.

Re-measuring dropped the headline unreferenced figures from 23/78/95% to 13/51/22%, and break-even from 15/30 to 9/30 sessions. Your counter-example is now a test case on both sides of the implementation.

[amiddavid]

I dont see improvement-plan in the docs

[amiddavid]

docs/results/improvement-plan.md does exist on main (verified with git cat-file -e main:docs/results/improvement-plan.md) and is in the mkdocs nav, so the link resolves on the published site. It's just not in this PR's diff, so GitHub can't render it as a clickable target here.

[amiddavid]

can u explain this more, I'm not following

[amiddavid]

Expanded. The short version: Tier 2 is a reference that arrived transformed, so by definition no substring match can find it — what's countable is a symptom. If a model turn states a numeric value appearing nowhere in any earlier message, it computed that number from something, and that something was almost certainly a tool output. 2% of turns look like that on interactive traffic, which is why a zero-LLM first version is viable.

Two caveats now stated, and the second is a self-inflicted one worth knowing: it's a lower bound (only numeric transformations leave this trace — reworded prose is invisible), and tightening the identifier rules also blinded the proxy, since bare numbers now need 5+ digits and most computed values are small. So its 0% on LOCA means "none among tokens the tokenizer still accepts", not "none" — on a corpus with 0% closed and 40% opaque, the honest reading is that Tier-2 references there are common and simply unmeasured.

[amiddavid]

if it a recent turn, it might not had the chance to be referenced, dont we need to guard from this ?

[amiddavid]

Good catch, and no — there was no guard, which was a real gap. An output near the tail has had no chance to be referenced, so scoring it as unused would make a batched pass preferentially cut the most recent context, which is the worst possible choice. mask avoids this with keep_recent; coref had nothing.

Added min_later_turns (default 8): an output with fewer model turns after it is treated as open regardless of everything else. Worth noting what the state was before — the measurement had bounded this bias (LOCA's raw 95% fell to 70% when restricted to outputs with 20+ later turns) but nothing in the component guarded against it. Bounding a bias in a report is not the same as not having it in the code.

[amiddavid]

dont tune it, but still matters?

[amiddavid]

Fair — that phrasing was self-contradictory. Rewritten to what's actually true: closed_dist is load-bearing but flat. Set it to 0 and the closed class stops existing, so it certainly matters; but anywhere in 4–40 gives the same answer within 2–3 points, so there's no return on tuning it. Leave it at the default and spend the effort on open_reps, which moves the answer 18 points across the same kind of range.

[amiddavid]

love this. 💌
though it depends how much is being cut isn't it? and for the determinsitic one, its cheap to calculate and can tell u how much deferring is happening.

I would also appreciate some thought on what it means for larger context windows that are now more and more frequent.... up to 1M

[amiddavid]

Thanks — and both of your points landed in the doc.

On "it depends how much is being cut": yes, and more sharply than I'd written it. The agent-compaction prize is a step function, not a slope — you either drop below the threshold or you don't, and cutting 90% of what was needed to get there is worth nothing. Which argues for sizing the batch against the threshold distance, something min_batch_frac cannot currently express. Noted as a limitation.

On deterministic measurement: agreed, and it's the cheapest real metric available here — compare the API-reported usage against the documented compaction threshold and count the turns of headroom the cut bought. No benchmark scoring, no seeds, no LLM judge. It isn't in the metrics yet; it should be.

On 1M windows — I worked this through and the answer surprised me. Break-even is scale-invariant. Rearranged, S × T > 11.5 × W is T > 11.5 × (W/S) — it depends only on the ratio of rewritten suffix to cut mass, never on absolute size. A 1M transcript with the same density of cuttable mass needs the same T. So a bigger window neither rescues nor damns the token economics; it only moves when the trigger fires. What improves the ratio is cutting a larger share of what lies after the shallowest cut — an argument for cutting deep and rarely, not for cutting more.

Three things do genuinely change, now a table in §7 of the cheat sheet: cache-read becomes the entire bill (so coref is a cost play at 1M rather than a fit play — the strongest argument for it there); the agent's own compaction recedes to ~967k, making that prize rarer but much larger; and the index cost scales linearly, so an incremental per-session index stops being an optimization and becomes a requirement.

[amiddavid]

since I'm not familiar with this repo yet, I would appreciate if you can in a comment explain this more

[amiddavid]

Expanded both from first principles rather than by name.

TailOnly is a helper on Ctx answering "may I safely modify the message at index i?" It returns false for anything the provider has already cached, because editing cached content breaks the prefix hash and forces a cache-write of everything after it. Every other age-based offloader (mask, failed_run, collapse) consults it and declines. coref deliberately ignores it — reaching into the cached prefix is the point, since by the time a session crosses the threshold all the mass is back there — which is exactly why its spend is budgeted rather than forbidden.

repairLostFreeze needs the background first: an offloader freezes its replacement text against the original's content hash and replays it every turn so the bytes stay stable. If the store drops that record (TTL, eviction), it would normally decline to act at depth — but then the message reverts to full text, which is itself a prefix change. So mask and failed_run may re-derive even deep in the prefix: their replacement is a pure function of (content, config), so re-deriving reproduces byte-for-byte what the provider already cached. coref must never do this, because its decision depends on the whole transcript — re-deriving against a longer one can yield a different class and different bytes, the precise flip the repair exists to prevent.