fix(test): identify pi's compaction summarization without sniffing its prompt wording - #47
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
mainhas been red since 2026-09-20 in theTest (pi latest, …)matrix cell only — the0.83.0,0.84.1and devDependency cells, Typecheck and the tarball job all pass (last green nightly: 09-19). Every open PR inherited the same two failures, so each one looked self-inflicted:traces the compaction summarization call so the session total matches pi—0 !== 1carries the compaction summary after a compaction, not the messages it replacedThis is a stale test fixture, not a plugin defect. No
src/change: the plugin traced correctly throughout.Root cause
The mock provider identified pi's compaction-summarization request by the
<conversation>XML wrapper around the history:pi 0.87.1 (earendil-works/pi#9908, split-turn summaries being refused by Claude Fable 5.1) rewrote the turn-prefix summarization prompt to markdown headings. The main history path still uses
<conversation>, which is why 0.86.0 was unaffected and 0.87.1 is not:The summarization call still happened — the mock just stopped recognizing it, fell through to the ordinary stage-based agent script, and answered a summarization request with a tool call. Compaction could then never complete, so nothing was traced. Measured on unmodified
main, usingtools.length === 0as ground truth for "this is a summarization call":0 !== 1Note this is invisible in the CI log: the suite prints TAP output only, never provider request bodies.
What changed
completeSummarization, which bypasses the agent loop and therefore declares no tools; every agent-loop call offers the built-in tools. Verified stable across pi 0.83 → 0.87.stagecounts visible tool results, and a compaction replaces the ones it summarized — so the script rewound, re-served theread README.mdit had just summarized away, grew the context back over the threshold, and pi compacted again without end (11k+ provider calls until the harness SIGKILL). The guard keys on the mock's own summary text, not pi's.Verification
Full suite green on every matrix version, including 0.86.0, which previously livelocked:
latest)Typecheck clean. Also verified end-to-end by applying this patch to two open PRs and running against
latest: #41 → 141/141, #46 → 143/143 — #46's 143 is exactly the total CI reported as "143 tests, 141 pass, 2 fail".