Skip to content

fix(mcp): inject send_feedback once, on the first tools/list page - #4953

Merged
gesh merged 3 commits into
mainfrom
posthog/mcp-feedback-pagination
Sep 15, 2026
Merged

gesh merged 3 commits into
mainfrom
posthog/mcp-feedback-pagination

Conversation

@gesh

@gesh gesh commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

Problem

The SDK appended the virtual send_feedback tool to every tools/list page. A cursor-following client concatenates pages, so it saw the tool once per page.

Changes

Append the tool once, at the end of the first page — the page every client reads, including clients that never follow cursors. "First page" means a request with no cursor; an empty-string cursor is a continuation page. A single-page catalogue keeps today's behavior at zero extra cost.

Name conflicts are the MCP owner's responsibility, with cheap page-local detection instead of a catalogue walk:

  • A real tool named send_feedback on the first page wins: the SDK warns, skips injection, and forwards its calls (the existing first-page probe on the call path, unchanged).
  • A real tool on a later page is shadowed. The SDK logs a warning when a client fetches that page.
  • Both warnings point at the fix the SDK already ships:
instrument(server, posthog, {
  collectFeedback: { toolName: 'posthog_feedback' },
})

The call path, get_more_tools, PostHogMCP, and the option surface are untouched.

  • Tests: a paginated-catalogue suite (first-page-only injection, empty-string cursor semantics, first-page collision, later-page shadowing) plus real-server probes on both SDK majors, including a colliding paginated catalogue and a per-request factory.

Release info Sub-libraries affected

Libraries affected

  • @posthog/mcp (not in the template list)

Checklist

  • Tests for new code
  • Accounted for the impact of any changes across different platforms
  • Accounted for backwards compatibility of any changes (no breaking changes!)
  • Took care not to unnecessarily increase the bundle size

If releasing new changes

  • Changeset file added (.changeset/mcp-feedback-pagination.md)

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Authored with Claude in PostHog Desktop. Reworked from an earlier revision that walked the whole catalogue before every injection and interception; the walk, its budgets, and its fail-safe plumbing are gone in favor of the owner-responsibility contract above. Verified: 883 unit tests, all four harness lanes green on both SDK majors, lint and build clean.


Created with PostHog Desktop

@gesh gesh self-assigned this Sep 14, 2026
@gesh
gesh marked this pull request as ready for review September 14, 2026 09:36
@gesh
gesh requested review from a team as code owners September 14, 2026 09:36
@gesh
gesh removed request for a team September 14, 2026 09:37
@greptile-apps

greptile-apps Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
### Issue 1
packages/mcp/src/extensions/instrumentation.ts:853
**Collision state leaks across clients**

If a long-lived server returns role- or session-specific tool catalogues, one client that lists a real `send_feedback` tool sets this server-wide flag permanently. Other clients then lose virtual-tool injection and feedback-call interception even when their catalogues have no collision. Concurrent page requests can also process the final page before the colliding page and advertise both tools. Scope collision state to each session or list traversal, and reset it when a new enumeration begins.

### Issue 2
packages/mcp/harness/dual-era/probe-feedback.mjs:136
**Probe leaves fetch failures unhandled**

The new `fetch()` and response parsing can reject without a `catch`, so the probe exits with an unstructured exception instead of recording a failed check. The repository requires asynchronous calls that may fail to be handled with `try/catch`. Wrap this request and parsing path and report failures through `check`; this requirement must be satisfied before merging.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "test(mcp): add a plain collectFeedback l..." | Re-trigger Greptile

Comment thread packages/mcp/src/extensions/instrumentation.ts Outdated
Comment thread packages/mcp/harness/dual-era/probe-feedback.mjs Outdated

@gesh gesh left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated comment by QA Swarm — not written by a human

QA Swarm review complete — see inline comments. Verdict and evidence in the pinned summary comment.

Comment thread packages/mcp/src/extensions/instrumentation.ts Outdated
Comment thread packages/mcp/src/__tests__/tools-list-envelope.test.ts Outdated
Comment thread packages/mcp/src/types.ts Outdated
Comment thread packages/mcp/src/extensions/instrumentation.ts
Comment thread packages/mcp/src/extensions/instrumentation.ts Outdated
Comment thread packages/mcp/harness/dual-era/probe-pagination.mjs
Comment thread packages/mcp/src/extensions/instrumentation.ts Outdated

gesh commented Sep 14, 2026 •

Copy link
Copy Markdown
Member Author

Note

🤖 Automated comment by QA Swarm — not written by a human

Multi-perspective review: router (cheap-first pass) + delegated reviewers (qa-team, paul-reviewer, xp-reviewer, security-audit as warranted)

Verdict: 💬 APPROVE WITH NITS (round 6 @ 3a28019)

The PR was rewritten from scratch on top of main: the whole-catalogue ownership walk (and its budgets, loop guards, _meta forwarding) is gone, replaced by an owner-responsibility contract — first-page-only injection, page-local collision detection, and the collectFeedback: { toolName } rename as the documented fix. Round 6 reviewed the new ~585-line diff as a fresh change: LOW danger, HIGH confidence, no delegation. One MEDIUM finding, out of scope but worth tracking.

Key findings

  • 🟡 MEDIUM — get_more_tools still injects on every page (instrumentation.ts:827–841). The sibling block above the fix has no first-page guard, so paginated catalogues still duplicate get_more_tools per page — the defect class this PR fixes for send_feedback. Pre-existing, not a regression; moot if reportMissing is removed as planned, otherwise worth a follow-up.

Superseded by the rewrite

Round 4/5's three open findings (final-page single-page-client visibility, get_more_tools check asymmetry, ownership-cache leak across catalogues) all targeted the catalogue-walk implementation, which no longer exists. The new revision injects on the first page (resolving the single-page-client concern directly), performs no walk anywhere, and caches ownership only from served pages.

Convergence

None — single reviewer round.

Reviewer summaries

Reviewer Assessment
🧭 router (round 6, sonnet) LOW danger, HIGH confidence, no delegation. Traced both dispatchers through the shared getTracedToolsList, verified cursor-presence semantics ("" = continuation), and ran the suites: tools-list-envelope 12/12, probe-pagination 18/18, probe-feedback 13/13.
Previous rounds (5)

round 1 @ 1a35a9c — ⚠️ REQUEST CHANGES: 1 HIGH (sticky-flag lifetime), 3 MEDIUM, 2 LOW, 1 NIT. Six threads actioned in 52c2974.
round 2 @ 52307c7 — 💬 APPROVE WITH NITS: 1 MEDIUM, 1 NIT. Both fixed in 97e4020.
round 3 @ 5e0df8b — 💬 APPROVE WITH NITS: no new findings; oxfmt CI fix landed in 5e0df8b.
round 4 @ 74d32cb — 💬 APPROVE WITH NITS: 2 MEDIUM, 2 LOW, 1 NIT; two closed, three carried.
round 5 @ 6958b32 — 💬 APPROVE WITH NITS: delta-only review, nothing new; three round-4 findings left open, all later superseded by the round-6 rewrite.


Automated by QA Swarm — not a human review

@gesh gesh left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated comment by QA Swarm — not written by a human

QA Swarm round 2 — reviewed the delta 1a35a9c..52307c7. The refactor is behaviour-preserving and every check is green; two findings inline. Verdict in the pinned summary.

Comment thread packages/mcp/src/extensions/instrument-highlevel.ts Outdated
Comment thread packages/mcp/README.md Outdated
@gesh gesh added the stamphog label Sep 14, 2026 — with PostHog
Comment thread packages/mcp/src/extensions/instrumentation.ts Outdated
@veria-ai

veria-ai Bot commented Sep 14, 2026 •

Copy link
Copy Markdown

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 2 · PR risk: 0/10

gesh commented Sep 14, 2026

Copy link
Copy Markdown
Member Author

Review feedback is addressed in 74d32cb. The open decision on the collision-flag lifetime is resolved with option 2: the flag is gone, and collision state is scoped to each list traversal.

How it works now

  • isToolAdvertised follows nextCursor through the whole raw catalogue, with a 100-page cap. A real owner on any page keeps its calls, so the call path needs no memory of earlier listings.
  • The final tools/list page re-walks the raw listing when the request came through a cursor. A real owner on any earlier page blocks injection. A single-page catalogue is checked locally, with no extra handler calls.
  • MCPAnalyticsData.feedbackToolShadowed is removed. No state persists between requests, so one client's filtered catalogue cannot suppress the virtual tool for other clients, and a per-request factory gets the same verdict as a long-lived instance.

Thread disposition

  • Greptile P1, Veria "cross-client feedback suppression", QA Swarm HIGH (sticky-flag lifetime): fixed as above. The QA Swarm reproduction (3 pages, real owner on the middle page, per-request factory) is now a harness fixture in probe-pagination.mjs and passes on both majors.
  • Greptile P2 (unhandled fetch in probe-feedback.mjs): this was fixed in 5e0df8b, before this round. The fetch and the parse sit in a try/catch and report through check.
  • QA Swarm MEDIUM (tests assert a reused-instance boundary): the boundary no longer exists. The test comments now say so, and two new unit tests exercise fresh instances per request.
  • QA Swarm LOW (probe cannot catch the per-request defect): the colliding-catalogue fixture closes this. Its listing check fails against the previous commit on v2.
  • get_more_tools stays per-page by design; the code comment in getTracedToolsList records why. The non-paginating-client trade-off stays documented in the README.

Divergence from posthog-python: the Python SDK keeps a per-instance flag. The JS SDK cannot, because per-request factories are a first-class JS topology, so it asks the catalogue instead. The observable contract (virtual tool once, on the final page; a real owner always wins) is the same.

Cost note: the re-walk runs only on the final page of a genuinely paginated enumeration, and on calls that use the configured feedback tool name. Both are rare next to ordinary tool calls.

Verified locally: 882/882 unit tests, probe-pagination 8/8 and probe-feedback 6/6–7/7 on both majors, lint and format clean.

@stamphog stamphog Bot removed the stamphog label Sep 14, 2026

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved yet — waiting on the conditions below.

Re-add the stamphog label to request another review once you have addressed this.

Deterministic gates denied this PR (package.json script/hook edit tripped the deny-list, and size/scope classified it T2-never), so it cannot be auto-approved regardless of code quality. The package.json change itself only adds a new probe invocation to existing test scripts (no lifecycle hooks), but a prior automated review flagged a real cross-client state-leak issue that later commits appear to have redesigned away (re-walking the raw catalogue instead of caching a flag) — a human should confirm that fix is complete before merging, since this changes MCP tool interception/dispatch behavior.

  • 👍 on the PR from greptile-apps[bot].
  • Gate denial: package.json scripts changed (deny-listed), plus PR classified T2-never by size/scope
  • Reviewer (Greptile P1, veria-ai) flagged collision-state leaking across clients/sessions; later commits appear to redesign the mechanism to re-walk the catalogue instead of caching state, but this should be confirmed by a human reviewer rather than assumed fixed
  • No CODEOWNERS/ownership match found, so there's no clear owning-team assurance path
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✗ matches: deps_toolchain (scripts/hooks changed in package.json)
size ✓ 497L, 9F substantive, 784L/12F incl. docs/generated/snapshots — within ceiling
tier ✗ classified as T2-never: T2-never (784L, 12F, two-areas, fix)
stamphog 2.0.0b4 .stamphog/policy.yml @ 74d32cb · reviewed head 74d32cb

@gesh gesh left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated comment by QA Swarm — not written by a human

QA Swarm round 3 — reviewed 424c45a..74d32cb (router on the cheap tier, one mandatory delegation to the stronger tier on the cursor-walk and the two SDK-major wrappers). The three findings from rounds 1 and 2 are confirmed fixed. Verdict and evidence in the sticky summary comment.

Comment thread packages/mcp/src/extensions/instrumentation.ts Outdated
Comment thread packages/mcp/src/extensions/instrumentation.ts Outdated
Comment thread packages/mcp/src/extensions/instrumentation.ts
Comment thread packages/mcp/src/extensions/instrumentation.ts
Comment thread packages/mcp/src/extensions/instrumentation.ts Outdated
@gesh gesh added the stamphog label Sep 14, 2026 — with PostHog
@stamphog stamphog Bot removed the stamphog label Sep 14, 2026

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved yet — waiting on the conditions below.

Re-add the stamphog label to request another review once you have addressed this.

Gates denied this PR (deny-listed package.json script edit plus a size/scope classification that requires human sign-off), and it modifies MCP tool-injection/collision-detection logic — risky surface. Automated reviewers (QA Swarm, Greptile, Veria) converged on approval after fixes, but one MEDIUM finding is explicitly left as an open product decision for a human ("needs your call"), so this needs a maintainer to make that call rather than auto-approve.

  • 👍 on the PR from greptile-apps[bot].
  • Gate denied: tier classified T2-never (770 lines/12 files) and deny-list matched a package.json script change — both require human review per policy.
  • The package.json change itself is benign (adds a new integration probe to existing test:integration:sdk-v1/v2 scripts, no lifecycle hooks or new script keys).
  • QA Swarm round 5 leaves an open design decision unresolved: whether injecting on the final page (current behavior) vs. first page is the right tradeoff for non-paginating clients — this is a product call, not a bug, but needs a human to confirm.
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✗ matches: deps_toolchain (scripts/hooks changed in package.json)
size ✓ 482L, 9F substantive, 770L/12F incl. docs/generated/snapshots — within ceiling
tier ✗ classified as T2-never: T2-never (770L, 12F, two-areas, fix)
stamphog 2.0.0b4 .stamphog/policy.yml @ 6958b32 · reviewed head 6958b32

@gesh gesh changed the title fix(mcp): inject send_feedback only on the final tools/list page fix(mcp): inject send_feedback once, on the first tools/list page Sep 14, 2026
Comment thread packages/mcp/src/extensions/instrumentation.ts Outdated
@gesh
gesh requested a review from a team September 14, 2026 17:02
Comment on lines +1030 to +1033
// Falsy, not `=== undefined`: a server never issues an empty-string cursor,
// so a falsy cursor can only be a client's spelling of "first page" — the
// same reading `nextCursor` gets everywhere else in this file.
if (!feedbackOptions || request.params?.cursor) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MCP allows an empty string as a cursor. This check treats cursor: "" as the first page. The two nextCursor checks also treat "" as the end of the list.

I tested this case. The SDK adds send_feedback twice. If a later page contains a real tool with that name, the SDK handles its calls instead of the application.

Please check for null or undefined instead. Please also correct the test that describes empty cursors as invalid.

MCP cursor requirements

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9e65f15. You are right, and the spec is explicit: "Don't make any determination based on cursor value other than whether a non-null value was provided (e.g. an empty string is a valid cursor and thus MUST NOT be treated as the end of results)."

Every cursor decision now turns on presence alone. Three sites changed: the first-page test is request.params?.cursor != null, the end-of-walk test is response.nextCursor == null, and the third folded into the merged function. The comment claiming a server never issues an empty cursor is gone.

The test that described empty cursors as invalid now pins the opposite, and two more cover the return direction:

  • treats an empty-string cursor as a continuation page, not the first one
  • reads an empty-string nextCursor as more pages, so a real owner beyond it blocks injection
  • keeps a real owner's calls when only an empty-string cursor reaches its page

Your reproduction is also a harness fixture now, on both majors. Against the previous commit it reports exactly what you saw:

✗ v1 · empty-string cursor · send_feedback is listed once, on the first page
  {"pageOne":["page_one_tool","send_feedback"],"pageTwo":["page_two_tool","send_feedback"]}

Comment on lines +729 to +731
for (let page = startCursor === undefined ? 0 : 1; page < MAX_OWNERSHIP_PROBE_PAGES; page++) {
const params = cursor === undefined ? {} : { cursor }
const response = (await listHandler({ method: 'tools/list', params }, extra)) as CompatibleToolsListLike

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These page requests omit the original request’s _meta. A list handler can require fields from _meta, such as protocolVersion.

I tested a handler with this requirement. The first page succeeds. The internal request fails because protocolVersion is missing. The SDK then omits the feedback tool.

This test passes before the PR change and fails after it.

Please preserve the required metadata when the SDK requests more pages. Apply the same change to the check made before a tool call.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9e65f15. Confirmed: 2026-07-28 makes io.modelcontextprotocol/protocolVersion and clientCapabilities required on every request, and "A request missing any required field is malformed; the server MUST reject it with JSON-RPC error code -32602." A bare { cursor } probe cannot pass a handler that enforces that, and the walk's fail-safe then drops the tool.

Internal page reads now carry the triggering request's _meta. Applied on both paths, since the merged function takes the request.

progressToken is the one key left out. It opts the client's request into progress notifications, and an internal read must not report progress against it — clearest on the call path, where the token belongs to a tools/call and the read is a tools/list. A unit test pins that it stays behind.

One detail worth recording for whoever reads this next: the harness fixture keys on a vendor _meta key rather than a reserved one. SDK v2 lifts the reserved io.modelcontextprotocol/* keys out of params._meta into the envelope before a handler runs, so a fixture keyed on a reserved field would only exercise v1. Failing against the previous commit:

✗ v1 · _meta · injection survives a list handler that requires request metadata  ["page_one_tool"]
✗ v1 · _meta · the pre-call walk survives it too  {"content":[{"type":"text","text":"called: send_feedback"}]}

Same two on v2. Three unit tests cover the listing path, the call path, and the progressToken exclusion.

Comment on lines +725 to +729
// A `startCursor` means the caller already examined one page locally, so
// it consumes one unit of the budget. The listing and call-time walks then
// reach the same total distance, and a tool near the budget's edge can
// never be advertised without also being callable.
for (let page = startCursor === undefined ? 0 : 1; page < MAX_OWNERSHIP_PROBE_PAGES; page++) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional design suggestion:

Two functions share the rules for checking whether an application tool owns a name. One checks the first page. The other checks later pages and assumes that one page has already used part of the limit.

Please consider putting these rules in one function. Give it the original request and the first page, if available.

That function can check all pages, preserve metadata, and enforce the page limit. Callers would no longer need to manage these details.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken, in 9e65f15. Both defects above lived in the duplicated rules, so merging was the smaller change overall.

isToolAdvertised and the inline first-page scan are now one function:

async function findRealToolOwner(
  server: MCPServerLike,
  toolName: string,
  request: MCPRequestLike,
  extra: CompatibleRequestHandlerExtra | undefined,
  logger: LoggerFn,
  servedPage?: CompatibleToolsListLike
): Promise<boolean | undefined>

servedPage is the page the caller already holds — first-page injection passes the response it is about to serve. Its tools are scanned in place, the walk starts at its own nextCursor, and it still spends one unit of the page budget, so the listing walk and the call-time walk reach the same distance. The tools/call path omits it and walks from the start. Callers no longer manage cursors, metadata or the budget.

One change beyond your three comments, flagged so it is visible rather than buried: the walk now has a 5-second budget beside its 25-page one. The page cap bounds how many reads it makes, not what they cost, and the README already notes that the first page does not reach the client until the walk ends. Expiry takes the fail-safe path that already existed, so there is no new branch.

Verification: 896 unit tests, both integration lanes green, lint and format clean. The seven new unit tests and three new probe fixtures all fail against the previous commit on both majors.

@gesh gesh added the stamphog label Sep 15, 2026

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved yet — waiting on the conditions below.

Re-add the stamphog label to request another review once you have addressed this.

Deterministic gates denied this PR: it changed a package.json test script list (flagged by the toolchain deny-list), and it exceeds the size/complexity ceiling for auto-review (packages/mcp/src/extensions/instrumentation.ts is a substantial, security-relevant rewrite of tool-call interception logic). Gate denials require a human decision regardless of the review activity already on the thread.

  • 👍 on the PR from greptile-apps[bot].
  • Gates denied: package.json test:integration scripts were edited (adding a new probe invocation) — not a malicious hook, but it trips the toolchain deny-list and needs a human sign-off.
  • Gates denied: change exceeds the size/complexity ceiling for auto-review (993 substantive lines, T2-never tier, two-areas scope) covering core MCP tool-listing/interception logic.
  • The change is otherwise well-covered (QA Swarm rounds, Greptile, Veria security bot, and a human APPROVE from @lucasheriques), but that assurance doesn't override the hard gate denial.
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✗ matches: deps_toolchain (scripts/hooks changed in package.json)
size ✗ too large for auto-review (993L substantive in global — ceiling is 800L; 993L, 9F total, 1611L/12F incl. docs/generated/snapshots)
tier ✗ classified as T2-never: T2-never (1611L, 12F, two-areas, fix)
stamphog 2.0.0b4 .stamphog/policy.yml @ 9e65f15 · reviewed head 9e65f15

@stamphog stamphog Bot removed the stamphog label Sep 15, 2026
The SDK appended the virtual send_feedback tool to every tools/list
page, so a cursor-following client saw it once per page. Now the tool
is appended only to the first page (the request with no cursor; an
empty-string cursor is a continuation page), which every client reads.

Name conflicts are the MCP owner's responsibility. Detection is cheap
and page-local: a real first-page tool with the name wins (the SDK
warns, skips injection, and forwards its calls through the existing
first-page probe), and a real tool on a later page is shadowed, with a
warning logged when a client fetches that page. Both warnings point at
the collectFeedback.toolName rename option.

Generated-By: PostHog Desktop
Task-Id: 3b751ad5-c551-4ad2-b90b-e50438622b88
@gesh
gesh force-pushed the posthog/mcp-feedback-pagination branch from f4d036b to 3a28019 Compare September 15, 2026 10:46

@gesh gesh left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA Swarm review complete. See inline comments.

Comment thread packages/mcp/src/extensions/instrumentation.ts Outdated
Flatten the nested first-page/collision conditionals into one if/else-if
chain keyed on an isFirstPage constant. Behavior unchanged.

Generated-By: PostHog Desktop
Task-Id: 3b751ad5-c551-4ad2-b90b-e50438622b88
Extract the feedbackOnFirstPageOnly predicate shared by three probe
assertions, and give setupPaginatedServer one options object instead of
two positional parameters. Behavior unchanged.

Generated-By: PostHog Desktop
Task-Id: 3b751ad5-c551-4ad2-b90b-e50438622b88
@gesh gesh added the stamphog label Sep 15, 2026

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved yet — waiting on the conditions below.

Re-add the stamphog label to request another review once you have addressed this.

Gates denied: the PR modifies package.json test-script entries (adding a new integration probe invocation) and exceeds the automatic size/complexity ceiling across two areas, so it can't be auto-approved even though the manifest change itself is benign (no lifecycle hooks, no lockfile-affecting deps). The change has substantial independent review (QA Swarm rounds, Greptile, Veria security bot all landed with fixes applied and no open concerns), but policy requires a human to sign off once gates deny on tier/size grounds.

  • 👍 on the PR from greptile-apps[bot].
  • Gate denial: tier classified T2-never due to size (660 lines, 7 files, two areas) and a deny-listed package.json scripts change (test:integration:sdk-v1/v2 entries), requiring human sign-off despite no lifecycle-hook or dependency additions.
  • Extensive review history (QA Swarm, Greptile, Veria) shows all raised issues fixed, but no human reviewer has approved the current head.
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✗ matches: deps_toolchain (scripts/hooks changed in package.json)
size ✓ 484L, 4F substantive, 660L/7F incl. docs/generated/snapshots — within ceiling
tier ✗ classified as T2-never: T2-never (660L, 7F, two-areas, fix)
stamphog 2.0.0b4 .stamphog/policy.yml @ 1194ac3 · reviewed head 1194ac3

@lucasheriques lucasheriques left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving at 1194ac3: traced listing-time vs call-time rules across page-1 owner, later-page owner, per-request factory, single page, and empty-string cursor; unit suite green on the stacked head (885 tests).

@gesh
gesh merged commit 2a6ddb9 into main Sep 15, 2026
63 checks passed
@gesh
gesh deleted the posthog/mcp-feedback-pagination branch September 15, 2026 16:00
gesh added a commit to PostHog/posthog-python that referenced this pull request Sep 17, 2026
* fix(mcp): inject virtual tools once, on the first tools/list page

The SDK advertises two virtual tools into tools/list: get_more_tools
(report_missing) and send_feedback (collect_feedback). A client
concatenates every page into one list, so each may appear on exactly
one page.

get_more_tools had no page gate at all, so a cursor-following client
saw it once per page. send_feedback went on the last page only, hiding
it from every client that never follows nextCursor. Both now go on the
first page -- the page every client reads -- matching @posthog/mcp.
An empty-string cursor is a valid opaque cursor, so it reads as a
continuation page rather than the first one.

get_more_tools also had no collision handling anywhere: no warning, no
shadow state, and unconditional interception on all four adapter
paths, so a host tool named get_more_tools was silently swallowed and
the agent got PostHog's canned reply. Both tools now share one
kind-keyed resolver, so this cannot drift again:

- A real tool owning the name on the first page wins: warn, skip
  injection, dispatch its calls normally.
- A real tool appearing only on a later page is shadowed, since page
  one cannot see page two. Warn when that page is served. @posthog/mcp
  behaves the same way.
- Ownership is also settled at call time, which covers a call reaching
  a process that never served a listing -- the ordinary multi-pod case.
  FastMCP and v2 MCPServer are asked via their tool registry; raw
  low-level servers have none, so the SDK asks the host's own
  tools/list handler, and only on a name match.
- Configuring both virtual tools with one name is detected and warned
  about; missing-capability wins, as every call path already assumed.

Warnings name the option that renames PostHog's tool and go to the
posthog.mcp stdlib logger as well as the logger option, so a
default-configured host actually sees them.

Two places matched the missing-capability name literally rather than
as configured: a renamed virtual tool picked up a conversation_id
argument the default-named one never got, and a real tool named
get_more_tools lost its context injection and its $mcp_intent.
mutate_tool_schema now takes is_sdk_virtual_tool from its caller
instead of guessing from the name.

PostHogMCP.prepare_tool_call honours original_tool for the
missing-capability tool as it already did for feedback.

Ports PostHog/posthog-js#4953 and PostHog/posthog-js#4967.

Generated-By: PostHog Desktop
Task-Id: 989fd424-fd24-4ae4-8682-548b87f761f6

* fix(mcp): never read an injected virtual tool back as a real one

Review found two ways the SDK mistook its own virtual tool for a
host tool.

A host may return the SAME tools/list result object from every
request -- a module-level constant, or its own cache. The appends
mutated that object in place, so the next read of it showed PostHog's
virtual tool sitting in what looked like the host's catalogue. The SDK
reported a collision against itself, stopped intercepting, and handed
the agent "Unknown tool: send_feedback" instead of recording its
feedback. Appends now build a copy and the adapters serve that, so
every read of a listing is a faithful view of what the host wrote.
Reported by @marandaneto, with a reproduction.

On the PostHogMCP path, _is_virtual_tool_name matched on the name
alone, in both directions: a host's own tool called get_more_tools was
skipped for context injection and silently lost its $mcp_intent, while
a host re-preparing an already-prepared list had PostHog's descriptor
treated as a host collision -- warning about ourselves. Both now
compare against the descriptor the SDK would build for that name; the
description is ours and, unlike the schema, survives the
model-injection pass. Flagged by Greptile.

Also stop re-invoking the host's tools/list handler from the call path
once a listing has been observed: its collision state already carries
the answer. The probe now runs only while this process has served no
listing -- the multi-pod case it exists for -- so a stateful or
expensive handler is not re-entered on every virtual-tool call.
Flagged by Greptile.

Generated-By: PostHog Desktop
Task-Id: 989fd424-fd24-4ae4-8682-548b87f761f6

* chore(mcp): trim the changeset to what users need to know

Generated-By: PostHog Desktop
Task-Id: 989fd424-fd24-4ae4-8682-548b87f761f6

* chore(mcp): name which renamed tool got a stray conversation_id

Only the missing-capability name was matched literally; the feedback
name already resolved correctly, so a renamed send_feedback was never
affected.

Generated-By: PostHog Desktop
Task-Id: 989fd424-fd24-4ae4-8682-548b87f761f6

* fix(mcp): drop the dead observed_listing flag and pin the probe cost

`observed_listing` was declared and read but never assigned, so the
guard never activated. My reply on #962 claimed otherwise -- the claim
was wrong, not the code.

Keep the per-call behaviour, which is the correct one, and fix the
text instead. `virtual_tool_collisions` is per-server state rewritten
by whichever listing ran last, so a raw server serving different
catalogues to different callers would answer one caller from
another's listing. Asking per call cannot go stale that way, and
`@posthog/mcp` probes per call for the same reason. Gating on a
first-listing flag would also have falsified the README sentence that
the call-time checks are the reliable signal for per-caller tool sets.

test_listing_stops_the_call_path_reprobing_the_host could not fail:
the probe closes over the original handler at wrap time, so replacing
the `request_handlers` entry observed nothing. Replaced with a test
that counts inside the host's own handler and pins the actual cost --
one probe per call to a virtual-tool name, none for ordinary tool
traffic. Writing it turned up a fourth invocation that is the MCP
SDK's own `req is None` cache repopulation on a call to an unlisted
name, so the test excludes that rather than miscounting it as ours.

README now states the real cost and offers renaming as the way to
avoid the check.

Reported by veria-ai and QA Swarm on #962.

Generated-By: PostHog Desktop
Task-Id: 989fd424-fd24-4ae4-8682-548b87f761f6

* chore(mcp): claim only the context injection, not the intent

A real tool of yours named `get_more_tools` now gets `context` injected
unconditionally, but whether its value is captured as `$mcp_intent`
still depends on this instance having served a tools/list. That half
belongs with the attribution work in the follow-up PR, so don't claim
it here.

Generated-By: PostHog Desktop
Task-Id: 989fd424-fd24-4ae4-8682-548b87f761f6

* fix(mcp): delegate the call when ownership cannot be determined

The call-time ownership check has three outcomes, not two: the host
owns the name, the host does not, or the question could not be asked --
a raw low-level server whose own tools/list handler is failing. The
third collapsed into "does not own", so the SDK intercepted.

That is the wrong direction. Guessing toward interception breaks the
host's tool, silently, for as long as their handler stays unwell, to
protect an analytics affordance. Guessing toward delegation costs one
visible failed call to a tool of PostHog's, which nothing depends on.
An analytics SDK does not get to break the product it measures, which
is the rule the rest of this package already follows.

`raw_listing_owns_tool_name` now returns Optional[bool], and the four
call sites intercept only on a definite False. Registry lookups are
unchanged: "not found" there is an answer, not a failure. The host gets
a warning naming the tool and the reason.

Also removes a divergence from @posthog/mcp, which gates on
`isToolAdvertised(...) === false` and delegates for the same reason. My
earlier reply on this PR claimed we already matched it; we did not.

Generated-By: PostHog Desktop
Task-Id: 989fd424-fd24-4ae4-8682-548b87f761f6

* docs(mcp): trim the changeset and README to what a host needs

The changeset listed five bullets, two of them restating internals.
Cut to the three things a host upgrading would notice, plus the
listed_tool_names shift for anyone charting it.

The README section repeated the warning texts the code already emits
and the option snippet twice. Cut to the rules and their limits.

Also one real inconsistency in the code: a page's tool names were
gathered two ways, once through a helper and once inline, and the
helper's name shadowed the unrelated $mcp_listed_tool_names event
field. Now one helper, named advertised_tool_names.

Generated-By: PostHog Desktop
Task-Id: 989fd424-fd24-4ae4-8682-548b87f761f6

* fix(mcp): stop the ownership probe corrupting the tool cache

The call-time ownership probe added in this branch runs the host's own
tools/list handler. When that handler is registered through the SDK's
`@server.list_tools()` decorator, mcp 1.x rebuilds `Server._tool_cache`
from the tools it returns on every invocation - and the probe, unlike the
wrapper's `req is None` branch, never re-ran schema injection afterwards.

That cache is what the SDK validates real tool arguments against, so one
ordinary `get_more_tools` call left every later call to a real tool
rejected with "Additional properties are not allowed ('context' was
unexpected)" for sending the argument we advertised, until the next
client-facing listing healed it. An analytics SDK breaking the host's
tools is the failure this branch exists to prevent.

The probe also closed over the handler captured at instrument() time, so
a host registering tools/list afterwards had ownership answered from a
catalogue no client ever sees - a confident wrong answer that swallowed
their real tool. `@posthog/mcp` re-captures the handler for this reason;
read the current one instead and report the question as unanswerable when
it is no longer ours, so the call is delegated.

Both paths are covered by tests that fail without the fix.

Generated-By: PostHog Desktop
Task-Id: f66265f3-22a8-4eb5-9bf1-bf174c1c0c65

* refactor(mcp): decide virtual-tool injection from the page, not stored state

`virtual_tool_collisions` existed to carry a listing's answer to the call
path. It was not needed for either job.

At listing time the page's own tools are already in hand, so the decision
is local: on a first page inject the name or warn that a real tool has
it, on a later page warn that ours already shadows theirs. Four branches,
no state - the shape `@posthog/mcp` uses.

At call time every site already gates on a stronger signal than the set
ever was: the tool registry on FastMCP and v2 MCPServer, the listing
probe on raw low-level servers. Those answer correctly in a process that
never served a listing, which is exactly where the set was empty and so
silently wrong - the multi-pod case it was meant to cover.

Names now resolve through `enabled_virtual_tool_names` everywhere, which
keeps the fix that matters: with `report_missing` off, a real tool called
`get_more_tools` is an ordinary tool and keeps its `context` captured as
intent.

Attribution for a real tool that *collides* with a virtual tool's name -
`$mcp_intent` and `$mcp_conversation_id` - is dropped here and belongs to
the stacked follow-up, which can read it off the ownership answer the
call path already computes rather than from state that goes stale.

Generated-By: PostHog Desktop
Task-Id: f66265f3-22a8-4eb5-9bf1-bf174c1c0c65

* fix(mcp): never answer "not yours" from a lookup that failed

Three fixes from review of the two commits before this one.

`_name_owned_by_real_tool`'s standalone-fastmcp branch caught every
exception from `get_tool` and answered False - "no real tool owns this
name" - on the assumption that an unknown name raises. In fastmcp 3.x it
does not: `get_tool` returns None for an unknown name and raises only
when the lookup itself fails, and its provider chain can reach a mounted
or proxied upstream over the network. So a connection blip was read as
"the name is free", and a call to a host tool that shares a virtual
tool's name was swallowed and answered with PostHog's canned reply and
isError=False - a fabricated success over a tool that never ran.

Until the commit before this one a listing-derived collision flag made
interception impossible for a name the host owned, which masked this.
Removing that flag left the lookup deciding alone. Catch fastmcp's
not-found and disabled errors as a real answer; delegate on anything
else, which both call sites already do for None.

The continuation-page "shadowed" warning no longer fires for a name a
first page already blocked. Nothing of ours is advertised in that case
and the host's tool runs, so telling them "the real tool will not run"
sent them chasing a bug that is not there.

The foreign-handler guard now covers a removed handler as well as a
replaced one, and says so once. The virtual tools stay advertised when
another layer wraps tools/list - a chained wrapper still runs our
injection - but nothing behind them is ever intercepted, and a silent
stop is invisible in the captured data. The README already promised this
was logged.

Also corrects a `raw_listing_owns_tool_name` docstring that described an
approach never taken (reading `Server._tool_cache`), which would have
invited removing the re-injection that repairs it, and sweeps five
comments describing the deleted collision state.

Generated-By: PostHog Desktop
Task-Id: f66265f3-22a8-4eb5-9bf1-bf174c1c0c65

* fix(mcp): don't warn that a tool we never injected is shadowing theirs

Three review fixes.

The continuation-page "shadowed" warning skipped a name a first page had
blocked, but not one the other virtual tool had won under the duplicate
name check. Configure both tools to one name and a host tool by that name
on a later page drew two warnings, one of them for a tool that was never
advertised. Skip any kind that did not make it onto the first page,
whichever way it lost.

The foreign-handler warning covers a removed handler as well as a
replaced one, so it says so, and both branches now have a test - the
removed one had none, which is why the wording went unnoticed.

Also corrects the comment on the ownership lookup's except branch. It
claimed a raised exception meant a mounted or proxied upstream had
blipped. It cannot: fastmcp gathers its providers with
`return_exceptions=True` and drops the failures, so an upstream blip
reads back as a plain None, indistinguishable from "no such tool", and
resolves to False without reaching that branch. What does reach it is the
visibility, transform and auth work layered on top of the providers. The
provider case is unchanged from before this SDK grew an ownership check -
on the missing-capability path it is strictly better, since that path
took no ownership check at all.

Generated-By: PostHog Desktop
Task-Id: f66265f3-22a8-4eb5-9bf1-bf174c1c0c65

* docs(mcp): record why a degraded provider cannot be told from an absent tool

A `FastMCP` is an `AggregateProvider`, which gathers its providers with
`return_exceptions=True` and drops the failures, so an unreachable mounted
or proxied sub-server reads back as a plain None from `get_tool` -
indistinguishable from "no such tool". If that sub-server owned a real
tool by a virtual tool's name, we treat the name as free and intercept.

Worth writing down because the obvious fix does not work: the same
failure is dropped from `list_tools` too, so a listing fallback returns
the same blind answer, and fastmcp 3.x exposes no error strategy to opt
out of. A later reader would otherwise rediscover this and add a probe
that cannot help.

Narrower than it first reads, which the note also records: during the
outage the host's tool is missing from tools/list as well, so it could
not have been dispatched either way. Only a provider that recovers
between the check and dispatch loses a call that would have worked.

Behaviour unchanged.

Generated-By: PostHog Desktop
Task-Id: f66265f3-22a8-4eb5-9bf1-bf174c1c0c65

* refactor(mcp): collapse the virtual-tool appenders and trim the prose

The kind-keyed resolver put per-tool policy in one place, but the layer
below it stayed four copies. `append_get_more_tools`, `append_send_feedback`
and their two v2 twins differed only by descriptor builder and
`inputSchema` vs `input_schema`, and the block that called them was
repeated verbatim in all three adapters.

- One `append_virtual_tool_by_kind` behind one `virtual_tool_descriptor`
  switch, applied through one `apply_virtual_tool_injection`. The
  missing-capability-then-feedback order the `duplicate` rule depends on is
  now stated once instead of implied in three places.
- Both raw probes use `advertised_tool_names` rather than a third copy of
  the same comprehension.
- `VirtualToolInjection` wrapped a dict and two `.get`s that nothing reads
  any more, so the resolver returns the dict.
- Dead branches out: the `missing_name or name` fallback the
  `is_missing_capability` property already rules out, the unreachable
  `options is None` guard in the feedback appenders, and an `event` looked
  up before a branch that never used it.

One behaviour change. `_name_owned_by_real_tool` on the low-level adapter
was tri-state — a lookup that raises means "could not answer", and the call
is delegated — but the FastMCP and v2 registry paths still read every
failure as "the name is free" and swallowed the host's own tool. All three
now share the contract, with a test per adapter that makes the registry
raise and asserts the real tool runs. Both fail without the change.

Prose: the PR was adding roughly 0.7 lines of comment per line of code
against ~0.2 in the same files. Cut the narrative, the `@posthog/mcp`
commentary and the sentences that had reached three copies; kept the
`_tool_cache` re-injection reason, the late-bound handler lookup, the
empty-string cursor rule, the JS-parity warning and the provider note,
which was stated twice and is now stated once.

Tests: `_make_paged_lowlevel`, `_list_page`, `_call_request` and
`_ECHO_TOOL` were defined in both virtual-tool test files; they move to
`_helpers_lowlevel`, kept out of `_helpers` because that one loads under
both SDK majors. Dropped one resolver test subsumed by its neighbours.

515 passed on mcp 1.x, 440 passed / 19 skipped on 2.x, mypy and the public
API snapshot clean.

Generated-By: PostHog Desktop
Task-Id: b86d25ed-d899-4e29-a52f-cd0ab29ae38d

* docs(mcp): cut the changeset to the page rule and the warnings

Four bullets down to two sentences, matching the one-line house style of
the changesets around it.

Drops the claim that a real tool of yours sharing a virtual tool's name
"now wins". That holds on the first page; on a later page this PR
deliberately inverts it, so as a release note it would tell a host in that
shape their tool is safe when it is not. The README and the PR body carry
the full rule. What a reader needs from a changelog is that the names can
collide and that PostHog says so, which the warning sentence gives them.

Also drops the reused-result-object fix and the listed_tool_names note:
both are repairs to behaviour this release introduces, invisible to anyone
upgrading from the last one.

Generated-By: PostHog Desktop
Task-Id: b86d25ed-d899-4e29-a52f-cd0ab29ae38d

* fix(mcp): keep one precedence when both virtual tools share a name

`instrument()` already handled a host that configures
`missing_capability_tool_name` and `collect_feedback.tool_name` to the
same string: it drops the feedback tool, keeps missing-capability
precedence, and warns "duplicate". The `PostHogMCP` dispatcher did
neither.

`prepare_tool_list` fell through to its "name already taken" branch, so
the feedback descriptor went unadvertised with no warning.
`prepare_tool_call` then set `is_feedback` and `is_missing_capability`
both true, and the README's dispatch snippet tests `is_feedback` first,
so every missing-capability call became a bogus $mcp_feedback capture --
the inverse of the precedence the other path sets on purpose.

Also folds four copies of the ownership-probe warning into one helper,
and names the three collision variants, so a typo is a type error rather
than silently picking the "blocked" wording.

Generated-By: PostHog Desktop
Task-Id: 7047b34a-b562-40ab-ae65-8b0f24700e57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants