Skip to content

Commit 2b4e106

Browse files
fix(host-mcp): evict idle MCP sessions instead of leaking them (#1685)
* fix(host-mcp): evict idle MCP sessions instead of leaking them The in-process session store keyed transports, servers, owners, and engines by session id and only ever deleted an entry on `onsessionclosed`, which the SDK fires on `DELETE /mcp`. Nothing sends that DELETE: the client SDK's `transport.close()` aborts locally and puts nothing on the wire, a crashed client cannot send it, and `enableJsonResponse` leaves no stream whose teardown could stand in for it. Every initialize therefore pinned an McpServer, its tool registry, and an ExecutionEngine until the process exited. Measured against ghcr.io/usefulsoftwareco/executor-selfhost:1.5.42, 500 sessions opened without a DELETE grow RSS by 346 MiB (709 KiB each, linear, no plateau); the same 500 with a DELETE grow it by 13 MiB. Stamp each session on create and on every forwarded request, then sweep on a timer and dispose anything idle past the TTL. Eviction is what the streamable HTTP spec allows a server to do, and the store already renders an unknown id as the existing "not-found" (404 -32001), which is a client's cue to re-initialize. * feat(host-selfhost): expose EXECUTOR_MCP_SESSION_IDLE_TTL_MS The store's idle window is only useful if an operator can tune it: a client that cannot tolerate re-initializing needs a longer TTL, and diagnosing one needs eviction off entirely (0). Parse it the same way as EXECUTOR_SANDBOX_TIMEOUT_MS, refusing to boot on a malformed value. * Thread self-host config into the MCP seams and correct the eviction rationale makeSelfHostMcpSeams called loadConfig() inside the factory, hiding an env read behind construction and moving a boot-time throw (loadConfig refuses a malformed EXECUTOR_MCP_SESSION_IDLE_TTL_MS) off the boot path. The app already has the resolved config, so pass it in. The store comment claimed enableJsonResponse means no long-lived stream exists. It only governs how a POST carrying requests answers; the bare 202 for notifications/initialized still cues the client to open the GET stream, so nearly every session holds one. Eviction ignores it on purpose, which is what cloud already does past its running-lease ceiling. * Add a self-host e2e scenario for idle MCP session eviction * Hold an MCP session while a request is in flight, and shut its engine down The idle sweep read only a last-seen stamp, written before the store awaits transport.handleRequest. A call slower than the idle window was therefore indistinguishable from an abandoned session, and the sweep closed the transport, the server, and the engine underneath the request still using them. Count the requests inside handleRequest per session, skip a session with any, and restamp when a call ends so idleness measures from completion. Disposal deleted the engine reference without running engine.shutdown, so the detached sandbox fibers a paused execution holds kept running - and kept querying the host database handle - after the session was gone. Every disposal path now shuts the engine down: sweep eviction, the dispose seam, store close, onsessionclosed, and the eager close of a transport that never minted an id. A close failure was swallowed whole, which made a leaked handle invisible. Keep it best-effort, but log it at warning with the session id and the handle. Wire the store close hook into the self-host server. startServer discarded closeDb, so a graceful shutdown left every live session and the libSQL handle open; only the test web-handler path ever released them. --------- Co-authored-by: Rhys Sullivan <39114868+RhysSullivan@users.noreply.github.com>
1 parent 1908dd6 commit 2b4e106

10 files changed

Lines changed: 656 additions & 20 deletions

File tree

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
"@executor-js/host-mcp": patch
3+
---
4+
5+
Evict idle MCP sessions instead of holding them for the lifetime of the process. The in-process session store only released a session when the client sent `DELETE /mcp`, which the MCP client SDK's `transport.close()` never sends and a crashed client cannot send, so every `initialize` permanently retained an `McpServer`, its tool registry, and an `ExecutionEngine`. Sessions are now stamped on create and on each request, and a timer disposes anything idle past `sessionIdleTtlMs` (30 minutes by default). An open server-to-client stream does not defer eviction, matching how cloud's session alarm destroys a session once it passes its running-lease ceiling; an evicted id answers 404 `-32001`, which is the client's cue to re-initialize.
6+
7+
A request in flight holds its session: idleness counts from when a call ends, not from when it started, so a tool call slower than the idle window is never cut off mid-flight. Disposal also shuts the session's execution engine down rather than only dropping the reference, which is what ends its detached sandbox fibers, and a handle that fails to close is now logged with its session id instead of being discarded silently.

apps/host-selfhost/src/app.ts

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -76,9 +76,7 @@ export const makeSelfHostApp = async (options: MakeSelfHostAppOptions = {}) => {
7676
const { identityLayer, authHandler, betterAuth } = await resolveAuthProviders(dbHandle);
7777

7878
// ---- the in-process MCP serving seams (+ shutdown hook) ----------------
79-
// Pass the pinned public origin so browser-approval URLs are reachable behind
80-
// a reverse proxy (not the internal 127.0.0.1 bind from the request URL).
81-
const mcp = makeSelfHostMcpSeams(dbHandle, betterAuth, config.webBaseUrl);
79+
const mcp = makeSelfHostMcpSeams(dbHandle, betterAuth, config);
8280

8381
// CLI device-login discovery (`executor login`). Points the CLI at Better
8482
// Auth's device endpoints; `requestFormat: "json"` because those endpoints

apps/host-selfhost/src/config.ts

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,11 @@ export interface SelfHostConfig {
5151
* minutes (the same pattern as MCP_PAUSED_SESSION_IDLE_TIMEOUT_MS on cloud).
5252
*/
5353
readonly sandboxTimeoutMs: number | undefined;
54+
/**
55+
* How long an MCP session may sit idle before the in-process store evicts it,
56+
* or undefined for the store's own default (30 minutes). 0 disables eviction.
57+
*/
58+
readonly mcpSessionIdleTtlMs: number | undefined;
5459
/**
5560
* How long a connection's persisted remote tool catalog stays fresh, in ms.
5661
* `undefined` takes the SDK default (15 minutes); `null` disables time-based
@@ -163,6 +168,7 @@ export const loadConfig = (): SelfHostConfig => {
163168
organizationName: process.env.EXECUTOR_ORG_NAME ?? "Default",
164169
orgSlug: resolveOrgSlug(),
165170
sandboxTimeoutMs: resolveSandboxTimeoutMs(),
171+
mcpSessionIdleTtlMs: resolveMcpSessionIdleTtlMs(),
166172
toolsSyncTtlMs: resolveToolsSyncTtlMs(),
167173
};
168174
};
@@ -183,6 +189,23 @@ const resolveSandboxTimeoutMs = (): number | undefined => {
183189
return Math.floor(parsed);
184190
};
185191

192+
// How long an MCP session may sit idle before the store evicts it. 0 disables
193+
// eviction, which restores the old behaviour of holding every session for the
194+
// lifetime of the process — only useful for diagnosing a client that cannot
195+
// tolerate re-initializing.
196+
const resolveMcpSessionIdleTtlMs = (): number | undefined => {
197+
const raw = process.env.EXECUTOR_MCP_SESSION_IDLE_TTL_MS;
198+
if (!raw) return undefined;
199+
const parsed = Number(raw);
200+
if (!Number.isFinite(parsed) || parsed < 0) {
201+
// oxlint-disable-next-line executor/no-try-catch-or-throw, executor/no-error-constructor -- boundary: refuse to boot on a malformed operator knob
202+
throw new Error(
203+
`EXECUTOR_MCP_SESSION_IDLE_TTL_MS ${JSON.stringify(raw)} is not a non-negative number of milliseconds`,
204+
);
205+
}
206+
return Math.floor(parsed);
207+
};
208+
186209
// The org slug doubles as a URL segment (`/<slug>/policies`), so an
187210
// operator-set value must fit the shared grammar and avoid reserved root
188211
// segments (api, mcp, login, …) — a colliding slug would shadow real routes.

apps/host-selfhost/src/mcp/index.ts

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import type {
1010

1111
import { BetterAuth, type BetterAuthHandle } from "../auth/better-auth";
1212
import type { SelfHostDbHandle } from "../db/self-host-db";
13+
import type { SelfHostConfig } from "../config";
1314
import { selfHostMcpAuth } from "./auth";
1415
import {
1516
makeSelfHostMcpSessionStore,
@@ -128,13 +129,24 @@ const makeApprovalHandler =
128129
* instance provided; it still requires `IdentityProvider` from the resolved
129130
* identity seam. Returns the three seam Layers plus the `close()` lifetime hook
130131
* the app wires into shutdown.
132+
*
133+
* Takes the already-resolved `SelfHostConfig` rather than reading it here: the
134+
* app loads it once at boot, and `loadConfig()` refuses to boot on a malformed
135+
* operator knob, so calling it from a seam factory would both hide an env read
136+
* behind construction and move that failure off the boot path.
131137
*/
132138
export const makeSelfHostMcpSeams = (
133139
dbHandle: SelfHostDbHandle,
134140
betterAuth: BetterAuthHandle,
135-
webBaseUrl?: string,
141+
config: SelfHostConfig,
136142
): SelfHostMcpSeams => {
137-
const sessionStore = makeSelfHostMcpSessionStore(dbHandle, webBaseUrl);
143+
// The pinned public origin keeps browser-approval URLs reachable behind a
144+
// reverse proxy (not the internal 127.0.0.1 bind from the request URL).
145+
const sessionStore = makeSelfHostMcpSessionStore(
146+
dbHandle,
147+
config.webBaseUrl,
148+
config.mcpSessionIdleTtlMs,
149+
);
138150
const auth: Layer.Layer<McpAuthProvider, never, IdentityProvider> = selfHostMcpAuth.pipe(
139151
Layer.provide(Layer.succeed(BetterAuth)(betterAuth)),
140152
);

apps/host-selfhost/src/mcp/session-store.ts

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ export { McpEngineBuildError } from "@executor-js/host-mcp/in-memory-session-sto
3636
export const makeSelfHostMcpSessionStore = (
3737
db: SelfHostDbHandle,
3838
webBaseUrl?: string,
39+
sessionIdleTtlMs?: number,
3940
): InMemoryMcpSessionStore =>
4041
makeInMemoryMcpSessionStore(
4142
makeMcpBuildServer(
@@ -48,7 +49,10 @@ export const makeSelfHostMcpSessionStore = (
4849
selfHostAnalytics.record(`artifact_${action}`, { via: "agent" }),
4950
},
5051
),
51-
{ webBaseUrl },
52+
{
53+
...(webBaseUrl === undefined ? {} : { webBaseUrl }),
54+
...(sessionIdleTtlMs === undefined ? {} : { sessionIdleTtlMs }),
55+
},
5256
);
5357

5458
/** The `McpSessionStore` envelope seam over a freshly built in-process store. */

apps/host-selfhost/src/serve.ts

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ const selfHostHttpMiddleware = (betterAuth: BetterAuthHandle) =>
109109

110110
export const startServer = async (): Promise<void> => {
111111
const config = loadConfig();
112-
const { AppLayer, betterAuth } = await makeSelfHostApp();
112+
const { AppLayer, betterAuth, closeDb } = await makeSelfHostApp();
113113

114114
// Serve the built SPA, split by cacheability so a redeploy is picked up at
115115
// once instead of stranding browsers on a stale shell:
@@ -141,6 +141,16 @@ export const startServer = async (): Promise<void> => {
141141
Effect.addFinalizer(() => Effect.promise(() => disposeAnalytics())),
142142
);
143143

144+
// Server-scope finalizer: release what the app opened at boot — every live
145+
// MCP session (transport, server, and its execution engine, whose detached
146+
// sandbox fibers keep querying the DB until the engine is shut down) and then
147+
// the shared libSQL handle itself. `makeSelfHostApiHandler` runs this in its
148+
// `dispose`, so tests have always released it; the long-lived server dropped
149+
// the hook on the floor and leaked both across a graceful shutdown.
150+
const AppResourcesLive = Layer.effectDiscard(
151+
Effect.addFinalizer(() => Effect.promise(() => closeDb())),
152+
);
153+
144154
// OTLP export, or `Layer.empty` when no collector is configured (see
145155
// ./telemetry). The `http.server` envelope span each request's `withSpan`
146156
// children parent under is NOT wired here: `HttpEffect.toHandled` already
@@ -162,7 +172,11 @@ export const startServer = async (): Promise<void> => {
162172
// in scope while the app's layers build, or spans created during construction
163173
// resolve the default no-op tracer and are silently dropped.
164174
await BunRuntime.runMain(
165-
Layer.launch(Layer.merge(ServerLive, AnalyticsFlushLive).pipe(Layer.provide(TelemetryLive))),
175+
Layer.launch(
176+
Layer.mergeAll(ServerLive, AnalyticsFlushLive, AppResourcesLive).pipe(
177+
Layer.provide(TelemetryLive),
178+
),
179+
),
166180
);
167181
};
168182

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
// Selfhost-only: an MCP session that goes idle past the store's TTL is
2+
// reclaimed, and the next request on that session id gets the 404 / -32001 cue
3+
// that tells a client to re-initialize.
4+
//
5+
// Why this scenario boots its OWN instance instead of using the shared one:
6+
// the idle window is a BOOT-TIME operator knob
7+
// (EXECUTOR_MCP_SESSION_IDLE_TTL_MS), so there is no way to shrink it on a
8+
// running server. Waiting out the 30-minute default is not an option, and
9+
// shrinking it on the SHARED instance would silently evict sessions underneath
10+
// every other selfhost scenario. A dedicated instance on its own port and data
11+
// dir keeps the short window contained to this file.
12+
//
13+
// Why the wait is a single silent sleep rather than a poll loop: every request
14+
// the store forwards restamps that session's last-seen time. A poll loop is
15+
// itself the traffic that keeps the session alive, so it could never observe an
16+
// eviction. The scenario stays completely silent for one window, then makes
17+
// exactly one request.
18+
//
19+
// Auth is the Better Auth session cookie, not an OAuth bearer: self-host's MCP
20+
// auth provider accepts the cookie/api-key identity path, and the credential is
21+
// not what is under test here — session lifetime is.
22+
import { mkdtempSync, rmSync } from "node:fs";
23+
import { tmpdir } from "node:os";
24+
import { join } from "node:path";
25+
26+
import { expect } from "@effect/vitest";
27+
import { Effect } from "effect";
28+
29+
import { scenario } from "../src/scenario";
30+
import { RunDir, Target } from "../src/services";
31+
import { claimAndBoot } from "../src/ports";
32+
import { isBootReadinessTimeout } from "../setup/boot";
33+
import { bootSelfhost } from "../setup/selfhost.boot";
34+
import { SELFHOST_ADMIN, signInSession } from "../targets/selfhost";
35+
36+
/** The idle window this instance runs with — small enough that the sweep, not
37+
* the TTL, sets the pace. */
38+
const IDLE_TTL_MS = 2_000;
39+
40+
/**
41+
* How long to leave the session untouched. The store floors its sweep interval
42+
* at 30s, so an idle session is disposed at the first tick that falls at least
43+
* one TTL after its last request. Waiting two full sweep intervals plus the TTL
44+
* means the assertion can never race a tick that has not fired yet.
45+
*/
46+
const QUIET_WINDOW_MS = 2 * 30_000 + IDLE_TTL_MS + 5_000;
47+
48+
interface JsonRpcErrorBody {
49+
readonly error?: { readonly code?: number; readonly message?: string };
50+
}
51+
52+
scenario(
53+
"MCP · an idle self-host session is evicted and answers 404 -32001 until the client re-initializes",
54+
// Own vite dev boot (cold on a fresh checkout) plus a 67s quiet window, so
55+
// this needs materially more than the project's 180s default.
56+
{ timeout: 420_000 },
57+
Effect.gen(function* () {
58+
// Selfhost-shaped scenario: yielded for the target name in failures, and so
59+
// the file reads like its neighbours.
60+
yield* Target;
61+
const runDir = yield* RunDir;
62+
63+
const dataDir = mkdtempSync(join(tmpdir(), "executor-selfhost-idle-ttl-"));
64+
65+
// A distinct env var (not E2E_SELFHOST_PORT, which the shared instance has
66+
// already published into this worker's env) so the claim actually probes
67+
// and locks a free port instead of returning the shared one.
68+
const booted = yield* Effect.promise(() =>
69+
claimAndBoot(
70+
[{ envVar: "E2E_SELFHOST_IDLE_TTL_PORT", offset: 6, label: "selfhost idle-ttl vite dev" }],
71+
async (ports) => {
72+
const port = ports.E2E_SELFHOST_IDLE_TTL_PORT!;
73+
const baseUrl = `http://localhost:${port}`;
74+
const procs = await bootSelfhost({
75+
port,
76+
webBaseUrl: baseUrl,
77+
admin: SELFHOST_ADMIN,
78+
dataDir,
79+
logFile: join(runDir, "idle-ttl-boot.log"),
80+
mcpSessionIdleTtlMs: IDLE_TTL_MS,
81+
});
82+
return { teardown: procs.teardown, value: baseUrl };
83+
},
84+
{ label: "selfhost idle-ttl", retryWhen: isBootReadinessTimeout },
85+
),
86+
);
87+
88+
yield* Effect.gen(function* () {
89+
const baseUrl = booted.value;
90+
const mcpUrl = new URL("/mcp", baseUrl).toString();
91+
const { cookieHeader } = yield* Effect.promise(() => signInSession(baseUrl, SELFHOST_ADMIN));
92+
93+
const initialize = async (): Promise<Response> =>
94+
fetch(mcpUrl, {
95+
method: "POST",
96+
headers: {
97+
cookie: cookieHeader,
98+
"content-type": "application/json",
99+
accept: "application/json, text/event-stream",
100+
},
101+
body: JSON.stringify({
102+
jsonrpc: "2.0",
103+
id: 1,
104+
method: "initialize",
105+
params: {
106+
protocolVersion: "2025-06-18",
107+
capabilities: {},
108+
clientInfo: { name: "idle-ttl-e2e", version: "1" },
109+
},
110+
}),
111+
});
112+
113+
const listTools = async (sessionId: string, id: number): Promise<Response> =>
114+
fetch(mcpUrl, {
115+
method: "POST",
116+
headers: {
117+
cookie: cookieHeader,
118+
"mcp-session-id": sessionId,
119+
"mcp-protocol-version": "2025-06-18",
120+
"content-type": "application/json",
121+
accept: "application/json, text/event-stream",
122+
},
123+
body: JSON.stringify({ jsonrpc: "2.0", id, method: "tools/list" }),
124+
});
125+
126+
// 1. A client initializes and the session serves.
127+
const opened = yield* Effect.promise(initialize);
128+
expect(opened.status, "initialize succeeds").toBe(200);
129+
const sessionId = opened.headers.get("mcp-session-id");
130+
expect(sessionId, "initialize returns a session id").toEqual(expect.any(String));
131+
132+
const working = yield* Effect.promise(() => listTools(sessionId!, 2));
133+
expect(working.status, "the fresh session serves a request").toBe(200);
134+
135+
// 2. Idleness, driven by the clock and nothing else. Touching the session
136+
// here — even to poll — would restamp it and defeat the measurement.
137+
yield* Effect.sleep(`${QUIET_WINDOW_MS} millis`);
138+
139+
// 3. The evicted id is gone, and says so in the shape a client acts on:
140+
// 404 tells it the id is dead, -32001 is the session-lifecycle code.
141+
const afterIdle = yield* Effect.promise(() => listTools(sessionId!, 3));
142+
const afterIdleBody = yield* Effect.promise(() => afterIdle.text());
143+
// A session that was NOT evicted answers with the whole tool catalog, so
144+
// the diagnostic is truncated — the status is the assertion, and a full
145+
// tools/list dump in the failure output helps nobody.
146+
expect(
147+
afterIdle.status,
148+
`an idle session is evicted, so its id 404s; body starts: ${afterIdleBody.slice(0, 200)}`,
149+
).toBe(404);
150+
// oxlint-disable-next-line executor/no-json-parse -- boundary: the raw JSON-RPC error frame this scenario asserts on, never decoded into a domain type
151+
const parsed = JSON.parse(afterIdleBody) as JsonRpcErrorBody;
152+
expect(parsed.error?.code, "the 404 carries the session-lifecycle code").toBe(-32001);
153+
154+
// 4. The cue is actionable: re-initializing gets a NEW, working session.
155+
const reopened = yield* Effect.promise(initialize);
156+
expect(reopened.status, "the client can re-initialize after eviction").toBe(200);
157+
const newSessionId = reopened.headers.get("mcp-session-id");
158+
expect(newSessionId, "re-initialize returns a session id").toEqual(expect.any(String));
159+
expect(newSessionId, "re-initialize issues a different session").not.toBe(sessionId);
160+
161+
const afterReinit = yield* Effect.promise(() => listTools(newSessionId!, 4));
162+
expect(afterReinit.status, "the re-initialized session serves a request").toBe(200);
163+
}).pipe(
164+
Effect.ensuring(
165+
Effect.promise(async () => {
166+
await booted.teardown();
167+
rmSync(dataDir, { recursive: true, force: true });
168+
}),
169+
),
170+
);
171+
}),
172+
);

e2e/setup/selfhost.boot.ts

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,10 @@ export interface SelfhostBootOptions {
2424
/** Shrink the sandbox execution budget (EXECUTOR_SANDBOX_TIMEOUT_MS) so
2525
* deadline scenarios prove their race in seconds. Omit for production. */
2626
readonly sandboxTimeoutMs?: number;
27+
/** Shrink the MCP session idle window (EXECUTOR_MCP_SESSION_IDLE_TTL_MS) so
28+
* the eviction scenario proves in seconds what otherwise takes 30 minutes.
29+
* Omit for production; the app then uses the store's own default. */
30+
readonly mcpSessionIdleTtlMs?: number;
2731
}
2832

2933
export const bootSelfhost = async (options: SelfhostBootOptions): Promise<BootedProcesses> => {
@@ -57,6 +61,9 @@ export const bootSelfhost = async (options: SelfhostBootOptions): Promise<Booted
5761
...(options.sandboxTimeoutMs !== undefined
5862
? { EXECUTOR_SANDBOX_TIMEOUT_MS: String(options.sandboxTimeoutMs) }
5963
: {}),
64+
...(options.mcpSessionIdleTtlMs !== undefined
65+
? { EXECUTOR_MCP_SESSION_IDLE_TTL_MS: String(options.mcpSessionIdleTtlMs) }
66+
: {}),
6067
},
6168
logFile: options.logFile,
6269
},

0 commit comments

Comments
 (0)