Repository navigation
fix(js-sdk): keep envd HTTP/2 sessions within Node's session memory budget - #2083
Closed
devin-ai-integration[bot] wants to merge 6 commits into
Closed
devin-ai-integration[bot] wants to merge 6 commits into
devin-ai-integration[bot] wants to merge 6 commits into
Conversation
Node charges queued request bodies against a 10 MB per-session budget and resets the stream whose response arrives while it is exhausted (NGHTTP2_ENHANCE_YOUR_CALM). Since undici 8 all envd REST requests of a process shared one session. envd REST dispatchers now carry one request per HTTP/2 session across the connection pool, and a dispatcher whose session Node reports as out of memory is retired and replaced. Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
🦋 Changeset detectedLatest commit: b2a5dbb The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Contributor
Package ArtifactsBuilt from 9a305a2. Download artifacts from this workflow run. JS SDK ( npm install ./e2b-dockerfile-utils-0.1.2-devin-1793722226-h2-session-memory-envd-writes.0.tgz ./e2b-2.62.2-devin-1793722226-h2-session-memory-envd-writes.0.tgzCLI ( npm install ./e2b-cli-2.23.1-devin-1793722226-h2-session-memory-envd-writes.0.tgzCode Interpreter JS SDK ( npm install ./e2b-code-interpreter-2.9.2-devin-1793722226-h2-session-memory-envd-writes.0.tgzDesktop JS SDK ( npm install ./e2b-desktop-2.5.1-devin-1793722226-h2-session-memory-envd-writes.0.tgzPython SDK ( pip install ./e2b_dockerfile_utils-0.1.1+devin.1793722226.h2.session.memory.envd.writes-py3-none-any.whl ./e2b-2.62.0+devin.1793722226.h2.session.memory.envd.writes-py3-none-any.whlCode Interpreter Python SDK ( pip install ./e2b_code_interpreter-2.11.7+devin.1793722226.h2.session.memory.envd.writes-py3-none-any.whlDesktop Python SDK ( pip install ./e2b_desktop-2.7.2+devin.1793722226.h2.session.memory.envd.writes-py3-none-any.whl |
…wser runner) Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
mishushakov
marked this pull request as ready for review
November 3, 2026 16:17
Member
|
check above comments |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…onError Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Member
|
@claude review |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
mishushakov
approved these changes
Nov 3, 2026
…TP/2 streams A 1-stream-per-session cap starves the origin: undici 8's Pool holds every request behind its first (protocol-probe) client until it drains, so one long-lived REST request blocked all others to that sandbox. Keep sessions multiplexed and instead admit buffered upload bodies so that together they stay within 6 MB of Node's 10 MB session memory budget. Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
A buffered upload larger than the 6 MiB budget was admitted alone but still shared the pooled HTTP/2 session, so any read whose response landed while it filled Node's session memory was reset (16 MB write + 10 reads: 30/30 reads failed, also on base). Such uploads now go through a single-use dispatcher that is closed after the response, like undici 7 gave them in 2.27.1. Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
…owser runner) Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
Member
|
proceeding with #2086 instead until the Node issue is fixed |
mishushakov
added a commit
that referenced
this pull request
Nov 3, 2026
…TP/2 dispatchers (#2086) ## Summary Fixes [SDK-425](https://linear.app/e2b/issue/SDK-425): concurrent `sandbox.files.write()` calls fail with `TypeError: fetch failed` → `NGHTTP2_ENHANCE_YOUR_CALM` since `e2b` ≥ 2.35 (undici 8). Simpler alternative to #2083 (byte budget + one-off sessions), same measurements. **Cause.** The RST is sent by Node's own http2 client, not by E2B infra. Node queues each buffered request body in full against the session's 10 MB `maxSessionMemory`; if a response HEADERS frame arrives on *any* stream while that budget is exhausted, Node resets that stream with `ENHANCE_YOUR_CALM`. undici 7 kept one request per session (so our 10-connection pool = 10 budgets); undici 8 multiplexes, so every envd REST request in the process shares one session and one budget — parallel or large uploads reset each other and the reads next to them. Streams destroyed before their body is flushed (timeouts, aborts, killed sandboxes) additionally leak their queued bytes on Node ≤ 24 (fixed on `main` by nodejs/node#67113), so a session degrades until every small write on it fails — the customer's "climbing since Monday". **Fix (JS SDK only — Python uses pyqwest/hyper, not Node http2):** 1. `Filesystem` POSTs `/files` (octet-stream, streamed, gzip and multipart uploads) through `createEnvdUploadFetch` — an HTTP/1.1 dispatcher pool of its own (`E2B_ENVD_UPLOAD_CONNECTIONS`, default 50, like `E2B_ENVD_RPC_CONNECTIONS`); everything else (`read`, `makeDir`, RPC, health…) stays on HTTP/2, and `httpVersion` / `E2B_HTTP_VERSION` keep governing those. The `ConnectionOpts.httpVersion` JSDoc documents the exception. ```ts // sandbox/filesystem/index.ts const uploadFetch = createEnvdUploadFetch({ proxy: connectionConfig.proxy }) … this.envdApi.api.POST('/files', { …, fetch: this.uploadFetch }) ``` One request per connection, no shared memory budget — what undici 7 effectively gave 2.27.1 (which capped uploads at 10 per process; this pool allows 50, beyond that uploads queue in undici with their request timeout running). 2. `buildDispatchedFetch` (`src/undici.ts`): a request that fails with `ERR_HTTP2_STREAM_ERROR` + `NGHTTP2_ENHANCE_YOUR_CALM`/`NGHTTP2_INTERNAL_ERROR` (`isPoisonedHTTP2SessionError`) retires that dispatcher and builds a fresh one for subsequent requests. The failed request is **not** replayed (envd may already have applied the write). Only the dispatcher that served the failing request can replace the current one. Safety net for whatever else might fill a session. No API change. **Measured** (built package, real sandboxes, Node 24.22; ALPN confirmed: writes `http/1.1`, reads `h2`): | scenario | 2.62.1 | this PR | |---|---|---| | 10 × 2 MB parallel writes, 5 rounds | 16/50 failed | 0/50 | | 30 × 1 MB, 20 × 1 MB, 4 × 16 MB, customer pattern (makeDir + 10 parallel small writes ×200) | — | 0 failed | | 16 MB write + 10 concurrent `files.read()` | 30/30 reads failed | 0/33 | | poisoned session (`maxSessionMemory=1`, aborted big writes) | 50/50 fail, forever | 0 failed | | streamed (async iterable), gzip, multipart, string, aborted upload | — | all OK | **Upstream:** backport request on nodejs/node#67113 (nodejs/node#67113 (comment)) and undici issue nodejs/undici#6149. Link to Devin session: https://app.devin.ai/sessions/98f09d080013403b954d3f7b07c51b3b Open in Devin Desktop: https://app.devin.ai/desktop/session/98f09d080013403b954d3f7b07c51b3b?variant=devin Requested by: @mishushakov --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mish@e2b.dev <mish@e2b.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes SDK-425: concurrent
sandbox.files.write()calls fail withTypeError: fetch failed→NGHTTP2_ENHANCE_YOUR_CALMsincee2b≥ 2.35 (undici 8).Cause. The RST is sent by Node's own http2 client, not by E2B infra. Node queues each buffered request body in full against the session's 10 MB
maxSessionMemory; if a response HEADERS frame arrives on any stream while that budget is exhausted, Node resets that stream withENHANCE_YOUR_CALM. undici 7 kept one request per session (so our 10-connection pool = 10 budgets); undici 8 multiplexes, so every envd REST request in the process shares one session and one budget. Streams destroyed before their body is flushed (timeouts, aborts, killed sandboxes) additionally leak their queued bytes on Node ≤ 24 (fixed onmainby nodejs/node#67113), so a session degrades over time until every small write on it fails — matching the customer's "climbing since Monday".Fix (JS SDK only — Python uses pyqwest/hyper, not Node http2):
src/envd/uploadBudget.ts:files.write()runs each buffered/filesupload throughenvdUploadBudget.run(size, signal, …), a process-wide semaphore over request-body bytes withENVD_MAX_INFLIGHT_BODY_BYTES = 6 MiB. Bodies are admitted FIFO while they fit together; streamed bodies and every non-upload call (read,makeDir, RPC, health…) cost 0 and are never held. Waiters honour the request'sAbortSignal.Filesystem.uploadAdmission: a buffered body larger than the budget is not paced but sent throughcreateEnvdSoloFetch→buildSingleUseDispatchedFetch: a fresh single-connection dispatcher (its own HTTP/2 session, so its own 10 MB) that is closed after the response — what undici 7 gave every request in 2.27.1. This keeps a big upload from resetting the reads and small writes multiplexed next to it. HTTP/1.1 skips both 1 and 2.src/undici.ts: when a request fails withERR_HTTP2_STREAM_ERROR+NGHTTP2_ENHANCE_YOUR_CALM/NGHTTP2_INTERNAL_ERROR(isPoisonedHTTP2SessionError), the fetcher retires that dispatcher and builds a fresh one for subsequent requests. The failed request is not replayed (envd may already have applied the write). Only the dispatcher that served the failing request can replace the current one.An earlier revision used
maxConcurrentStreams: 1instead; dropped because undici 8'sPoolparks all requests behind its first ("protocol probe") client until it drains, so one long-lived streamedfiles.readstarved every other REST call to that sandbox.No API change.
Measured (built package, real sandboxes, Node 24.22):
files.read()maxSessionMemory=1, aborted big writes)Upstream: backport request on nodejs/node#67113 (nodejs/node#67113 (comment)) and undici issue nodejs/undici#6149 (poisoned session kept in the pool).
Link to Devin session: https://app.devin.ai/sessions/98f09d080013403b954d3f7b07c51b3b
Open in Devin Desktop: https://app.devin.ai/desktop/session/98f09d080013403b954d3f7b07c51b3b?variant=devin
Requested by: @mishushakov