Skip to content

feat(transport): expose http2_max_local_error_reset_streams on Endpoint - #2708

Closed
gautamg795 wants to merge 1 commit into
grpc:masterfrom
convex:client-http2-max-local-error-reset-streams
Closed

gautamg795 wants to merge 1 commit into
grpc:masterfrom
convex:client-http2-max-local-error-reset-streams

Conversation

@gautamg795

Copy link
Copy Markdown

Adds Endpoint::http2_max_local_error_reset_streams, the client-side counterpart to the existing Server::http2_max_local_error_reset_streams.

Why this is needed. h2 caps the number of local error resets a connection may accumulate over its lifetime (max_local_error_reset_streams, default 1024); once past it, h2 closes the connection with GOAWAY(ENHANCE_YOUR_CALM, "too_many_internal_resets"), failing every in-flight RPC on it. A long-lived client Channel that legitimately cancels many server-streaming RPCs — e.g. dropping the response stream on client disconnect or timeout — steadily accrues these resets: an abandoned stream yields a counted STREAM_CLOSED reset once h2 has forgotten the stream and the peer's in-flight DATA arrives. Under sustained load this trips the limit and tears the connection down.

Server already exposes this knob (for the symmetric server-side case), but Endpoint does not — so today there is no way to raise or disable the limit on an outbound connection without patching tonic.

Behavior. Mirrors the server method's Option<usize> signature. Non-breaking: when unset, hyper's default (currently 1024) is unchanged; only an explicit call overrides it, and None disables the limit (advisable only for trusted peers, e.g. internal services).

Opening as a draft for maintainer feedback on the approach.

🤖 Generated with Claude Code

Adds the client-side counterpart to Server::http2_max_local_error_reset_streams
so callers can raise or disable h2's per-connection local-reset limit. Without
it, a long-lived client connection that cancels many server-streaming RPCs
accrues local resets and is eventually torn down with
GOAWAY(ENHANCE_YOUR_CALM, "too_many_internal_resets"). Leaving it unset
preserves hyper's default (1024).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@linux-foundation-easycla

Copy link
Copy Markdown

CLA Not Signed

@gautamg795

Copy link
Copy Markdown
Author

Closing for now — going to validate this against our internal services first before bringing it upstream. Will reopen once the approach is confirmed. Thanks!

@gautamg795 gautamg795 closed this Jun 26, 2026
@adriangb

Copy link
Copy Markdown
Contributor

@gautamg795 were you ever able to validate? We are hitting this as well, and this seems like the right fix (in addition to hyperium/h2#983). Wdyt of re-opening this?

@LucioFranco

Copy link
Copy Markdown
Member

@adriangb if you are able to test this branch and get back to me I thin we can merge this looking at the diff it seems pretty safe.

adriangb added a commit to pydantic/grpc-rust that referenced this pull request Oct 28, 2026
Adds Endpoint::http2_max_concurrent_reset_streams and
Endpoint::http2_reset_stream_duration, which set hyper's client options of
the same name. A long-lived channel that drops many response streams with
DATA still in flight can then make h2 remember those resets long enough to
ignore the late DATA, rather than count it toward the local error reset
limit and close the connection with GOAWAY(ENHANCE_YOUR_CALM).

Also adds Endpoint::http2_max_local_error_reset_streams, the client-side
counterpart of the Server option, so a caller can raise the limit or
disable it with None. Unset keeps hyper's default of 1024. This follows
grpc#2708.

Requires hyper 1.10, which added the client reset_stream_duration option.
adriangb added a commit to pydantic/grpc-rust that referenced this pull request Oct 28, 2026
Adds Endpoint::http2_max_concurrent_reset_streams and
Endpoint::http2_reset_stream_duration, which set hyper's client options of
the same name. A long-lived channel that drops many response streams with
DATA still in flight can then make h2 remember those resets long enough to
ignore the late DATA, rather than count it toward the local error reset
limit and close the connection with GOAWAY(ENHANCE_YOUR_CALM).

Also adds Endpoint::http2_max_local_error_reset_streams, the client-side
counterpart of the Server option, so a caller can raise the limit or
disable it with None. Unset keeps hyper's default of 1024. This follows
grpc#2708.

Requires hyper 1.10, which added the client reset_stream_duration option.
@gautamg795

Copy link
Copy Markdown
Author

hey @adriangb, I didn't get around to validating and had just closed the PR because it was purely vibe coded and I didn't even intend for my agent to open the PR 😅
I believe the approach is sound though, and should be pretty close to what you want for the fix

LucioFranco pushed a commit that referenced this pull request Oct 29, 2026
Replaces #2969, combined here as
suggested.

## Motivation

When a client resets many streams, a server can still send DATA for
them. h2 remembers only 50 reset streams for 1 s by default. DATA for a
forgotten stream counts toward the local error reset limit (1024), and
reaching it closes the connection with `GOAWAY(ENHANCE_YOUR_CALM,
"too_many_internal_resets")`. `Endpoint` exposes none of these h2 client
options today, so a caller must build the connection with hyper directly
to change them.

## Solution

- Add `Endpoint::http2_max_concurrent_reset_streams(usize)`: how many
reset streams h2 remembers.
- Add `Endpoint::http2_reset_stream_duration(Duration)`: how long h2
remembers a reset stream.
- Add `Endpoint::http2_max_local_error_reset_streams(impl
Into<Option<usize>>)`, the client counterpart of the `Server` option, as
in #2708 (closed). Unset keeps
hyper's default of 1024, and `None` disables the limit.
- Raise tonic's `hyper` requirement from `1` to `1.10`, the first
release with the client `reset_stream_duration` option.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants