Repository navigation
Add a proxy config key for the upstream connection idle timeout - #900
Merged
Merged
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Negative timeout validation and snapshot deep-copy handling remain unresolved; release documentation is also missing.
3 open issues
What changed in this PR
Adds upstream_connection_idle_timeout_ms for HTTP, HTTP/2, and gRPC upstream clusters.
Changes:
- Adds proxy configuration parsing.
- Applies Envoy upstream idle timeouts.
- Adds XDS golden coverage.
| File | Reviewed change |
|---|---|
agent/xds/testdata/secrets/custom-upstream-connection-idle-timeout.latest.golden |
Adds HTTP golden output. |
agent/xds/testdata/secrets/custom-upstream-connection-idle-timeout-http2.latest.golden |
Adds HTTP/2 golden output. |
agent/xds/testdata/routes/custom-upstream-connection-idle-timeout.latest.golden |
Adds HTTP route golden output. |
agent/xds/testdata/routes/custom-upstream-connection-idle-timeout-http2.latest.golden |
Adds HTTP/2 route golden output. |
agent/xds/testdata/listeners/custom-upstream-connection-idle-timeout.latest.golden |
Adds listener golden output. |
agent/xds/testdata/listeners/custom-upstream-connection-idle-timeout-http2.latest.golden |
Adds HTTP/2 listener golden output. |
agent/xds/testdata/endpoints/custom-upstream-connection-idle-timeout.latest.golden |
Adds endpoint golden output. |
agent/xds/testdata/endpoints/custom-upstream-connection-idle-timeout-http2.latest.golden |
Adds HTTP/2 endpoint golden output. |
agent/xds/testdata/clusters/custom-upstream-connection-idle-timeout.latest.golden |
Verifies configured cluster timeout output. |
agent/xds/testdata/clusters/custom-upstream-connection-idle-timeout-http2.latest.golden |
Verifies HTTP/2 cluster timeout output. |
agent/xds/resources_test.go |
Registers the new golden test cases. |
agent/xds/config/config.go |
Defines the new proxy configuration field. |
agent/xds/clusters.go |
Applies the timeout to upstream clusters. |
Review effort: Lite
joec4i
force-pushed
the
upstream-connection-idle-timeout
branch
from
October 19, 2026 04:33
96354b0 to
9b7fe43
Compare
upstream_connection_idle_timeout_ms sets common_http_protocol_options idle_timeout on every http, http2 and grpc upstream cluster. Envoy's default is 1h and Consul had no way to change it. When an intention change drains a destination sidecar's public listener, Envoy closes the connections still idle at the end of the drain. A stateful firewall between the two hosts that has already forgotten an idle connection (GCP's VPC firewall does after 10 minutes) drops that FIN, so the caller keeps the connection and its next request on it gets a reset and a 503. Closing idle upstream connections on the caller before the firewall forgets them, set globally in proxy-defaults, avoids that. Unset by default, so existing proxies get the same config as before. 0 disables the timeout, and negative values are ignored because Envoy would reject them along with every HTTP upstream cluster. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
joec4i
force-pushed
the
upstream-connection-idle-timeout
branch
from
October 19, 2026 05:27
9b7fe43 to
e9627b2
Compare
alowde
approved these changes
Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds
upstream_connection_idle_timeout_msto the proxy config. It setscommon_http_protocol_options.idle_timeouton every http, http2 and grpc upstream cluster. Envoy's default is 1h and Consul had no way to change it. Unset by default, so existing config doesn't change. 0 disables the timeout, and negative values are ignored.When an intention change drains a destination sidecar, Envoy closes the connections still idle at the end of the drain. If a stateful firewall between the hosts has already forgotten an idle connection (GCP's VPC firewall does after 10 minutes), it drops that FIN, the caller keeps the connection, and its next request gets a reset and a 503. Setting this in
proxy-defaultsbelow the firewall's timeout makes callers close idle connections first. tcp upstreams aren't covered; theirs is thetcp_proxyidle timeout.To be backported to
1.20.x-bcand1.22.x-bcafter merge.Testing & Reproduction steps
custom-upstream-connection-idle-timeout, its-http2variant, andterminating-gateway-upstream-connection-idle-timeout. No existing goldens change.agent/xdsandagent/proxycfgtests pass.--drain-time-s 30), and a 20s conntrack timeout on the Nomad client standing in for the firewall. Each run: one request, an intention change on the destination, then another request 40s later. Without the key, the caller's socket is still ESTABLISHED after the drain and the request gets a 503 (upstream_cx_destroy_remote_with_active_rq+1). With it set to 10000, the caller closes the connection at 10s (upstream_cx_idle_timeout+1) and the request gets a 200. Same onv1.20.6-bc.2with Envoy 1.33.14.Links
HttpProtocolOptions.idle_timeoutPR Checklist
🤖 Generated with Claude Code