Skip to content

feat: Let HTTPProxy rules set the response timeout - #897

Open
0xmc wants to merge 2 commits into
mainfrom
fix/httpproxy-route-timeout
Open

0xmc wants to merge 2 commits into
mainfrom
fix/httpproxy-route-timeout

Conversation

@0xmc

@0xmc 0xmc commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Every response through a load balancer was cut at about 15 seconds, so streams, large downloads and slow APIs failed mid-transfer with nothing in the proxy API to change it.

The generated route carried no timeout, so Envoy applied its own default, and the only way around it was an Envoy Gateway policy aimed at a route the user never created.

Each proxy rule now carries the Gateway API timeouts, with the request timeout defaulting to one hour and capped at the same limit the operator already enforces on policies.

A timeout of zero is refused, because Envoy Gateway would drop the idle timeout with it and leave abandoned streams open.

Test plan

  • A 20-second streaming response completes through the edge with the default, and the same stream is cut when the rule sets a 5-second timeout
  • A timeout above the maximum or of zero is refused at admission with a message naming the field and the limit
  • Unit, envtest and lint pass
  • After release, a production proxy streams past 15 seconds and the per-route timeout counter stops rising on routes that were cut

Fixes #459

0xmc added 2 commits November 10, 2026 14:34
The generated HTTPRoute carried no timeouts, so Envoy applied its own
15-second default to every response. Streams, large downloads and slow
APIs were cut mid-transfer, and nothing in the HTTPProxy API could
change it. Customers reached past the operator with a
BackendTrafficPolicy on the generated route or gateway.

Each rule now carries the Gateway API timeouts, copied to the route.
The request timeout defaults to one hour, the cap the operator already
enforces on BackendTrafficPolicy, so a timeout set on a proxy and one
set on a policy meet the same limit. Zero is refused: Envoy Gateway
drops the route's idle timeout along with a disabled request timeout.

Key changes:
- Add timeouts to HTTPProxyRule with a default request timeout of 1h
- Copy the rule's timeouts onto the generated HTTPRoute rule
- Cap request and backendRequest at the configured maximum and refuse 0s
- Add an e2e scenario that streams 20 s through the edge with the
  default and sees the same stream cut by a 5 s rule timeout
- Replace the BackendTrafficPolicy workaround in the plugin skills

Fixes #459
The CRD default applies only when a rule leaves timeouts unset. A rule
that set backendRequest alone left request empty, and Envoy's 15-second
default returned for that rule. The mutating webhook now fills request
with the default whenever it is missing, and the e2e scenario covers a
rule that sets only backendRequest.

The default is fixed at one hour while the cap is operator config. A cap
below the default would turn every proxy without a timeout of its own
Accepted=False on the next reconcile, so operator config now refuses a
cap below the default at startup.

Key changes:
- Fill a missing request timeout in the HTTPProxy defaulter
- Refuse an httpMaxRequestTimeout below the HTTPProxy default
- Assert the defaulted request timeout on a partly set rule end to end
@0xmc
0xmc marked this pull request as ready for review November 10, 2026 20:14
@0xmc
0xmc requested a review from a team as a code owner November 10, 2026 20:14
@0xmc
0xmc enabled auto-merge November 10, 2026 20:27

@ecv ecv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes.

The new request: 1h default silently overrides any shorter timeout a user already set with a BackendTrafficPolicy. In Envoy Gateway v1.8.1, getEffectiveRequestTimeout (internal/xds/translator/route.go:407-418) gives the HTTPRoute rule timeout precedence and only falls back to the policy's timeout.http.requestTimeout when the rule has none. After this change every generated rule carries one, so a user's deliberate 30s fail-fast limit becomes 1h on the first reconcile, with nothing in status to say so.

Ask: leave the rule timeout unset, or skip copying the default, when a BackendTrafficPolicy with requestTimeout targets the proxy, and cover that case in the chainsaw test.

Otherwise sound. Zero is refused for both fields in the webhook and controller, backendRequest is held to request by CEL, and the default is idempotent.

Follow-ups:

  • The new startup check refuses httpMaxRequestTimeout below 1h, which was valid before. No known deployment sets it, but say so in the release notes.
  • The default moves every existing proxy from Envoy's 15s to 1h, so release it on a tag of its own and watch edge connection counts.
  • Lowering httpMaxRequestTimeout later turns stored proxies above it Invalid at reconcile; document that in the config docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streaming responses are cut off after fifteen seconds

2 participants