Repository navigation
cache: honor backend phase timeouts in origin fetches - #4372
Closed
larry-dalmeida wants to merge 1 commit into
Closed
larry-dalmeida wants to merge 1 commit into
larry-dalmeida wants to merge 1 commit into
Conversation
larry-dalmeida
force-pushed
the
larry-almeida-zse-cache-correctness-pr
branch
from
October 23, 2026 06:47
f26e248 to
ebcd7ff
Compare
larry-dalmeida
force-pushed
the
larry-almeida-zse-cache-timeouts-pr
branch
from
October 23, 2026 06:49
a429323 to
509147c
Compare
This was referenced Oct 23, 2026
This was referenced Oct 26, 2026
Wire TCP dial, response-header, TLS handshake, and expect-continue backend options to the cache client transport. Preserve dial-only semantics instead of using http.Client.Timeout. Match proxy defaults for zero header/expect-continue settings. Reproduce the ignored header deadline and verify timeout SIE retention, unchanged timestamps, and healthy recovery using a real delayed origin. Signed-off-by: Larry D Almeida <hello@larrydalmeida.com> Signed-off-by: Larry D Almeida <hello@larrydalmeida.com>
larry-dalmeida
force-pushed
the
larry-almeida-zse-cache-timeouts-pr
branch
from
October 26, 2026 14:37
509147c to
7e48ec0
Compare
larry-dalmeida
force-pushed
the
larry-almeida-zse-cache-correctness-pr
branch
from
October 26, 2026 14:37
ebcd7ff to
e003f9a
Compare
1 of 2 tasks
|
Docker image "registry-write.opensource.zalan.do/teapot/skipper:7e48ec0fb570d5e26b9ad551ec7a9d25dfef209d" is not based on an approved base image. Any production deployment relying on this image will be blocked. To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Zalando Container Registry namespace |
Collaborator
Author
|
Superseded by #4394. Changes consolidated into five focused commits with published e2e evidence. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
Apply configured backend phase timeouts to cache-origin requests. These bound transport phases, not the entire request or response-body read.
How to Verify
Shared setup
make verifystarts or reuses Minikube, the mock origin and the stack; runs the smoke and coalescing checks; and validates supported dashboard queries. It prints a report path and exits non-zero on failure. For outage evidence, runmake chaos SKIPPER_CHECKOUT=../skipper-pr-4387 NAME=origin-stale; it includes its own timed load. Requirements: Python 3.9+, Make, Bash, Git, curl, kubectl, Minikube, Docker and Go compatible with Skipper'sgo.mod. Free ports: 8080, 8081, 6379, 4318, 9091, 9999, 9911 (8089 for endless load). Dynamic URLs:.runtime/urls.txt; stop withmake stop.Validated on #4387 (
d767f636): smoke: 6,398 requests, 0 failures; origin-stale: 155 failed revalidations across 77 routes. Smoke metric checks: 42 PASS, 2 fault-only NOT EXERCISED, 0 FAIL.Focused tests
Expected: 2 focused tests pass. Delayed-header coverage verifies the 50 ms response-header timeout, unchanged last-good stale body, then MISS/HIT recovery.
origin-staleinjects immediate 503s, not delays; origin p95 does not measure Skipper's wait.Checklist
Rollback Plan
Revert this PR to restore previous cache-origin timeout behavior.
References & Additional Context
Depends on #4371. The combined candidate passed a 60,000-request local exercise; the larger staircase campaign remained incomplete after a load-generator admission drop.
E2E Testing Results
E2E runs exercising origin fetches with the stack applied. These runs set no backend phase timeouts and injected no origin delay, so no timeout fired. Full results: larry-dalmeida/skipper-cache-e2e-tests/results
Test Scenarios:
Validation Panels:
Not shown by these runs: a backend phase timeout firing. Time to First Hit (
skipper_cache_first_hit_seconds) is added by #4385 later in the stack, so it is not evidence for this PR.