[larry-dalmeida]

Summary of changes

Adds per-route cache result tracking (hit/miss/stale) and first-hit observation time.

How to Verify

Shared setup

mkdir skipper-review && cd skipper-review
git clone https://github.com/zalando/skipper.git skipper && cd skipper
for PR in 4369 4370 4371 4372 4384 4385 4386 4387; do
  git fetch origin pull/$PR/head:pr-$PR
  git worktree add ../skipper-pr-$PR pr-$PR
done
cd ..
git clone https://github.com/larry-dalmeida/skipper-cache-e2e-tests.git
cd skipper-cache-e2e-tests
make verify SKIPPER_CHECKOUT=../skipper-pr-4387

make verify starts or reuses Minikube, the mock origin and the stack; runs the smoke and coalescing checks; and validates supported dashboard queries. It prints a report path and exits non-zero on failure. For outage evidence, run make chaos SKIPPER_CHECKOUT=../skipper-pr-4387 NAME=origin-stale; it includes its own timed load. Requirements: Python 3.9+, Make, Bash, Git, curl, kubectl, Minikube, Docker and Go compatible with Skipper's go.mod. Free ports: 8080, 8081, 6379, 4318, 9091, 9999, 9911 (8089 for endless load). Dynamic URLs: .runtime/urls.txt; stop with make stop.

Validated on #4387 (d767f636): smoke: 6,398 requests, 0 failures; origin-stale: 155 failed revalidations across 77 routes. Smoke metric checks: 42 PASS, 2 fault-only NOT EXERCISED, 0 FAIL.

Focused tests

go -C ../skipper-pr-4385 test -race -count=1 ./filters/cache -run '^TestRouteLabel$'
go -C ../skipper-pr-4385 test -race -count=1 ./metrics -run '^(TestPrometheusCacheRouteResult|TestPrometheusCacheFirstHitSeconds)$'

Expected: 3 focused tests pass. skipper_cache_route_result_total{route,result} labels configured route IDs, so cardinality follows route count, not URLs. Dashboard PromQL derives route families only for IDs matching <family>__<variant>; other setups can group by route or adapt the expression. First-hit metric has samples.

Checklist

• Added regression tests and documentation.
• Confirm CI passes on the current head.
• Confirm passess on manual run of e2e tests for all known scenarios including past bugs (results: larry-dalmeida/skipper-cache-e2e-tests/results/README.md)

Rollback Plan

Revert this PR to restore previous metrics behavior. Revert #4386 first if merged.

References & Additional Context

Depends on #4384; followed by #4386. Adds per-route result distribution and first-hit timing metrics.

E2E Evidence

Latest-chain verification: smoke report (6,398 requests, 0 failures). The stack run uses #4387, which includes this PR; see the shared setup above.

[larry-dalmeida]

Superseded by #4395, stacked above #4394. Changes consolidated with published metric, dashboard and fault-test evidence.