Scale idle applications to zero without degrading UX #3418

🤖 Generated by the Agentic Engineer

Problem

Non-critical HTTP applications currently retain fixed replicas while idle. That consumes schedulable capacity, but scaling them directly to zero can send the first request to a Service with no ready endpoints and can materially worsen time to first render.

Kubernetes 1.37 promotes HPA scale-to-zero to Beta for Object and External metrics. Platform already installs KEDA 2.20.2 and has min-zero node-autoscaler overflow pools, but it has no request-holding HTTP scaler, no ScaledObject/zero-replica WebApp shape, and no cold-start user-experience contract.

Production is currently pinned to Kubernetes 1.36.4 on Talos 1.13.9 and retains six static baseline nodes. Pod scale-to-zero creates realised infrastructure savings only when it prevents or removes an overflow node; otherwise it creates capacity headroom. No provider-spend claim should be made without node-count evidence.

Audience and outcome

As a platform operator,
I want suitable non-critical applications to scale to zero through an opt-in, request-buffered path,
so that idle capacity can be reclaimed without failed first requests or an unacceptable first-render regression.

Hypothesis and success signal

If request activation is held until a restored replica is Ready, application and image caches are prepared deliberately, and only cold-start-tolerant workloads opt in, then idle replicas can reach zero without degrading the user journey.

Success requires all of the following:

• Zero failed first requests during measured scale-from-zero trials.
• Cold and warm TTFB/FCP/LCP remain within an explicitly agreed workload budget.
• Authenticated and anonymous routes preserve correct cache and privacy semantics.
• A production observation shows an overflow node was avoided or removed, or the outcome is honestly classified as capacity headroom rather than realised savings.
• Critical control-plane, networking, DNS, identity, observability, database, GitOps, and autoscaling components remain always on.

Smallest useful path

1. Add a default-off, request-buffered scale-to-zero capability to the Platform WebApp abstraction and prove it with whoami.
2. Prepare and measure one real user-facing workload before opting it in; wedding-app is the first candidate, while its PostgreSQL database remains warm.
3. Evaluate Pod checkpoint/restore separately as a future warm-start accelerator; do not couple the alpha feature to the first production pilot.

The linked sub-issues are the delivery structure and source of truth.

Guardrails

• Feature-flag first: disabled by default and tested in both states.
• Re-check the then-current Kubernetes, KEDA HTTP activation, Talos, container-runtime, and Gateway API support before implementation.
• Do not use CPU or memory as the zero-to-one activation signal.
• Do not share-cache authenticated or personalized responses.
• Do not publish private billing evidence or claim savings from allocation alone.
• A failed UX or node-count gate rolls the workload back to a warm replica; it does not relax the gate.

Definition of done

• Every delivery and evaluation child is linked as a real sub-issue and reaches its own definition of done.
• At least one real workload completes a default-off → enabled → rollback-capable production pilot with user-path evidence.
• The measured outcome distinguishes user-experience impact, capacity reclaimed, and any proven overflow-node reduction.
• The supported workload classes and permanent exclusions are documented.

References

• Kubernetes 1.37 HPA scale-to-zero
• KEDA scaling model
• KEDA HTTP Add-on request holding
• Platform roadmap parent: Roadmap: a resilient, current, well-abstracted GitOps platform #2043