Rollout check fails a deploy on one dropped public connection and cannot say why #5170

🤖 Generated by the Agentic Engineer

Evidence

On 2026-10-20 23:39Z the production deploy of #5129 (a Kyverno policy change, unrelated to data products) failed at Verify the deployed data product UI release and removed #5129 from the merge queue:

{"complete":false,"failure":"public_contract_incomplete","phase":"public_contract","role":"descriptor_library","reason":"transport_failed"} (run 39513604314, job 118944678905)

• The rollout comparison itself passed. The failure was the last public fetch (descriptor.js); earlier fetches to the same host in the same pass succeeded.
• The file was healthy: 13,301 bytes, and three probes about two minutes later returned 200 in under 0.3 s.
• It is the only transport_failed in the 215 deploys that reached this check since 2026-10-07 (read for Production deploys fail intermittently at the data product rollout check and evict unrelated pull requests #4545). Each eviction costs a re-queue and roughly one more deploy cycle (~30 min) for every PR behind it.

Problem

fetch() in scripts/verify-data-product-ui-rollout.sh maps only curl exits 6, 28, 35 and 60 to a named reason, and sends curl's stderr to /dev/null. Every other exit — a refused connection (7), an empty reply (52), a reset during receive (56) — becomes transport_failed. So the log cannot say what happened, and a single dropped connection fails the deploy outright even though the step has time left within its existing deadline.

Hypothesis

If the check names the curl exit code and retries a public fetch once when the connection itself failed, a one-off dropped connection stops evicting unrelated PRs, and any repeat names its cause.

Acceptance criteria

• A refusal from a public fetch carries curl's numeric exit code as a fixed field (no response data, hostnames or addresses in the log).
• A connection-level failure (exit 7, 52 or 56) is retried once within the remaining deadline; a retry never extends the deadline.
• DNS, TLS, timeout, HTTP status, body and header failures are not retried and keep their current reasons.
• The script's own tests cover: one dropped connection then success (passes), two dropped connections (fails with the exit code), and a non-retryable failure (fails on the first attempt).

Success signal

No deploy evicted by transport_failed over the next 4 weeks; any public-check refusal in that window names its curl exit.

Context: split from #4545, which closed after its own rollout-comparison criteria were met.