[ideaship]

Two scripts are fixed and one is deleted. render-images.py and
set-versions.py are made to check the HTTP status of the files they fetch
from the release repository; render-ansible-requirements.py held a third copy
of the same bug but has been dead since 2023, so it is removed rather than
fixed.

A 429 from raw.githubusercontent.com was parsed as if it were the versions
file. Every config build fetches base.yml and images.yml unauthenticated,
raw.githubusercontent.com rate-limits per IP, and neither render-images.py
nor set-versions.py looked at the status code before handing r.text to
yaml.full_load — so a rate limit that clears in a minute failed the build
with a traceback naming neither the status nor the URL.

The symptom is not even stable: it depends on whether the error body happens to
parse as YAML. GitHub's 199-byte 429 body gives a yaml composer error about
the response body; an HTML error page parses as a bare string and fails further
down with a TypeError once it is indexed like a dict. grep -rn raise_for_status src/ returned nothing — no call site in this repo checked a
status.

What is here

Two commits, reviewable independently:

1. Guard and retry the release repo fetches. Both scripts fetch through a
   fetch_yaml() helper — requests.Session whose adapter carries urllib3
   Retry (total=4, backoff_factor=1.5, status_forcelist 429/500/502/
   503/504) followed by raise_for_status(). A self-clearing rate limit is now
   waited out instead of failing the build; a persistent one reports its status
   and full URL.
2. Remove the unused render-ansible-requirements.py. Dead since 2023 and
   unrelated to the fix except that it held a fourth copy of the same unguarded
   requests.get — see below.

Two decisions worth a reviewer's attention

The helper is duplicated, not shared. gilt.yml copies each script alone
into the operator's configuration repo, runs it there, and rms it —
render-images.py into environments/manager/, set-versions.py into
environments/. There is no importable sibling module at the point of use, so
a single shared helper is not reachable. ~20 lines are duplicated deliberately.

raise_on_status=False is deliberate. With urllib3's default, an exhausted
status_forcelist retry raises RetryError — "too many 429 error responses"
against a connection pool, no URL. With it off, the final response comes back
and raise_for_status() names the status and the full URL, the same shape a
never-retried status produces. One error shape for every failure, and it always
names the URL.

The adapter is also mounted for http://, because VERSIONS_URL and
IMAGES_URL are environment-overridable and an http:// override would
otherwise silently skip the retries.

Why the second commit deletes a script rather than guarding it

render-ansible-requirements.py rendered
environments/manager/requirements.yml. That output file no longer exists:

• manager: remove ansible requirements.yml file #362 removed it and moved the role and collection versions
  into environments/manager/run.sh
• remove render-ansible-requirements.py from gilt.yml #365 then dropped the script from gilt.yml

Nothing has invoked it since — not gilt.yml, Makefile, tox.ini,
.zuul.yaml, environments/ or inventory/, and no other OSISM repository
refers to this copy. It did not look dead because it kept being maintained:
#382, #383 and #384 swept
changes across the render scripts seven weeks after the unwiring and picked it
up along the way, which is how it acquired the unguarded requests.get in the
first place.

The functionality survives, reimplemented: the osism-ansible container image
carries its own render-ansible-requirements.py under files/src, which reads
base.yml from a git checkout of osism/release made during the image build and
imports no HTTP client at all. It never had this failure mode.

Verification

Against a local server reproducing the 429 pattern:

scenario	result
429, 429, 200	recovers in 3.0s, returns parsed YAML
persistent 429	429 Client Error: Too Many Requests for url: ... after 21s of backoff
404	fails immediately, no pointless retries

Worst case adds ~21s per URL, and render-images.py fetches two.

Against live GitHub, running what generics-tox-test-* runs
(python3 src/render-images.py) for all three pinned versions — latest,
9.1.0, 9.0.0: all render, and the output is byte-identical to main in
every case, so the change is behaviour-preserving on the happy path.

flake8 and python-black pass at the versions the Zuul jobs pin.

In CI, on patchset 321fdec

osism/check is green — all 10 jobs
(buildset).
Three of them execute the changed code rather than just linting it, each
against live GitHub with a different pin:

job	MANAGER_VERSION	build
generics-tox-test-latest	latest	3bdbc2b7
generics-tox-test-stable	9.1.0	cf83435e
generics-tox-test-stable-legacy	9.0.0	ca7eec1f

In generics-tox-test-latest, python3 src/render-images.py exits 0 in 1.15s
and the cat images.yml that follows it is byte-identical to the 56 lines
main renders. These three jobs were themselves exposed to this bug whenever
CI tripped the rate limit; they now exercise the fix on every change.

Limitation

No automated test accompanies this. There is no tracked test suite for these
scripts, and the failure needs an HTTP server returning controlled statuses to
exercise; the verification above was run by hand.

🤖 Generated with Claude Code