chore(scripts): let seeding establish its own port-forwards - #643
Merged
Conversation
Seeding needs localhost:8080 and localhost:5432, but nothing established either -- they were a documented manual prerequisite. That was never workable, for a reason that is structural rather than a lapse in diligence: stop_port_forward()'s legacy reaper matches any "-n tmi-platform port-forward svc/" command, so a hand-started forward is killed as an "orphan" by the next dev-up/dev-restart/dev-down. It then resurfaces as a confusing failure inside the seed rather than as a missing prerequisite. Adds ensure_port_forward(name), modelled on tmi_common.ensure_oauth_stub: probe, return quietly if already healthy, otherwise start and wait. It is generic over the three named dev forwards; only seeding is wired to it for now. A forward whose recorded kube context is no longer the current one counts as unhealthy and is replaced rather than reused -- a live supervisor pointing localhost at another cluster is the #580 failure mode. scripts/run-dbtool.py is the single wiring point, which covers every way seeding is invoked: make cats-seed, make e2e-seed, and the cats plugin's seed hook (hence make cats-fuzz). Skipped for --oci (external ADB, no Service to forward) and when --server is not loopback. Postgres gets a first-class supervised forward, pidfile-tracked like the others so stop_port_forward() tears it down deliberately. It is deliberately NOT started by dev-up: 5432 collides with a locally installed PostgreSQL, and a developer who never runs CATS should not have to care. On a collision _preflight_port already hard-fails naming the squatter; the message now adds that a local PostgreSQL commonly holds that port, and why seeding must not silently use it. Two fixes found while doing this: - start_redis_port_forward() called the blanket stop_port_forward(), which cleared every pidfile. That forced the server forward to be started afterwards to survive -- an ordering constraint that would silently extend to any third forward -- and destroyed the postgres forward on every dev-up/dev-restart. Each starter now stops only its own pidfile. Squatters are still handled by the existing marker-scoped orphan reaping and _preflight_port. - ensure returned ~3s before the port was bound, so the contract was "started" rather than "usable" and an immediate connect could race it. It now waits for the bind, as ensure_oauth_stub does. test_server_port_forward_is_k3s_only is widened for the one new call site, the by-name dispatch inside ensure_port_forward, whose callers know the server URL is loopback rather than knowing cluster_target. Any other unguarded call site still fails. The campaign still hits the NodePort directly -- this path only ever carries low-volume seeding traffic, which is what #463/#578 actually protect. Verified against the live k3s cluster: cats-seed starts the forward and seeds from a cold start; a second call is a no-op; the forward now survives dev-restart; stop_port_forward() still tears all three down deliberately. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WnrVyySuyJFqcMxB8q3pNc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Seeding needs
localhost:8080andlocalhost:5432, but nothing established either — theywere a documented manual prerequisite.
Why the manual approach could never work
This isn't a lapse in developer diligence.
stop_port_forward()'s legacy reaper matches any-n tmi-platform port-forward svc/command, so a hand-started forward is killed as an"orphan" by the next
dev-up/dev-restart/dev-down. It then resurfaces as a confusingfailure inside the seed rather than as a missing prerequisite.
The change
ensure_port_forward(name), modelled ontmi_common.ensure_oauth_stub: probe → returnquietly if healthy → otherwise start and wait. Generic over the three named dev forwards;
only seeding is wired to it for now, per the agreed scope.
A forward whose recorded kube context is no longer the current one counts as unhealthy and
is replaced rather than reused — a live supervisor pointing localhost at another cluster is
the #580 failure mode.
scripts/run-dbtool.pyis the single wiring point, which covers every way seeding is invoked:make cats-seed,make e2e-seed, and the cats plugin'sseedhook (hencemake cats-fuzz).Skipped for
--oci(external ADB, no Service to forward) and when--serverisn't loopback.Postgres gets a first-class supervised forward, pidfile-tracked so
stop_port_forward()tearsit down deliberately. It is not started by
dev-up— 5432 collides with a locallyinstalled PostgreSQL, and a developer who never runs CATS shouldn't have to care. On a
collision
_preflight_portalready hard-fails naming the squatter; the message now adds thata local PostgreSQL commonly holds that port, and why seeding must not silently use it.
Two bugs found while doing this
start_redis_port_forward()called the blanketstop_port_forward(), clearing everypidfile. That forced the server forward to start afterwards to survive — an ordering
constraint that would silently extend to any third forward — and destroyed the postgres
forward on every
dev-up/dev-restart. Each starter now stops only its own pidfile.ensurereturned ~3s before the port was bound, so the contract was "started" ratherthan "usable" and an immediate connect could race it. It now waits for the bind, as
ensure_oauth_stubdoes.Test change, called out explicitly
test_server_port_forward_is_k3s_onlyis widened for the one new call site — the by-namedispatch inside
ensure_port_forward, whose callers know the server URL is loopback ratherthan knowing
cluster_target. Any other unguarded call site still fails. The campaign stillhits the NodePort directly; this path only ever carries low-volume seeding traffic, which is
what #463/#578 actually protect.
Verification
make test-dev-scripts— 164 tests, OK (10 new).make lint— 0 issues. Only Python changed,so the Go gates don't apply.
Against the live k3s cluster:
make cats-seedwith 5432 downensurecallPort-forward already running— no churnmake dev-restartstop_port_forward()ensurereturns🤖 Generated with Claude Code
https://claude.ai/code/session_01WnrVyySuyJFqcMxB8q3pNc