fix(rebalance): check needs no signer, and a failed check is as loud as a fired one - #76
Merged
Merged
Conversation
…as a fired one `check` is the command meant to run unattended on a timer. It reads two balances and a price and signs nothing, but cli.ts built a KMS account before the switch regardless of command, so it needed AWS credentials it never used. It died on an expired SSO session -- a scheduled monitor that fails whenever a human's login lapses is not a monitor. Three things, because the first alone would be reintroduced by the next refactor: 1. createReadClient() builds chain reads only, and check() is typed to ReadClients, so tsc refuses any future edit that reaches for a signer there. 2. cli.ts takes injectable `readClients` / `signingClients`, and cli.test.ts asserts the signing factory is NEVER called for `check` or `quote` -- with a negative control asserting `deposit` DOES call it, so the first two cannot pass for the wrong reason. The regression this guards against is hoisting client creation above the switch again: that typechecks, leaves CI green, and breaks only the unattended path. 3. A failed check now alerts. Unattended, a crash into a log nobody reads is the same failure as an alert that reaches nobody, so a run that cannot complete posts "FAILED TO RUN ... Inventory is UNKNOWN, not healthy" to the same webhook a real finding would use, and still exits non-zero so a scheduler's OnFailure catches the case where the webhook is what broke. Two further defects found while testing: - cli.ts ran main() at module scope, so importing it executed the CLI and threw on loadConfig before a test could reach anything. Now guarded on being the entry point. - `--alert` validated ALERT_WEBHOOK_URL only at the point of sending, which is after the healthy-verdict early return. A --alert run with no webhook configured therefore looked fine for as long as the inventory was fine, and would have failed for the first time on the run that finally had something to say. Validated up front now. Found by a test written against the old behaviour, which was right to fail. Verified against the failure that exposed it: with AWS_PROFILE unset, the AWS config and credentials files pointed at /dev/null and the SSO session still expired, `pnpm rebalance check` runs and exits 0. Its own logic then reports sub 15 at cNGN 53.2%, action none. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two things found while confirming the entry-point guard fires from every real
invocation path, as asked.
**The guard did not survive a symlink.** `import.meta.url` is always the real
path; `process.argv[1]` is whatever was typed. Invoked through a symlink the two
differ, the guard does not fire, and the CLI prints nothing and exits 0:
$ ln -s .../dist/cli.js /tmp/rebalance
$ node /tmp/rebalance check
$ # no output, exit 0
Silent success is the worst failure available to a scheduled job, and it is
exactly the shape scheduling would produce: `node_modules/.bin/*` is a symlink,
so adding a `bin` entry creates it, as does a symlinked unit path. argv[1] now
resolves through realpath, and `isEntryPoint` is exported and tested against a
real symlink. Verified across all four paths -- pnpm/tsx, compiled dist, an
absolute path from another cwd, and through a symlink.
There is no container for this package: no Dockerfile, no bin entry, no image.
The built path that exists is `pnpm build` -> `dist/cli.js`, and it is covered.
**--heartbeat, so silence cannot pass for healthy.** A healthy `--alert` run
posts nothing, which is indistinguishable from a timer that stopped firing, a
host that went away, or credentials that lapsed. `--heartbeat` posts on every
run whatever the verdict, carrying the balances and share with it, so the
message is evidence rather than a bare ping. Intended shape: schedule `--alert`
often (pages only on a finding) and `--heartbeat` rarely (proves the checker is
alive). Stateless, so no last-run file to go stale.
A failed run still pages under either flag.
26 tests. The symlink test initially failed on a fixture of my own making --
macOS tmpdir() sits under a symlinked /var, so the hand-built module URL was not
the real path that Node would give.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Anything fired by hand into a shared ops channel must say so. Without it a drill is indistinguishable from a real finding, and someone spends their afternoon investigating a venue that is fine -- which is exactly what happened on 2026-09-21, when a forced-failure drill reached the channel unmarked. `--test` prefixes "[TEST] " to whatever is posted, on both the healthy/heartbeat path and the forced-failure path. Two tests pin both. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The defect
checkis the command meant to run unattended on a timer. It reads two balances and a price and signs nothing — butcli.tsbuilt a KMS account before the switch regardless of command, so it required AWS credentials it never used. It died on an expired SSO session.A scheduled monitor that fails whenever a human's login lapses is not a monitor.
Three changes, because the first alone would be undone
1.
createReadClient()builds chain reads only, andcheck()is typed toReadClients— tsc now refuses any future edit that reaches for a signer there.2. A test that proves it.
cli.tstakes injectablereadClients/signingClients, andcli.test.tsasserts the signing factory is never called forcheckorquote, with a negative control assertingdepositdoes call it — so the first two can't pass for the wrong reason.The regression guarded against is specific: hoisting client creation above the switch again. That typechecks, leaves CI green, and breaks only the unattended path.
3. A failed check now alerts. Unattended, a crash into a log nobody reads is the same failure mode as an alert that reaches nobody. A run that can't complete posts to the same webhook a real finding would use:
The wording is deliberate: a failed check must never read as a clean bill of health. It still exits non-zero, so a scheduler's
OnFailurecatches the case where the webhook itself is what broke.Two more defects found while testing
cli.tsranmain()at module scope, so importing it executed the CLI and threw onloadConfigbefore a test could reach anything. Now guarded on being the entry point.--alertvalidatedALERT_WEBHOOK_URLonly at the point of sending — which is after the healthy-verdict early return. Socheck --alerton a timer with no webhook configured would look fine for as long as the inventory was fine, and fail for the first time on the run that finally had something to say. Validated up front now.That second one was found by a test written against the old behaviour. It failed, and it was right to — I changed the code, not the test.
Verification
Against the exact failure that exposed it —
AWS_PROFILEunset, AWS config and credentials files at/dev/null, SSO still expired:21 tests pass;
./scripts/verify.sh nodegreen.Note what that output is:
check's own thresholds, price source and inventory definition reporting the verdict. A previous claim that "no rebalance is due" came from reading the balances directly after the command failed — which confirmed a reimplementation of the logic, not the logic itself.🤖 Generated with Claude Code