Skip to content

fix(rebalance): check needs no signer, and a failed check is as loud as a fired one - #76

Merged
robertleifke merged 3 commits into
mainfrom
fix/check-needs-no-signer
Sep 22, 2026
Merged

robertleifke merged 3 commits into
mainfrom
fix/check-needs-no-signer

Conversation

@robertleifke

Copy link
Copy Markdown
Contributor

The defect

check is the command meant to run unattended on a timer. It reads two balances and a price and signs nothing — but cli.ts built a KMS account before the switch regardless of command, so it required AWS credentials it never used. It died on an expired SSO session.

A scheduled monitor that fails whenever a human's login lapses is not a monitor.

Three changes, because the first alone would be undone

1. createReadClient() builds chain reads only, and check() is typed to ReadClients — tsc now refuses any future edit that reaches for a signer there.

2. A test that proves it. cli.ts takes injectable readClients / signingClients, and cli.test.ts asserts the signing factory is never called for check or quote, with a negative control asserting deposit does call it — so the first two can't pass for the wrong reason.

The regression guarded against is specific: hoisting client creation above the switch again. That typechecks, leaves CI green, and breaks only the unattended path.

3. A failed check now alerts. Unattended, a crash into a log nobody reads is the same failure mode as an alert that reaches nobody. A run that can't complete posts to the same webhook a real finding would use:

cNGN rebalance check FAILED TO RUN (sub 15): <why>. Inventory is UNKNOWN, not healthy — nothing has been checked.

The wording is deliberate: a failed check must never read as a clean bill of health. It still exits non-zero, so a scheduler's OnFailure catches the case where the webhook itself is what broke.

Two more defects found while testing

  • cli.ts ran main() at module scope, so importing it executed the CLI and threw on loadConfig before a test could reach anything. Now guarded on being the entry point.
  • --alert validated ALERT_WEBHOOK_URL only at the point of sending — which is after the healthy-verdict early return. So check --alert on a timer with no webhook configured would look fine for as long as the inventory was fine, and fail for the first time on the run that finally had something to say. Validated up front now.

That second one was found by a test written against the old behaviour. It failed, and it was right to — I changed the code, not the test.

Verification

Against the exact failure that exposed it — AWS_PROFILE unset, AWS config and credentials files at /dev/null, SSO still expired:

sub 15      USDC 307.991878469619934771 / cNGN 478661.280616
rate        1368.3155 (snapshot 2026-09-21T14:06:06.000Z)
valued      USDC $307.99 / cNGN $349.82 (cNGN 53.2%)
action      none
EXIT: 0

21 tests pass; ./scripts/verify.sh node green.

Note what that output is: check's own thresholds, price source and inventory definition reporting the verdict. A previous claim that "no rebalance is due" came from reading the balances directly after the command failed — which confirmed a reimplementation of the logic, not the logic itself.

🤖 Generated with Claude Code

robertleifke and others added 3 commits September 21, 2026 10:09
…as a fired one

`check` is the command meant to run unattended on a timer. It reads two
balances and a price and signs nothing, but cli.ts built a KMS account before
the switch regardless of command, so it needed AWS credentials it never used.
It died on an expired SSO session -- a scheduled monitor that fails whenever a
human's login lapses is not a monitor.

Three things, because the first alone would be reintroduced by the next
refactor:

1. createReadClient() builds chain reads only, and check() is typed to
   ReadClients, so tsc refuses any future edit that reaches for a signer there.

2. cli.ts takes injectable `readClients` / `signingClients`, and cli.test.ts
   asserts the signing factory is NEVER called for `check` or `quote` -- with a
   negative control asserting `deposit` DOES call it, so the first two cannot
   pass for the wrong reason. The regression this guards against is hoisting
   client creation above the switch again: that typechecks, leaves CI green, and
   breaks only the unattended path.

3. A failed check now alerts. Unattended, a crash into a log nobody reads is the
   same failure as an alert that reaches nobody, so a run that cannot complete
   posts "FAILED TO RUN ... Inventory is UNKNOWN, not healthy" to the same
   webhook a real finding would use, and still exits non-zero so a scheduler's
   OnFailure catches the case where the webhook is what broke.

Two further defects found while testing:

- cli.ts ran main() at module scope, so importing it executed the CLI and threw
  on loadConfig before a test could reach anything. Now guarded on being the
  entry point.
- `--alert` validated ALERT_WEBHOOK_URL only at the point of sending, which is
  after the healthy-verdict early return. A --alert run with no webhook
  configured therefore looked fine for as long as the inventory was fine, and
  would have failed for the first time on the run that finally had something to
  say. Validated up front now. Found by a test written against the old
  behaviour, which was right to fail.

Verified against the failure that exposed it: with AWS_PROFILE unset, the AWS
config and credentials files pointed at /dev/null and the SSO session still
expired, `pnpm rebalance check` runs and exits 0. Its own logic then reports
sub 15 at cNGN 53.2%, action none.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two things found while confirming the entry-point guard fires from every real
invocation path, as asked.

**The guard did not survive a symlink.** `import.meta.url` is always the real
path; `process.argv[1]` is whatever was typed. Invoked through a symlink the two
differ, the guard does not fire, and the CLI prints nothing and exits 0:

    $ ln -s .../dist/cli.js /tmp/rebalance
    $ node /tmp/rebalance check
    $                                  # no output, exit 0

Silent success is the worst failure available to a scheduled job, and it is
exactly the shape scheduling would produce: `node_modules/.bin/*` is a symlink,
so adding a `bin` entry creates it, as does a symlinked unit path. argv[1] now
resolves through realpath, and `isEntryPoint` is exported and tested against a
real symlink. Verified across all four paths -- pnpm/tsx, compiled dist, an
absolute path from another cwd, and through a symlink.

There is no container for this package: no Dockerfile, no bin entry, no image.
The built path that exists is `pnpm build` -> `dist/cli.js`, and it is covered.

**--heartbeat, so silence cannot pass for healthy.** A healthy `--alert` run
posts nothing, which is indistinguishable from a timer that stopped firing, a
host that went away, or credentials that lapsed. `--heartbeat` posts on every
run whatever the verdict, carrying the balances and share with it, so the
message is evidence rather than a bare ping. Intended shape: schedule `--alert`
often (pages only on a finding) and `--heartbeat` rarely (proves the checker is
alive). Stateless, so no last-run file to go stale.

A failed run still pages under either flag.

26 tests. The symlink test initially failed on a fixture of my own making --
macOS tmpdir() sits under a symlinked /var, so the hand-built module URL was not
the real path that Node would give.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Anything fired by hand into a shared ops channel must say so. Without it a
drill is indistinguishable from a real finding, and someone spends their
afternoon investigating a venue that is fine -- which is exactly what happened
on 2026-09-21, when a forced-failure drill reached the channel unmarked.

`--test` prefixes "[TEST] " to whatever is posted, on both the healthy/heartbeat
path and the forced-failure path. Two tests pin both.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@robertleifke
robertleifke merged commit 5a155b5 into main Sep 22, 2026
1 check passed
@robertleifke
robertleifke deleted the fix/check-needs-no-signer branch September 22, 2026 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant