Skip to content

feat(opencode): killswitch — block requests when quota drops below threshold#35

Open
iceteaSA wants to merge 6 commits into
cortexkit:mainfrom
iceteaSA:feat/killswitch
Open

feat(opencode): killswitch — block requests when quota drops below threshold#35
iceteaSA wants to merge 6 commits into
cortexkit:mainfrom
iceteaSA:feat/killswitch

Conversation

@iceteaSA
Copy link
Copy Markdown
Contributor

@iceteaSA iceteaSA commented May 21, 2026

Rebased onto upstream/main (v1.5.0) now that #34 (QuotaManager) is merged — this branch no longer depends on feat/quota-manager.

Per-account request blocking when remaining quota drops below configurable thresholds. Returns a synthetic 429 only when the request genuinely cannot be routed anywhere.

Features:

  • Per-account threshold overrides (5h and 7d windows), persisted to the sidecar killswitch block.
  • Eager quota refresh on the first request, with a fresh re-read before evaluation so the killswitch fires correctly even on the very first request (the pre-refresh snapshot is null).
  • Skip-main routing when main is killed (try surviving fallbacks). A synthetic 429 is returned only when main is killed and every fallback is killed or the request body is non-replayable (so it cannot be routed to a fallback). A request that is merely below the soft routing threshold still falls through to main.
  • killswitchPassesPolicy evaluates all present quota windows — a present below-threshold window still blocks even when the other window is missing (failClosed=false).
  • All killswitch / fallback decisions and the Retry-After computation read the fresh, token-aware QuotaManager cache, never the stale request-start storage snapshot.
  • Retry-After header set to the earliest quota reset across the unroutable accounts.
  • /claude-killswitch slash command (on / off / set / status) for runtime management.

Files:

  • packages/core/src/killswitch.ts — new
  • packages/core/src/accounts.ts — killswitch types and policy functions (shared DEFAULT_KILLSWITCH_THRESHOLDS)
  • packages/opencode/src/index.ts — killswitch fetch-gate + command registration
  • packages/opencode/src/tests/killswitch.test.ts + index.test.ts fetch-gate tests
  • README.md + packages/opencode/README.md — killswitch documentation

Summary by cubic

Adds a per-account killswitch and a unified quota cache in @cortexkit/anthropic-auth-core so OpenCode blocks low‑quota requests and returns 429 with Retry‑After when unroutable or when the quota API is backed off. Adds /claude-killswitch, enforces the killswitch across all routing paths, and keeps the sidebar accurate on background fallback storage updates and after async main refreshes.

  • New Features

    • Killswitch: per‑account thresholds (5h/7d), eager quota refresh with post‑refresh re‑read, skip main when killed, and synthetic 429 with Retry‑After (earliest reset + 60s) when no routable accounts remain or the body is non‑replayable; unified fallback selection via getRoutableFallbackAccounts; token‑aware reads for main and fallbacks; failClosedOnUnknownQuota respected when main quota is unknown; /claude-killswitch to view/toggle/set, persisted to the sidecar killswitch block; optional quota.refreshEveryNRequests.
    • Sidebar: auto‑refresh on background fallback storage changes via onFallbackStorageChanged (token refresh, quota update, or error) and preserve the active fallback after async main quota refreshes; refresh can happen without a request.
  • Bug Fixes

    • 429 only when main is killed and there are no routable fallbacks or the body is non‑replayable; routing thresholds no longer hard‑block.
    • Token‑aware fail‑closed read for main; fallback‑first routing and reactive fallback filters honor the killswitch using the manager cache.
    • Backed‑off quota API + fail‑closed returns 429 with accurate Retry‑After and clearer messaging; removed a shadowed request counter that broke every‑N refresh cadence; prevented sidebar clobbering of the active fallback and ensured sidebar updates without a request.

Written for commit 8c44a67. Summary will update on new commits.

Review in cubic

Greptile Summary

This PR introduces a per-account killswitch that hard-blocks requests when remaining quota drops below configurable thresholds, returning a synthetic 429 with Retry-After only when main is killed and no routable fallbacks remain (or the request body is non-replayable). It also fixes a sidebar clobber race where an async main-quota refresh could overwrite the active fallback account, and adds onFallbackStorageChanged so the sidebar refreshes on background quota updates.

  • Killswitch gate (packages/opencode/src/index.ts): eager quota refresh before evaluation, post-refresh re-read of mainQuota, unified getRoutableFallbackAccounts using getFallbackQuota for consistent token-aware quota reads across all routing paths, and a fail-closed/backoff 429 guard before the killswitch evaluation.
  • Policy logic (packages/core/src/accounts.ts): killswitchPassesPolicy iterates all windows with continue on missing entries so a present below-threshold window blocks even when the partner window is absent; killswitchRetryAfterSeconds returns the earliest reset time across all accounts + 60 s buffer.
  • /claude-killswitch command (packages/core/src/killswitch.ts): on/off/set/status with per-account overrides and set all shorthand, persisted to the sidecar config.

Confidence Score: 5/5

Safe to merge — the four issues flagged in earlier review rounds are all addressed in the current diff.

All four previously identified routing bugs — stale mainQuota bypassing first-request evaluation, killswitchPassesPolicy early-returning on the first missing window, inconsistent quota sources between allFallbacksKilled and survivingFallbacks, and the reactive fallback filter using stale a.quota — are fixed in this diff. Test coverage is comprehensive across unit and integration scenarios.

No files require special attention.

Important Files Changed

Filename Overview
packages/core/src/accounts.ts Adds KillswitchConfig type, killswitchPassesPolicy (correctly uses continue for missing windows to evaluate all present windows), killswitchRetryAfterSeconds, and onFallbackStorageChanged callback. Policy logic is sound.
packages/core/src/killswitch.ts New file implementing /claude-killswitch command parsing and execution. The set all branch assigns the same thresholds object to both updated.main and all accounts entries (noted in a previous outside-diff comment), but this is harmless in practice since no caller mutates the returned config in-place before JSON serialization.
packages/opencode/src/index.ts Core killswitch gate: fail-closed backoff check, eager quota refresh with post-refresh mainQuota re-read, getRoutableFallbackAccounts (unified killswitch-aware fallback selection), and synthetic 429 generation. Previously identified issues (stale mainQuota, inconsistent quota sources, reactive filter stale quota) are all addressed in this diff.
packages/opencode/src/tests/killswitch.test.ts New test file with thorough unit coverage for parseKillswitchCommandAction, killswitchPassesPolicy (including partial-window/failClosed=false cases), killswitchRetryAfterSeconds, and setKillswitchPersistent.
packages/opencode/src/tests/index.test.ts Adds integration tests for the killswitch fetch gate (non-replayable 429, routing-only threshold no-op, fail-closed unknown quota, sidebar correction after killswitch reroute) and sidebar regression tests.
packages/opencode/src/tests/accounts.test.ts Adds test verifying onFallbackStorageChanged fires when refreshQuotaForDueAccounts persists a storage change, covering the new callback mechanism.

Comments Outside Diff (1)

  1. packages/core/src/killswitch.ts, line 340-355 (link)

    P2 Shared thresholds object reference when set all is used

    When entry.account === 'all', the same thresholds object is assigned to updated.main and to every entry in accounts[id]. All these properties point to the same object in memory. Callers that receive updatedConfig and subsequently mutate threshold values (e.g., patching a single window in-place) would silently affect every account and the main entry simultaneously. Constructing a fresh object for each assignment ({ ...thresholds }) avoids the aliasing.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Reviews (12): Last reviewed commit: "Merge branch 'fix/sidebar-fallback-quota..." | Re-trigger Greptile

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 10 files

Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Fix all with cubic | Re-trigger cubic

Comment thread packages/opencode/src/index.ts Outdated
Comment thread packages/core/src/killswitch.ts Outdated
@iceteaSA iceteaSA force-pushed the feat/killswitch branch 4 times, most recently from d89dd9d to f667649 Compare May 22, 2026 17:08
@iceteaSA iceteaSA force-pushed the feat/killswitch branch 8 times, most recently from 0054553 to d125adb Compare June 3, 2026 18:21
Comment thread packages/opencode/src/index.ts Outdated
Comment thread packages/opencode/src/index.ts Outdated
Comment thread packages/core/src/accounts.ts
@iceteaSA iceteaSA force-pushed the feat/killswitch branch 2 times, most recently from 9092526 to 37961f1 Compare June 3, 2026 19:21
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented Jun 3, 2026

Want your agent to iterate on Greptile's feedback? Try greploops.

Comment thread packages/opencode/src/index.ts
@iceteaSA iceteaSA force-pushed the feat/killswitch branch 5 times, most recently from 56730ff to 20f6330 Compare June 3, 2026 20:52
…reshold

Self-review fixes folded in:
- Token-aware fail-closed read: const mainQuota = quotaManager.getMain(auth.access)
  so a previous main account's cached quota can't satisfy the fail-closed check
  or feed the killswitch eval after a main-account switch.
- Removed a stray inner 'let sessionRequestCount = 0' + unconditional increment
  that shadowed the process-scoped counter, which had left the active-route
  fallback every-N refresh reading a never-incremented counter.
@iceteaSA
Copy link
Copy Markdown
Contributor Author

iceteaSA commented Jun 4, 2026

Updated: this branch now carries the sidebar quota fix (#57), merged in to keep the sidebar-touching branches consistent. #57's commits appear in this PR's diff until #57 merges to main, after which the diff reduces to this PR's own changes. Merge order: #57 first.

iceteaSA added 2 commits June 4, 2026 08:22
…nged

The callback fires on any persisted fallback storage change (token refresh,
quota update, or error recording), not only quota fetches. Rename per review
feedback for an accurate API name.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant