feat(usage): measure every turn for the ADE router, and make model choice work for every provider - #1300
Merged
Conversation
…oice work for every provider Add a machine-local per-turn usage ledger (requested and served model, tokens, list-price and billed cost, quota burn per window) and a consent-gated daily usage research report that the account directory Worker stores once per install and day in D1, with per-address, fleet, storage, and retention limits. Fix model and effort selection from the UI for Grok, Copilot, Qwen, Kimi, Cursor, OpenCode, Pi, Droid, Claude, and Codex, and show a notice when a provider answers with a different model. Refs ADE-159. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
|
Important Review skippedToo many files! This PR contains 197 files, which is 97 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configurationConfiguration used: Repository: arul28/ADE/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: ⛔ Files ignored due to path filters (8)
📒 Files selected for processing (197)
You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_a34210fe-baa9-499f-8d3f-5e381b695f21) |
… -0 for a UTC offset gitleaks read `modelToken: "claude-opus-4.6"` in the ACP coordinator test as a generic API key; the test helper now takes `model`. Under TZ=UTC the offset tests built `-0` from `-getTimezoneOffset()`, while the report sends `0`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ADE cannot yet decide which model should run a turn, because it does not record what each turn cost, which model answered, or how fast each subscription window burns. Also, several providers ignored or lost the model and effort that the user picked in the UI.
Cause
-m, which fails for some ids, and a resumed session restored the old model.falsefor options that the user never touched. A Cloud follow-up refused to send when the catalog failed to load.Change and boundary
Measurement: the groundwork for the ADE router (Linear ADE-159)
<adeHome>/usage/turns-YYYY-MM.jsonl). Each row holds the requested model, the served model, tokens,apiEquivalentUsd(list price), the provider's billed cost where known, the plan units, and the compaction count. A quota ledger records window readings.ADE_CURSOR_DASHBOARD_USAGE) and Factory credit reading.ade usage turnsand the actionusage.getTurnUsageSummary.POST /usage-research/dailyin D1 tableusage_research_daily(migration0012).ADE_USAGE_RESEARCH=0turns it off. The Settings text now tells the user about the daily summary.installIdcomes from a local salt, so it cannot be linked to PostHog.accountRefis a salted hash.Model and effort routing from the UI
session/set_config_option, and CLI versions before 1.0.40 keep--reasoning-effort. Qwen sends only the ids it offers, anddefaulton a clear. Kimi sends the effort. A failed effort set is fatal only for Qwen.chat/cursorModelSelection.ts.~/.piglobals, and one list of thinking levels.servedModelMismatch.ts) for all providers. The ledger, the report, and the "answered with X" warning all use it.donefields, and replay keepscontextTokens.Deliberately not changed
docs/logging.md).Deploy note: merging deploys the account directory Worker and applies D1 migration
0012_usage_research.sql.Verification
/quality: a capped run with one fix round and one short check, as the owner asked. There were 55 findings: 0 Blocker, 1 High, 19 Medium, 35 Low. The run applied 50 and rejected 5 after I checked them against the code. The gate is empty.@factory/droid-sdk/node,three, and others. CLI typecheck: 6 errors, the same as the baseline.apps/account-directory:npm run typecheckis clean, andnpm testpasses 215 of 215 after the rebase.threeis missing from the local install.cli.test.ts,tuiClient): 1,950 tests pass. 2 fail locally because of a stale@factory/droid-sdkinstall. The same 2 fail at the base commit.validate-docspasses (275 files).xcrun swiftc -parsepasses on the 5 changed Swift files.Authored with Claude Opus 5.5 via Claude Code.
🤖 Generated with Claude Code
Note
High Risk
New unauthenticated writes land in the same D1 database as sign-in, and broad provider routing changes can alter which model runs or what users see on failure paths.
Overview
Adds per-turn usage measurement on the machine (JSONL ledger, quota burn rates,
ade usage turns/usage.getTurnUsageSummary) and a daily usage-research upload from desktop/CLI to the account-directory Worker atPOST /usage-research/daily, gated by product-analytics consent andADE_USAGE_RESEARCH=0.The Worker side stores reports in D1 (
0012_usage_research.sql) with strict envelopes, per-IP and fleet write caps, a storage ceiling with compare-and-swap totals, retention sweeps, and sharedsinkUtilsextracted from diagnostics. Deploy config/docs/release inventory pick up the newUSAGE_RESEARCH_*vars.Model and effort routing is tightened across providers (Grok/Qwen/Kimi via ACP config, Cursor selection split and Cloud follow-up behavior, Pi read-only globals and thinking tiers from runtime, Droid/Copilot/OpenCode edge cases, unified served-model mismatch handling). CLI TUI token stats now prefer exact context occupancy like desktop.
There is still no automatic router—this PR is measurement plus making the picked model/effort stick.
Reviewed by Cursor Bugbot for commit 0bfc874. Configure here.