feat(pricing): add user price overrides for models#560
Open
ozymandiashh wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #390.
What this adds
A user-defined per-model price override, so you can give a model a price that LiteLLM does not ship yet (for example DeepSeek V4 currently reports $0), or override an existing price, without waiting for an upstream pricing PR. Overrides take the highest precedence in cost computation.
# rates are USD per 1,000,000 tokens (the unit you read on pricing pages) codeburn price-override deepseek-v4 --input 0.27 --output 1.10 codeburn price-override deepseek-v4 --input 0.27 --output 1.10 --cache-read 0.07 --cache-creation 0.27 codeburn price-override --list codeburn price-override --remove deepseek-v4Why
LiteLLM is the source of truth for pricing, but it lags new or promotional prices, and some models (or private/internal model ids) never get a public entry. When a model has no entry it prices to $0, which makes its usage invisible in reports. This lets you set a price locally and keep an eye on cost in codeburn until the official entry lands, then drop the override. It also covers business/negotiated rates that differ from list price.
How it works
priceOverridesmap in~/.config/codeburn/config.json, keyed by model, withinput/output/ optionalcacheRead/cacheCreationrates in USD per 1,000,000 tokens.price-override [model]command mirroringmodel-alias, with--input/--output/--cache-read/--cache-creation(per 1M), plus--listand--remove. Setting requires at least--inputand--output, and every provided rate must be a finite number >= 0 (otherwise it exits non-zero with a clear message). The unit is documented in the command help.setPriceOverridesconverts the per-1M rates to the internal per-tokenModelCosts(dividing by 1e6, guarded by the existingsafePerTokenRate).getModelCostsconsults the overrides at each resolution tier it already uses for the snapshot, so an override wins wherever a snapshot entry of the same key would have matched.Two correctness details
getModelCostsresolves a model by exact key, then longest-prefix, then case-insensitive. The override check is interleaved at each of those tiers (override-exact, snapshot-exact, override-prefix, snapshot-prefix, override-ci, snapshot-ci), so an override is never silently bypassed by a fuzzier snapshot match. A more-specific exact entry still wins, so overridinggpt-5does not hijack agpt-5-minicall that has its own exact entry.localModelSavingshash). Changing a price override now also changes that hash (savings and overrides are combined into the value passed toensureCacheHydrated), so cached historical days recompute against the new price instead of showing the stale one. With no overrides configured, the hash is byte-for-byte the previous savings-only value, so nothing changes for existing users.Default behavior is unchanged
With no
priceOverridesconfigured, pricing resolution and the cache-invalidation hash are byte-for-byte identical to before. The feature is purely additive.Verification
tests/models.test.ts): an override prices a model that has no snapshot entry to a non-zero cost; an override wins over both a snapshot price and a configured alias for the same name; the per-1M to per-token conversion is exact ($1.00 / 1Mover 1,000,000 input tokens prices to exactly$1.00); cache rates default sanely when omitted; the override applies case-insensitively and by prefix but does not shadow a more-specific exact entry; two different override configs produce different cache-invalidation hashes while empty overrides leave the savings-only hash unchanged.tests/cli-price-override.test.ts, isolated HOME): set writesconfig.priceOverrides;--listshows it;--removedeletes it; an invalid rate is rejected with a non-zero exit.npx vitest run tests/models.test.ts tests/cli-price-override.test.tspasses (118 tests).npm testis green except the pre-existing locale-dependenttests/overview.test.tsassertion.npx tsc --noEmitis clean. The offline build succeeds.Files
src/config.ts(addpriceOverridestoConfig)src/models.ts(setPriceOverrides, override-awaregetModelCosts,getPriceOverridesConfigHash)src/usage-aggregator.ts(combine the savings and overrides hashes for cache invalidation)src/main.ts(load overrides at startup + theprice-overridecommand)tests/models.test.ts+tests/cli-price-override.test.ts(new)