fix(ai): only record a service tier the provider actually served - #4818
Conversation
The requested tier leaked into $ai_model_parameters wherever no response value overwrote it: openai error paths and the whole LangChain callback. Cost processing prices from that key, so tokens could price at a tier the provider never confirmed. service_tier leaves the request-side allowlist, and the LangChain callback merges the served tier from generationInfo. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Responses adapter stores it there for streaming and non-streaming results, and the middleware path carries no generationInfo at all. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
arnohillen
left a comment
There was a problem hiding this comment.
agent drafted, arno reviewed
Verified at 5d776db9. The allowlist removal in packages/ai/src/utils.ts and the served-tier merge at utils.ts:145-146 do what the title says; the three new served-tier tests in packages/ai/tests/callbacks.test.ts pin generationInfo, response_metadata, and the none-served case.
One minor note, not blocking: the same allowlist removal also stops the requested service_tier ('auto' / 'standard_only') reaching $ai_model_parameters on Anthropic events (packages/ai/src/anthropic/index.ts:248, 268, 306, 331 call getModelParams(body) with no served tier to merge). That is consistent with the PR intent, but the changeset and body name only the OpenAI error paths and LangChain, so an Anthropic integrator reading the changelog will not expect it.
The llmOutput.service_tier fallback at callbacks.ts:725 has no test; every new test passes llmOutput: {}.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
agent drafted, arno reviewed
Withdrawn; leaving review to the client-libraries team.
Cost processing prices only from this property: its writers assert served values, unlike $ai_model_parameters.service_tier, which released SDKs populated from the request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
langchain-js reports the served tier in response_metadata or generationInfo; its llmOutput carries token usage only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Problem
service_tierinside$ai_model_parametersis what PostHog cost processing prices from (feat(aio): price ai generations by their served service tier posthog#94200), so it must carry the tier the provider served — a requested tier can be refused.getModelParamscaptured the requested tier from the call's own params, and the served value only overwrote it on the OpenAI success paths. Two surfaces leaked the requested tier with billable usage attached: OpenAI error captures (partial usage from an errored stream, no response value to overwrite), and every LangChain-emitted generation (the callback captured invocation params at run start and never merged the response's tier).Changes
service_tierleavesgetModelParams' request-side allowlist: the key now appears only when a response supplied it, so its presence is its provenance. This closes both leaks at once and matches the Python SDK (feat(ai): capture the served service tier into model parameters posthog-python#920). It also stops Anthropic events recording the requested'auto'/'standard_only'— Anthropic capture passes no served value, so those events now carry no tier at all.response_metadata(the Responses adapter, streaming and non-streaming — also the only carrier on the middleware path) andgenerationInfo(the Completions adapter, streamed). Those are the only two places langchain-js reports it;llmOutputthere carries token usage only.Release info Sub-libraries affected
Libraries affected
Checklist