Repository navigation
Expect resolved_model in hosted platform e2e test responses - #3031
Conversation
#2999 added an optional top-level `resolved_model` field to HTTP inference responses. The hosted platform e2e tests assert exact response key sets, so 20 tests failed on staging once the new build was deployed there. The field is optional by design: some models do not report it, and staging and production are deployed independently, so the suite must pass against servers both with and without it. `required_response_keys()` in conftest strips the optional keys before the exact-set comparison and checks the shape of `resolved_model` when it is present. Unknown extra keys still fail the tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
🤖 Claude review started at commit New commits are not auto-reviewed. Add the |
|
👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely. 🚦 This PR is marked Ready for review, so automated Claude review will run — and every pass spends real credits. Warning 💸 The Claude reviewer bills in credits, not vibesAutomated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared. Draft when unsure, Ready when you mean it:
However you get there, arrive prepared:
Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.
|
|
Skills: review-http-api-server, review-topic-test-hygiene Reviewed the test-only change that makes the hosted-platform e2e assertions tolerant of the optional
No production behavior changes, so no changelog/version coordination is needed. Reviewed at HEAD: 19827b0 |
|
😎 PR passes the vibe-check and trust-me-bro verification. |
…olerating it Drop the optional-key helper. The 10 response assertions that receive `resolved_model` since #2999 now list it in the expected key set, so the suite also fails if the field disappears. Visualisation-only responses do not carry the field and stay unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
resolved_model field in hosted platform e2e testsresolved_model in hosted platform e2e test responses
What
The hosted inference e2e suite fails on staging with 20 failures (run 35393354896). Every failure is the same assertion: the response carries one extra key,
resolved_model.#2999 added
resolved_modelas a top-level field of HTTP inference responses. The tests intests/inference/hosted_platform_testscompare the exact key set of the response.Fix
Add
"resolved_model"to the expected key set in the 10 assertions that receive it: 4 intest_core_models.py(OCR, EasyOCR, CLIP, YOLO-World) and 6 intest_roboflow_models.py(object detection, instance segmentation, classification; plain and with-visualisation-and-payload). Net diff: 10 added lines.Visualisation-only responses do not carry the field (those tests passed in the failing run) and are unchanged.
The assertion is strict on purpose: the suite also fails if the field disappears.
Note for production runs
Production serverless is on
1.6.0-post1, which does not return the field. The production e2e workflow will fail frommainuntil 1.6.1 is deployed there. It is dispatch-only and is run after production deploys, so this is expected ordering, not a defect.Verification
pytest --collect-onlyon both files: 82 tests collected.blackclean.HOSTED_PLATFORM_TESTS_API_KEYis not available on my machine. The staging e2e workflow needs to be dispatched on this branch to confirm.🤖 Generated with Claude Code