[leeclemnet]

What does this PR do?

Adds resolved_model to structured inference responses so clients can identify the model that served each result. With USE_INFERENCE_MODELS=true, it reports the canonical model ID, package ID, backend, and quantization from the serving instance. Legacy models report their loaded model ID, with package details optional. Existing null fields and binary response formats are preserved.

Related Issue(s): n/a

Type of Change

• New feature (non-breaking change that adds functionality)

Testing

• Unit tests (passed in gpu container)
• Local staging server test on T4 VM, SDK v0 and v1 API

{
  "inference_id": "7885cedc-2b7b-443b-aa9f-dd4dbb7ae47f",
  "time": 0.08670124830678105,
  "image": {
    "width": 2048,
    "height": 1371
  },
  "resolved_model": {
    "model_id": "lee-sandbox/chess-pieces-fmhpz-131-rfdetr-small-t1",
    "model_package_id": "c70f32369a54d61e06ef4e6b56c82524",
    "backend": "trt",
    "quantization": "fp16"
  },
  "predictions": [
    {
      "x": 1720.0,
      "y": 1007.0,
      "width": 144.0,
      "height": 180.0,
      "confidence": 0.9199306964874268,
      "class": "black-rook",
      "class_id": 5,
      "detection_id": "e51c2831-cc93-461e-b185-40acad81bbd7"
    },
...

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code where necessary, particularly in hard-to-understand areas
• My changes generate no new warnings or errors
• I have updated the documentation accordingly (if applicable)

Additional Context

• First PR in the stack; merge before feat(api) add HTTP model package selection with isolated LRU cache entries #3000.
• Includes inference-models metadata for fresh loads, cached loads, and successful package fallbacks.
• Rollout for package metadata: release the included inference-models changes, bump the server dependency pin, rebuild the server image, then enable USE_INFERENCE_MODELS.

[github-actions]

👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely.

🚦 This PR is marked Ready for review, so automated Claude review will run — and every pass spends real credits.

Warning

💸 The Claude reviewer bills in credits, not vibes

Automated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared.

Draft when unsure, Ready when you mean it:

• 🌱 Not sure the PR is in good shape yet? Keep it (or set it back) as a draft — drafts pause review, so you can push and iterate without burning credits on a moving target.
• 💪 Feel strong about the contents? Mark it Ready for review and the reviewer will take a look.

However you get there, arrive prepared:

• 🧱 Bring a SOLID, thorough PR. Point your local agent at our skills/ to tune it to our guidelines first — or, if you are one of those fabled carbon-based contributors, read them yourself. A half-baked diff costs exactly the same to review as a finished one.
• ✅ Resolve every comment before you re-request review. Re-requesting with threads still open means paying twice for the same conversation.
• 🔁 Do not use CI review as an inner loop for a local agent. The reviewer is not a step-by-step debugger — do the unfolding locally and arrive with the answer, not the search.
• 🙋 If something looks off, ask a human. One question to a maintainer is cheaper and faster than three rounds of agent re-review chasing a misread.

Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.

• Prefer to skip automated review entirely? Add the skip-claude-review label.

[github-actions]

🤖 Claude review started at commit 01f0d929934353d17f0952ae735672065246e889.

New commits are not auto-reviewed. Add the claude-review label to request a re-review — the label is consumed when the review starts, so just add it again next time.

[github-actions]

@leeclemnet — one question needs your answer before this can advance.

⏳ This PR is on hold pending your answers — the review will not advance to sign-off until the IMPORTANT question below is answered.

IMPORTANT question

Is it intended that resolved_model is absent for models served by legacy classes when USE_INFERENCE_MODELS=True (the non-Windows default)?

Model.resolved_model (inference/core/models/base.py:113-120) branches on the global USE_INFERENCE_MODELS flag rather than on whether this instance is backed by an inference_models _model:

@property
def resolved_model(self) -> Any:
    if not USE_INFERENCE_MODELS:
        model_id = getattr(self, "endpoint", None) or getattr(self, "model_id", None)
        return ResolvedModel(model_id=model_id) if model_id is not None else None
    return getattr(getattr(self, "_model", None), "resolved_model", None)

Under the default flag-on config, many models are still served by legacy Model subclasses that never set _model, so this returns None and no metadata is attached. Concretely, ("object-detection", "yolo-world") is registered unconditionally to the legacy YOLOWorld class (inference/models/utils.py:760) and is not remapped to an adapter in the flag-on block (utils.py:806+), because "yolo-world".startswith("yolov") is False. So the self._attach_resolved_model_metadata(response) call this PR added to inference/models/yolo_world/yolo_world.py is a no-op in the default config — yolo-world responses report no resolved_model at all, not even the model_id the flag-off path would supply. The same applies to any other architecture that stays on a legacy class under flag-on (e.g. legacy grounding-dino / object-detection / classification variants not in the remap list).

• If this omission is intended (metadata only for inference_models-backed models under flag-on), no change is needed — but then the added attach calls in the legacy-only modules are dead in the default config, and the PR description's "Legacy models report their loaded model ID" holds only when the flag is off; worth a one-line note.
• If it is not intended, the property should fall back to endpoint/model_id whenever _model is absent (gate on _model presence, not the global flag), and a test should cover a legacy-served model under USE_INFERENCE_MODELS=True (the current tests only exercise the legacy path with the flag disabled).

Why this matters: a "not intended" answer confirms a real defect in the default production config affecting every legacy-served model; an "intended" answer resolves it with only a doc/wording tweak.

Please look at inference/core/models/base.py:113-120, inference/models/utils.py:756-760, and the flag-on remap block starting at utils.py:806. Unanswered questions may keep this PR out of a release.

---

New commits are not auto-reviewed — add the claude-review label to request a re-review (the label is consumed when the review starts, so just add it again next time).

Reviewed at HEAD: 01f0d92

[github-actions]

Review summary

Skills: review-http-api-server, review-inference-models-pkg, review-legacy-models-registries, review-topic-backward-compat-and-versioning, review-topic-external-contract-and-silent-fallback, review-core-infra, review-topic-test-hygiene

No confirmed blocking finding surfaced, but the PR is not merge-ready: one IMPORTANT question is open (see the on-hold comment above) and, until it is answered, the review will not sign off — its answer could itself turn into a defect.

What I verified positively:

• The inference_models auto-load cache path is well-guarded: a missing backend_type or missing/blank canonical_model_id invalidates the entry and forces a provider refresh (core.py:2290-2292, _verified_auto_cache_package_dir at core.py:2190-2197), so cast(str, cache_entry.canonical_model_id) at core.py:2442 is safe. Tests cover the fresh/cached/fallback cases.
• Backward-compat of the flag-on rollout: when the pinned inference_models build predates this change, getattr(getattr(self, "_model", None), "resolved_model", None) returns None and nothing is attached — matches the staged-rollout plan in the description.
• Serialization: HTTP responses go through orjson_response(..., exclude_none=True), so resolved_model (and its null sub-fields) are dropped when absent; the added null-compatibility tests guard the classes that switched base to InferenceResponse.
• inference_models/docs/changelog.md has the required ## Unreleased entry (§5).

Minor doubts (non-blocking)

• _is_response_dc_to_dict serializes r.resolved_model.model_dump() without exclude_none, so the instance-segmentation fast path emits model_package_id/backend/quantization: null where the pydantic+orjson path would drop them — a small cross-endpoint inconsistency.
• The changelog calls ResolvedModelMetadata "public", but it is not re-exported from inference_models/inference_models/__init__.py (only defined in entities.py).

Reviewed at HEAD: 01f0d92

[leeclemnet]

Is it intended that resolved_model is absent for models served by legacy classes when USE_INFERENCE_MODELS=True (the non-Windows default)?

Not intended. Fixed in f8c8c6e

[github-actions]

🤖 Claude review started at commit f8c8c6e4b94f734d1056e697e10d0884a2f5e539.

New commits are not auto-reviewed. Add the claude-review label to request a re-review — the label is consumed when the review starts, so just add it again next time.

[github-actions]

😎 PR passes the vibe-check and trust-me-bro verification.

[github-actions]

🤖 Claude review started at commit 523bb936cfa2e5973e6ff80121b1493ac91c516a.

New commits are not auto-reviewed. Add the claude-review label to request a re-review — the label is consumed when the review starts, so just add it again next time.

[github-actions]

😎 PR passes the vibe-check and trust-me-bro verification.