fix(mlx): reject image input when the runner has no vision processor

[zhast]

Bug

A chat request with an image against a runner whose vision processor is None succeeds silently and returns a hallucinated answer.

Reproduced with a solid red 8×8 PNG on a 4-node pipeline deployment: HTTP 200, content "Black". The runner log shows the prompt it actually ran ended in <|begin_of_image|><|image|><|end_of_image|> — one bare placeholder token, no vision embeddings.

Three things line up to make it silent:

1. utils_mlx.load_mlx_items catches any vision-processor load failure, logs it, and sets vision_processor = None (utils_mlx.py:221-227).
2. ExoBatchGenerator.submit only prepares images if self.vision_processor is not None; otherwise they are dropped (batch_generate.py:137), and a prepare_vision exception also "falls back to text-only".
3. The chat template still renders the image placeholder, so the model sees an image token with nothing behind it.

Nothing in the API gates on a vision capability, so the client has no way to know.

Fix

• check_vision_support() raises a request-level UnsupportedRequestError(ValueError) when task_params.images is non-empty and there is no processor; submit() calls it first, and a prepare_vision failure on a request with images now raises the same error instead of silently degrading. The text-only fallback is kept for requests without images.

• BatchGenerator.step() catches UnsupportedRequestError next to PrefillCancelled: _send_error to the client, then emit a terminal FinishedResponse for the task on both return paths of step(). The runner registers a task in active_tasks before step() runs, so without that terminal result the rejected task would stay active with no generator behind it and the runner loop would spin (I hit exactly that with a first version: four runners stuck in RunnerRunning at ~85% CPU with nothing in flight). The generic except Exception still re-raises, so runner-fatal errors behave as before.

• collect_chat_response() (non-streaming) now yields the same {"error": …} JSON object the SSE path emits instead of raising. The endpoint wraps it in a StreamingResponse whose 200 status is already committed, so the old raise ValueError left non-streaming clients with an empty body for any ErrorChunk that arrived before the first token — a pre-existing gap this bug made visible. Covered by test_collect_chat_response_error.py; the existing test_chat_completions_stream.py still passes.

The condition is identical on every rank (same images, same None), so all pipeline ranks reject and retire the task together and none is left waiting in a collective.

Tests

test_vision_gate.py — three pure tests, no model download: images + no processor raises with a clear message; text-only passes; the error is a ValueError so existing handlers keep working. ruff check, ruff format --check, basedpyright clean on the changed files.

Scope

Covers the default BatchGenerator path. SequentialGenerator builds its generator separately (batch_generator.py:300) and is not changed here.

🤖 Generated with Claude Code

[zhast]

Verified on a live 4-node pipeline deployment (both commits)

Same red 8×8 PNG that previously returned HTTP 200 → "Black":

Streaming — explicit error, then the stream terminates normally:

data: {"error":{"message":"1 image(s) were provided, but this model instance has no vision processor loaded; image input is not supported here.","type":"InternalServerError","param":null,"code":500}}
data: [DONE]

Runner health after two rejected requests: all four runners back to RunnerReady within 5 s, runner process at 0.0% CPU. (The first version of the fix, without the terminal FinishedResponse, left all four in RunnerRunning at ~85% CPU with nothing in flight — the retirement commit is what fixed that.) A text-only correctness suite run immediately afterwards: 4/4 correct at 21.5 tok/s, so text serving is unaffected.

One adjacent gap, pre-existing: the non-streaming path returns HTTP 200 with an empty body for the same request — the adapter breaks out of its loop on ErrorChunk with error_message set, but the caller never turns that into an error response. That is a separate, older behaviour (any ErrorChunk before the first token hits it); looking at whether it is small enough to include here.

[zhast]

Live verification of all three commits (4-node pipeline deployment)

Same red 8×8 PNG that originally returned HTTP 200 → "Black".

Non-streaming (previously an empty HTTP 200 body):

{"error":{"message":"1 image(s) were provided, but this model instance has no vision processor loaded; image input is not supported here.","type":"InternalServerError","code":500}}

Streaming:

data: {"error":{"message":"1 image(s) were provided, but this model instance has no vision processor loaded; image input is not supported here.","type":"InternalServerError","param":null,"code":500}}
data: [DONE]

Runner health: all four runners RunnerReady 5 s after the rejected requests, runner process at 0.0% CPU. A text-only correctness suite immediately afterwards: 4/4 correct at 21.7 tok/s.

Checks on the branch: ruff check, ruff format --check, basedpyright clean on every changed file; 9 tests pass across test_vision_gate.py, test_collect_chat_response_error.py, and the existing test_chat_completions_stream.py.