server: stop rejecting requests over options the client always sends - #369
Closed
CryptVenture wants to merge 1 commit into
Closed
server: stop rejecting requests over options the client always sends#369CryptVenture wants to merge 1 commit into
CryptVenture wants to merge 1 commit into
Conversation
A model whose spec omits `language` returned HTTP 500 for every request from
the WebUI, because build_openai_transcription_request wrote
request.options["language"] whenever the body carried the key -- which clients
do unconditionally, empty or not -- and spec-backed models validate their
request options strictly.
Reproduced against a running server on this branch's binary:
POST /v1/audio/transcriptions
{"model":"parakeet-tdt","audio":"...wav","language":"","text":"","options":{}}
-> {"error":{"message":"unknown Parakeet TDT request option: language"}}
Parakeet-TDT could not transcribe at all from the WebUI. AudioSR, ControlFoley,
MiDashengLM-Gen, PersonaPlex and HeartMuLa fail the same way through
/v1/tasks/run, which folds a top-level `language` into the option map.
Resolve the model's acceptance of `language` once at registration, next to the
existing accepts_reference_text flag and for the same reason (resolving per
request re-reads the embedded spec on the request thread), and forward the
field only where the contract takes it. A `language` the caller placed inside
`options` explicitly is left alone and still produces the strict rejection,
which is the correct signal there. The language continues to reach every model
through text_input, so nothing loses the feature -- qwen3_forced_aligner
requires it from there, so gating the input client-side would have broken it.
Three further server fixes in the same area:
Return the transcript detail fields from /v1/audio/transcriptions. The route
returned only {text, timing} while task_result_json already serialised
segments, speaker turns and word timings, so every ASR model was routed away
from the one endpoint that would return its timing data. The emitter is now
shared by both routes, and both carry the sample rate the spans are counted
in, without which a client cannot turn them into timestamps. Parakeet turns
out to have been producing word timings all along.
Prove the requested backend exists before binding the port. config.backend
defaults to Cuda and nothing validated it, so a machine without CUDA served
the UI, the model list and the package installer, and failed only when the
user pressed Run. It now fails at startup with the engine's own message:
audiocpp_server failed: CUDA backend requested but it is not registered in
this build (available: MTL:0 "Apple M4 Max" [GPU], BLAS:0 "Accelerate"
[ACCEL], CPU:0 "Apple M4 Max" [CPU])
Bound the bulk unload routes. Both called busy.acquire(0, ...), which
busy_guard.h documents as an unbounded wait, on exactly the wedged run an
operator calls these routes to clear; they now use the same bounded helper as
every other unload path. They also skip the ui_management gate that
/v1/models/unload enforces, and read models_ and model_index_ without holding
models_mutex_ -- handle_unload_all_models ranges over the vector that
/v1/models/load appends to, so a concurrent load could reallocate underneath
it.
Validation:
cmake --build build/macos-metal-tests --target audiocpp_server -j 12
ctest -j 6 # 53/53 pass
Backend tested: Metal (Apple M4 Max). Verified end to end against a live
server: Parakeet transcribes the exact UI body and returns 9 word timings with
sample_rate 24000; AudioSR accepts a request carrying language="en" that the
pre-change binary rejected; --backend cuda now exits 1 at startup on this
machine while --backend metal starts normally.
Known limitations: the CUDA and Vulkan startup probes were not exercised, only
the Metal and missing-CUDA paths. The probe initialises and immediately frees
one backend context, which is new work at startup on every deployment.
Owner
|
As mentioned in another PR, please split into separate PRs: language-option contract, ASR detail response, backend startup probe, unload locking so I can test each of them individually. |
This was referenced Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four server fixes in the request path. The first one is a hard blocker: a model whose spec does not declare
languagereturned HTTP 500 for every transcription sent from the WebUI.The blocker
build_openai_transcription_requestwroterequest.options["language"]whenever the request body carried the key. Clients send that field unconditionally, empty or not, and spec-backed models validate their request options strictly, so the model rejected its own client:Parakeet-TDT could not transcribe at all from the WebUI. AudioSR, ControlFoley, MiDashengLM-Gen, PersonaPlex and HeartMuLa fail the same way through
/v1/tasks/run, which folds a top-levellanguageinto the option map.The model's acceptance of
languageis now resolved once at registration, next to the existingaccepts_reference_textflag and for the same reason — resolving it per request re-reads the embedded spec on the request thread — and the field is forwarded only where the contract takes it. Alanguagethe caller placed insideoptionsexplicitly is left alone and still produces the strict rejection, which is the correct signal there. The language keeps reaching every model throughtext_input;qwen3_forced_alignerrequires it from there, so gating the input client-side would have broken forced alignment.The other three
/v1/audio/transcriptionsreturns the transcript detail fields. The route returned only{text, timing}whiletask_result_jsonalready serialised segments, speaker turns and word timings, so every ASR model was routed away from the one endpoint that would return its timing data. The emitter is now shared by both routes, and both carry the sample rate the spans are counted in — without it a client cannot turn them into timestamps. Parakeet turns out to have been producing word timings all along.config.backenddefaults to CUDA and nothing validated it, so a machine without CUDA served the UI, the model list and the package installer, and failed only when the user pressed Run. The probe initialises and immediately frees a backend, so it costs one context creation at startup and reports the same message the engine would have raised, at the point an operator can still act on it./v1/models/unloadand the unload-all path now require--ui-managementlike every other mutating UI route, snapshot the model list undermodels_mutex_, and acquire each model throughacquire_model_runinstead of an unbounded busy-wait.live_ingest_limitsreads its state under the same mutex.Scope
Server only; no model, engine or output behaviour changes. Affects the
/v1/audio/transcriptions,/v1/tasks/runand/v1/models/unloadroutes. Families checked against the option contract: Parakeet TDT, AudioSR, ControlFoley, MiDashengLM-Gen, PersonaPlex, HeartMuLa, Qwen3 forced aligner.Validation
Backend tested: Metal, Apple M4 Max, macOS 15.
Reproduced the 500 against a server built from
origin/main, then confirmed the same request succeeds on this branch. End-to-end run on this branch: OmniVoice speech render, then the resulting WAV through Parakeet TDT on/v1/audio/transcriptions, which returned"The quick brown fox jumps over the lazy dog."with nine word timings andsample_rate: 24000— fields the route previously dropped.Known limitations
The backend probe adds one backend init/free to startup. On Metal that is milliseconds; on a large CUDA machine it is the usual context creation cost, paid once.