Skip to content

server: stop rejecting requests over options the client always sends - #369

Closed
CryptVenture wants to merge 1 commit into
0xShug0:mainfrom
CryptVenture:pr/server-request-contract
Closed

server: stop rejecting requests over options the client always sends#369
CryptVenture wants to merge 1 commit into
0xShug0:mainfrom
CryptVenture:pr/server-request-contract

Conversation

@CryptVenture

Copy link
Copy Markdown
Contributor

Summary

Four server fixes in the request path. The first one is a hard blocker: a model whose spec does not declare language returned HTTP 500 for every transcription sent from the WebUI.

The blocker

build_openai_transcription_request wrote request.options["language"] whenever the request body carried the key. Clients send that field unconditionally, empty or not, and spec-backed models validate their request options strictly, so the model rejected its own client:

POST /v1/audio/transcriptions
{"model":"parakeet-tdt","audio":"...wav","language":"","text":"","options":{}}
-> {"error":{"message":"unknown Parakeet TDT request option: language"}}

Parakeet-TDT could not transcribe at all from the WebUI. AudioSR, ControlFoley, MiDashengLM-Gen, PersonaPlex and HeartMuLa fail the same way through /v1/tasks/run, which folds a top-level language into the option map.

The model's acceptance of language is now resolved once at registration, next to the existing accepts_reference_text flag and for the same reason — resolving it per request re-reads the embedded spec on the request thread — and the field is forwarded only where the contract takes it. A language the caller placed inside options explicitly is left alone and still produces the strict rejection, which is the correct signal there. The language keeps reaching every model through text_input; qwen3_forced_aligner requires it from there, so gating the input client-side would have broken forced alignment.

The other three

  • /v1/audio/transcriptions returns the transcript detail fields. The route returned only {text, timing} while task_result_json already serialised segments, speaker turns and word timings, so every ASR model was routed away from the one endpoint that would return its timing data. The emitter is now shared by both routes, and both carry the sample rate the spans are counted in — without it a client cannot turn them into timestamps. Parakeet turns out to have been producing word timings all along.
  • The requested backend is proved before the port is bound. config.backend defaults to CUDA and nothing validated it, so a machine without CUDA served the UI, the model list and the package installer, and failed only when the user pressed Run. The probe initialises and immediately frees a backend, so it costs one context creation at startup and reports the same message the engine would have raised, at the point an operator can still act on it.
  • Unload is gated and bounded. /v1/models/unload and the unload-all path now require --ui-management like every other mutating UI route, snapshot the model list under models_mutex_, and acquire each model through acquire_model_run instead of an unbounded busy-wait. live_ingest_limits reads its state under the same mutex.

Scope

Server only; no model, engine or output behaviour changes. Affects the /v1/audio/transcriptions, /v1/tasks/run and /v1/models/unload routes. Families checked against the option contract: Parakeet TDT, AudioSR, ControlFoley, MiDashengLM-Gen, PersonaPlex, HeartMuLa, Qwen3 forced aligner.

Validation

cmake -S . -B build/macos-metal-tests -DCMAKE_BUILD_TYPE=Release \
  -DAUDIOCPP_BUILD_NATIVE_MODEL_MANAGER=ON \
  -DAUDIOCPP_USE_SYSTEM_OPENSSL=ON \
  -DOPENSSL_ROOT_DIR="$(brew --prefix openssl@3)"
cmake --build build/macos-metal-tests -j 12
ctest --test-dir build/macos-metal-tests --output-on-failure

Backend tested: Metal, Apple M4 Max, macOS 15.

Reproduced the 500 against a server built from origin/main, then confirmed the same request succeeds on this branch. End-to-end run on this branch: OmniVoice speech render, then the resulting WAV through Parakeet TDT on /v1/audio/transcriptions, which returned "The quick brown fox jumps over the lazy dog." with nine word timings and sample_rate: 24000 — fields the route previously dropped.

Known limitations

The backend probe adds one backend init/free to startup. On Metal that is milliseconds; on a large CUDA machine it is the usual context creation cost, paid once.

A model whose spec omits `language` returned HTTP 500 for every request from
the WebUI, because build_openai_transcription_request wrote
request.options["language"] whenever the body carried the key -- which clients
do unconditionally, empty or not -- and spec-backed models validate their
request options strictly.

Reproduced against a running server on this branch's binary:

  POST /v1/audio/transcriptions
  {"model":"parakeet-tdt","audio":"...wav","language":"","text":"","options":{}}
  -> {"error":{"message":"unknown Parakeet TDT request option: language"}}

Parakeet-TDT could not transcribe at all from the WebUI. AudioSR, ControlFoley,
MiDashengLM-Gen, PersonaPlex and HeartMuLa fail the same way through
/v1/tasks/run, which folds a top-level `language` into the option map.

Resolve the model's acceptance of `language` once at registration, next to the
existing accepts_reference_text flag and for the same reason (resolving per
request re-reads the embedded spec on the request thread), and forward the
field only where the contract takes it. A `language` the caller placed inside
`options` explicitly is left alone and still produces the strict rejection,
which is the correct signal there. The language continues to reach every model
through text_input, so nothing loses the feature -- qwen3_forced_aligner
requires it from there, so gating the input client-side would have broken it.

Three further server fixes in the same area:

Return the transcript detail fields from /v1/audio/transcriptions. The route
returned only {text, timing} while task_result_json already serialised
segments, speaker turns and word timings, so every ASR model was routed away
from the one endpoint that would return its timing data. The emitter is now
shared by both routes, and both carry the sample rate the spans are counted
in, without which a client cannot turn them into timestamps. Parakeet turns
out to have been producing word timings all along.

Prove the requested backend exists before binding the port. config.backend
defaults to Cuda and nothing validated it, so a machine without CUDA served
the UI, the model list and the package installer, and failed only when the
user pressed Run. It now fails at startup with the engine's own message:

  audiocpp_server failed: CUDA backend requested but it is not registered in
  this build (available: MTL:0 "Apple M4 Max" [GPU], BLAS:0 "Accelerate"
  [ACCEL], CPU:0 "Apple M4 Max" [CPU])

Bound the bulk unload routes. Both called busy.acquire(0, ...), which
busy_guard.h documents as an unbounded wait, on exactly the wedged run an
operator calls these routes to clear; they now use the same bounded helper as
every other unload path. They also skip the ui_management gate that
/v1/models/unload enforces, and read models_ and model_index_ without holding
models_mutex_ -- handle_unload_all_models ranges over the vector that
/v1/models/load appends to, so a concurrent load could reallocate underneath
it.

Validation:
  cmake --build build/macos-metal-tests --target audiocpp_server -j 12
  ctest -j 6                                       # 53/53 pass
Backend tested: Metal (Apple M4 Max). Verified end to end against a live
server: Parakeet transcribes the exact UI body and returns 9 word timings with
sample_rate 24000; AudioSR accepts a request carrying language="en" that the
pre-change binary rejected; --backend cuda now exits 1 at startup on this
machine while --backend metal starts normally.

Known limitations: the CUDA and Vulkan startup probes were not exercised, only
the Metal and missing-CUDA paths. The probe initialises and immediately frees
one backend context, which is new work at startup on every deployment.
@0xShug0

0xShug0 commented Sep 1, 2026

Copy link
Copy Markdown
Owner

As mentioned in another PR, please split into separate PRs: language-option contract, ASR detail response, backend startup probe, unload locking so I can test each of them individually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants