Add TTS endpoint (Phase 1) - #21
Conversation
fd4c407 to
998f11d
Compare
998f11d to
dd2c171
Compare
Gateway TTS Phase 1 Branch Review
Executive summaryDo not merge the current branch. The core routed-speech implementation is substantial and its covered paths are healthy, but the branch has four production defects, six material documentation, verification, governance, or tooling defects, and five smaller record or history defects. The most consequential problems are unsupported local speech models downloading artifacts before refusal, unbounded permit ownership under slow streaming, profile cancellation ending as clean success, and a 30-second read timeout that also limits time to first response headers. The branch adds 3,283 lines and removes 81 across 43 files in ten commits over current master. Forty-one focused TTS tests pass. Formatting, the featureless gateway build, and warnings-denied Clippy for the four owning crates also pass. These results establish that the implemented batch speech path works under its covered conditions. They do not exercise the failure modes identified in this review. The original gripe list was directionally useful but not uniformly correct. Together batch synthesis works, and phase 1 originally excluded Together SSE. The defect is that later documentation advertises an impossible SSE path. The current Vibe Coder schema and pending YAML todo statuses are not valid merge objections because the run closed under an older contract and completion is tracked through step headings. By contrast, stale survey claims became actionable when the branch was rebased onto a base that had removed the named tools and renumbered the architecture. The recommended repair is a history rewrite that amends the owning commits rather than stacking cleanup at the tip. Preserve the phase 1 decision not to implement Together SSE, bound the audio relay, move unsupported-kind checks before provisioning, replace the unmanaged Python probe with a dependency-free Node live-gateway probe, remove direct provider calls, add the missing behavior tests, and reconcile every plan and design claim. Estimated engineering work is 24 to 32 hours, plus live-provider access and complete CI. Confidence is high because the main findings follow directly from control flow, commit history, current provider documentation, and reproducible repository checks. Contents
Decision and required conditionsMerge only after the author completes the following conditions:
This is a conditional no-go rather than a rejection of the routed-speech design. The feature can become merge-ready without changing its core architecture. Scope, method, and limitationsThe review compares local branch The review applied the root The review judged the branch against six criteria: user-visible protocol correctness, architecture and ownership conformance, bounded resource and cancellation behavior, tests that would detect each claimed failure, truthful documentation and provenance, and repairability within the existing commit structure. Local verification ran 14 shared-protocol speech tests, four gateway-config voice tests, one gateway-local refusal test, and 22 gateway speech integration tests. All 41 passed. Formatting, the featureless gateway check, and warnings-denied Clippy for The review did not rerun the credentialed live-provider probe, provoke paid-provider rate limits, or execute the complete cross-platform CI matrix. Repair estimates include implementation, focused tests, and documentation, but exclude queue time on external runners. Provider behavior can drift after the 2026-09-09 documentation snapshot. What the branch gets rightThe routed model-kind decision fits the repository. Speech models use the existing catalog, routing, dominion queue, upstream abstraction, and model discovery instead of adding a parallel service lifecycle. This preserves the gateway as the runtime credential owner and follows the established chat, embedding, and classifier shape. The batch speech path is well covered. The tests prove model rewriting, bearer separation, structural MP3 defaulting, byte-preserving response relay, MIME fallback for six audio formats, voice validation before queue admission, queue-full behavior, permit retention, client-disconnect cleanup, error-body secrecy, 429 and 503 mappings, and voice-union ordering. The branch also handles the new model kind safely in the launch mapping. These strengths narrow the repair. The recommendation does not replace the route, model kind, upstream method, or voices union. Severity-ranked findingsThe findings rank as follows. Estimates overlap where one relay change closes several defects.
F1. Reject local speech before provisioning
The branch promises that a local speech model fails because no local speech runtime exists. The failure happens too late. An operator who accidentally declares a local speech model can download a multi-gigabyte model, provision The strongest counterargument is that prefetching may help phase 2. That behavior is speculative and conflicts with the phase 1 statement that local speech is unsupported. A preparation command must not report success for a model the runtime cannot start. Fix the defect by extracting a side-effect-free supported-kind preflight. Run it before provisioning the shared server and before each model or companion download. Mixed profiles should preserve supported-model progress while returning a per-model failure for unsupported kinds. Add tests proving that an all-speech profile touches neither the server provisioner nor the model store, and that a mixed profile provisions only supported models. Estimated work is 2 to 4 hours across F2. Bound audio relay ownership independently of polling
Architecture invariant A5 requires every stream and wait to be bounded. Invariant A27 separately requires stream permits to survive until body termination and stalled reads to expire. The implementation satisfies A27 under ordinary polling but does not fully satisfy A5.
The request-side 4,096-character cap, queue-depth cap, error-body cap, client-disconnect drop chain, and typical provider behavior all reduce the likelihood of accidental occurrence. They do not create a finite successful-response lifetime. An accepted caller or faulty upstream can occupy every speech dominion slot indefinitely. An endpoint with no dominion avoids permit starvation but permits unbounded concurrent upstream tasks and sockets instead. A short whole-request timeout is not an acceptable fix. A 4,096-character request at slow playback can legitimately produce many minutes of audio, and raw PCM is large. The safer policy is a bounded background relay that owns the upstream stream, permit, and cancellation guard while feeding a small bounded channel to the HTTP body. The task should enforce a generous total lifetime, a byte ceiling, upstream idle, blocked downstream delivery, cancellation, and receiver drop independently. A reasonable static starting envelope is 60 minutes, 1 GiB, 30 seconds of upstream read idle after headers, and 60 seconds blocked on downstream delivery. Together documents Orpheus output as 24 kHz signed 16-bit PCM. One hour of mono audio at that rate is 172,800,000 bytes, about 165 MiB, before SSE framing. The proposed byte limit leaves more than six times that payload. These values still need production validation, named constants, and boundary tests. Configurability should wait until operators demonstrate a need because it would add schema, validation, UI, and documentation surface. Estimated work is 8 to 12 hours for a static bounded pump and deterministic tests. Amend the owning protocol and route commits, then update the plan, guide, and as-built design. Confidence in the need for a finite policy is high. Confidence in the suggested numeric values is medium because production duration distributions are not in the repository. F3. Make profile cancellation fail the body read
The branch correctly notes that no JSON envelope can follow audio bytes. That does not require clean EOF. Emit one body error when profile cancellation wins, then terminate. The bounded relay proposed in F2 should own this behavior so cancellation remains observable even when the downstream has stopped polling. Add a profile-switch integration test that reads one audio chunk, switches profile, and proves the next body read fails, the upstream drops, and the permit becomes available. This requires 1 to 2 hours if implemented alone and less when folded into F2. F4. Separate response-header and body-idle deadlines
The dedicated client applies This matters because Together batch synthesis is not the low-latency SSE mode. The accepted request can contain 4,096 characters, while live verification used about 90. A long request or cold provider can accept and potentially bill work before the gateway reports The strongest counterargument is that production providers usually return headers promptly or emit chunks during generation. No branch evidence establishes the latency distribution, so actual incidence remains uncertain. The code mechanism and resulting error classification are certain. Use a longer explicit first-response deadline, then enforce the shorter body-idle deadline in the relay itself. This design overlaps the bounded pump in F2 and avoids forcing one timeout value to serve two different phases. Add one test where headers arrive after the body-idle threshold but before the first-response threshold, and another where an opened body stalls. Estimated incremental work is 2 to 4 hours, mostly overlapping F2. Amend F5. Stop advertising Together SSE as a phase 1 capability
Together batch speech works. The live note proves MP3 and WAV output through the gateway. Together SSE does not work through this branch. Together's current documentation requires The source report states the correct scope at The practical consequence is false operator guidance. Preserve phase 1 scope and correct the plan, README, guide, parity terminology, verification note, and as-built design. State that OpenAI This correction requires 2 to 4 hours across six or seven documentation and verification files. Actual Together opaque SSE passthrough would require adding One small production defect remains even for an OpenAI SSE-capable upstream. F6. Remove the direct-provider credential path
Architecture invariant A19 says vendor credentials remain inside the gateway. The plan repeats that nothing above the gateway holds a vendor key, then requires a direct provider comparator. This is not an inferred objection. The introducing commit message states: “Violates: A2 - tools/gateway-tts-parity.py holds TOGETHER_API_KEY outside the gateway and calls the provider directly.” The live verification note confirms that the direct calls actually ran. The close commit leaves the admitted violation unresolved. Incremental secret-exposure risk is low because the script is opt-in, does not run in CI, and does not print or newly persist the key. The governance consequence is larger. The script bypasses gateway-owned redaction, routing, accounting, and error handling while the plan simultaneously treats credential isolation as a founding invariant. The strongest counterargument is that architecture invariants govern deployed components, not an operator's local oracle. The written rule contains no such exception, and the branch explicitly labels the implementation a violation. Review cannot silently invent an exemption. Preserve the invariant. Remove the direct Together half and dotenv parsing. The live tool should call only a separately configured gateway using its local bearer. Provider defaults can remain sourced to current documentation, while mocked tests prove exact outbound bodies. If maintainers want direct-provider probes, they should approve a narrow developer-tool exception with host pinning, no persistence, bounded calls, and no CI. Removing the direct half alone takes 1 to 2 hours across the script, plan, note, ledger, commit message, and as-built design. This work should be combined with F7. F7. Replace the unmanaged standalone Python tool
The repository previously used short standard-library Python fragments inside workflows. This branch adds its first tracked standalone The documented no-key skip path is already false on a clean machine. The binary-selection logic creates a second reproducibility defect. Python is not technically required. The repository already requires Node 22 and uses Replace the Python probe with a Node script that contacts only the gateway. Keep exact protocol behavior in Rust integration tests. Record a separate manual official-SDK smoke run against the gateway if compatibility with one client library is an acceptance concern. The replacement is estimated at 4 to 8 hours, including deterministic helper tests, a credentialed live rerun, and updates to the plan, note, ledger, guide references, and as-built design. Retaining Python is the weaker alternative: add pinned dependencies, an installation workflow, offline unit tests, and an explicit tooling exception. That option takes roughly 2 to 4 hours but leaves an additional language environment to maintain. F8. Add route-level tests for claimed behavior
The existing suite is broad, but five claimed boundaries lack direct coverage. First, the product requirements promise a specified unknown-model envelope on the speech route. Only the chat integration suite directly tests Second, every voices integration helper sends a bearer at Third, the plan and as-built design claim profile-switch cancellation truncates speech, but no speech integration test switches profile during an open body. This omission concealed F3 and the unpolled-cancellation portion of F2. Fourth, angle-bracket preservation is tested only through the optional live script. The generic passthrough test uses ordinary text. Offline CI could regress the specific hard requirement without failing. Fifth, a wire test accepts and round-trips The unknown-model, voices-authentication, emotion-tag, and SSE-composition gaps are individually low severity because shared mechanisms already have coverage. Profile-switch cancellation is medium severity because it is bespoke composition and concealed F3. Add all five direct pins in 1.5 to 2 hours. The no-provision and bounded-relay tests belong to F1 and F2 estimates. Tests should amend the commits that introduce each behavior rather than land in a catch-all tip commit. F9. Restore verifiable provenance after the rebase
The branch was rebased after live verification and documentation. Maintainers cannot inspect the exact trees that supposedly passed live verification. The stale-binary behavior in F7 also means the recorded source hash would not prove which executable was exercised even if the commit existed. The rebased content may be equivalent, but the report rulebook and the repository's own evidence discipline require verifiable names and numbers. Rerun the live check against a freshly built executable from a retained rewritten commit. Record the executable hash, commit, and tree hash, because subsequent message-only amendments can change the commit while preserving the tree. Update the note, as-built design, ledger, and commit messages. Estimated repository work is 1 to 2 hours, excluding provider and CI queue time. F14. Dispose architecture records before closure
Immediately before closure, The Vibe Coder contract in force when the TTS run closed required the operator alone to promote, reject, or leave each queue record open. It also required queue dispositions to live in an operator-authored drain commit, never a step or Close commit. The closure carries no disposition rationale and the accepted architecture contains neither record. This has no runtime consequence, but it removes the audit trail for two findings while claiming the plan is closed. Restore the records at the end of the amended implementation series, have the operator dispose each one explicitly in a separate drain commit, and only then replay closure. N69 appears malformed because it calls an undetermined fact a violation; that is a reason to reject it explicitly, not silently delete it. Repository work is under 1 hour. Operator review time is separate. Do not fold the disposition into F10. Make parity claims match actual assertions
The plan says the gateway and provider run the same assertions. The script runs the same call set but not the same checks. WAV, emotion tags, and three passthrough fields are asserted for both targets at Literal symmetry would be wrong because the gateway deliberately changes default format and adds a voices route. The defect is the promise, not the asymmetry. The script still provides useful live evidence for the shared subset. If F6 removes direct provider calls, rename the tool as a live gateway smoke probe and remove parity claims. If maintainers approve direct comparisons instead, represent target expectations explicitly and make known provider baselines fail on drift. Avoid a boolean mode flag whose meaning is spread through the function. The script and document correction is 1 to 2 hours, largely overlapping F6 and F7. F11. Correct the post-rebase survey and CI account
Most stale survey claims were true at the original TTS base The current plan names removed Node tools, claims a removed JSON ceiling file exists, says a removed UI pretest still runs, and cites A102 and A112 through A117. It also says only Do not restore any removed parser, walker, count, or ceiling. Amend the survey to record its original base and a post-rebase validation section against The CI wording needs a separate narrow correction. Plan line 80 calls a feature-focused subset “the exact CI gates,” while line 143 says every step ended with the full suite. Describe focused, component, and final verification accurately. Then obtain one exact-commit integrated CI result after the rewrite. Updating the plan and ledger takes 1 to 2 hours. F12. Complete public model-kind documentation
Five live documentation locations still enumerate the old three-kind set:
The code works because the type already carries F13. Synchronize terminal frontmatter metadata
Six frontmatter todos remain pending at The mismatch cannot reopen work, break closure, or change generated commit messages. It can still mislead a frontmatter-aware viewer or a maintainer reading the plan without knowing which state is authoritative. Repository precedent also exists for synchronizing every todo in the final execution commit. Amend F15. Keep the first retained speech commit safe
Combine Original gripes that do not standCurrent Vibe Coder tags are not retroactiveThe plan does not satisfy the current strict tag schema. That schema was introduced after the original TTS run closed. The run had no contract tags, step tags, exact capitalized Project Survey heading, survey status line, or component-test field because the governing tool did not yet require them. The local rebase happened later, but it replayed a closed run and did not reactivate Do not make current Vibe Coder syntax a merge condition. If maintainers want every newly merged plan file to parse under the current tool regardless of execution date, add a separate post-close migration commit and ledger note. Estimated optional work is 20 to 30 minutes. Confidence is high. Together SSE absence is not a batch-route failureThe source report deliberately scoped Together phase 1 to batch passthrough. MP3 and WAV batch calls succeeded live, and focused tests cover their gateway behavior. The inability to request Together's SSE mode is therefore not evidence that the delivered batch feature fails. F5 remains because user-facing documentation advertises more than the branch implements. Remote PR failures do not describe the rebased TTS codeThe remote pull request currently reports merge conflicts and two failed checks on its pre-rewrite lineage. The core check stopped on an unrelated STT Clippy diagnostic that current local master already fixes. The native lane stopped because the self-hosted runner lacked the required Rust 1.89 directory. The local branch passes the reviewed TTS tests, formatting, featureless check, and owning-crate Clippy. Nevertheless, the rewritten combined tree has no complete exact-commit CI result. The remote failures should be replaced by fresh results after push, not counted as TTS defects. Amendment plan by commitThe branch is already organized into behavior-sized commits. Preserve that structure and amend defects into the commits that introduced them. The recommended chronological rewrite is:
The estimates overlap. The bounded relay, timeout separation, cancellation test, and several missing tests should be implemented as one coherent stream-lifecycle change. The Node rewrite, credential correction, parity correction, and live rerun should likewise be one verification-tool change. Expected total engineering effort is 24 to 32 hours for the recommended scope. A developer already familiar with the gateway test harness is likely to land near the lower bound. External live-provider access and CI queue time are additional. Confidence is medium because deterministic backpressure tests are the largest uncertainty. The rewrite should begin at Verification required before mergeFocused behavior verificationThe amended local-provisioning commit must prove that unsupported kinds perform no artifact or metadata side effects. Test all-speech and mixed supported and unsupported profiles with injected provisioners and stores. The amended stream commit must prove all terminal paths. Required cases are exact byte limit, one byte over limit, total deadline, sub-idle upstream drip, slow downstream channel saturation, upstream idle after headers, delayed but accepted headers, client disconnect, profile cancellation, and upstream body error. Every case must prove permit release and admission of a subsequent request. The route suite must add unknown-model, unauthenticated voices, angle-bracket input, SSE content-type fallback, and the intentionally unsupported Together-native streaming shape. Existing batch MP3, WAV, queue, error, voice, and disconnect tests must remain green. The replacement live tool needs offline tests for gateway startup failure, readiness timeout, request failure, assertion failure, process cleanup, skip behavior, and secret-free output. The credentialed run must contact Together only through the gateway and record the retained commit and tree. Repository gatesRun formatting, warnings-denied Clippy, non-Workshop workspace tests, doctests, warnings-denied documentation, the featureless gateway build, dependency policy, supply-chain checks, both UI packages, and clean-tree verification. Rebuild the generated guide and prove that regeneration leaves no unexplained diff. Run the Workshop Windows and Linux lanes because the rebased base overlaps gateway configuration, routing, and integration harness files. Run MSRV because the branch adds public Rust types and dependency usage. The native Whisper lane remains externally blocked until the self-hosted runner supplies the configured Rust 1.89 binary directory. Do not infer that lane passing from local TTS results. Evidence closureRecord one exact rewritten commit for every local and remote result. Use tree hashes for live observations that survive message-only amendments. Remove or correct every invalid historical object reference. The final pull request must be conflict-free and every required check must report against the pushed head. ConclusionThe routed-speech architecture should proceed, but the current branch is not merge-ready. Its ordinary remote batch path is functional and well tested. The reasons to stop are narrower and concrete: unsupported local models perform side effects, stream ownership can outlive every useful bound, cancellation can look successful, the first-response timeout is too tightly coupled to body idle, documentation advertises an unavailable Together mode, and the verification tool knowingly violates credential ownership while introducing an unmanaged runtime. Amending the existing commit series is preferable to adding a repair stack. The defects align cleanly with the commits that introduced config, local launch behavior, protocol streaming, routing, verification, and documentation. A 24 to 32 hour repair budget is proportionate to the branch's 3,283 added lines and the need for deterministic backpressure tests. Final recommendation: request changes, complete the chronological amendment plan, rerun live verification through the gateway, and require a complete exact-commit CI matrix before merge. Confidence is high in the merge call and medium in the repair duration. ReferencesRepository evidence
External provider evidence
2026-09-09 04:02 - gpt-5.6-sol |
|
I rebased the branch and fixed merge conflicts. Let's go over this report and try to figure out what it means. |
|
This introduces the first Python source file in the project, are you sure we can't just do this in JS? |
c13d454 to
c4ec890
Compare
Local builds were pinned to an older Rust while CI linted and tested on stable, and that skew just produced three CI failures on a pull request that had passed every local check. This change puts local builds and CI on the same stable toolchain, removes the minimum-version declarations and the CI job that enforced them, and deletes the runner validation script together with its dedicated test. The three failures the skew exposed are fixed in place: an unnecessary-parentheses lint, a pair of lint expectations consolidated under a new name, and a paused-clock test structurally incompatible with its real-time rendezvous. - `rust-toolchain.toml` now selects `channel = "stable"`, so local builds track the same toolchain CI uses; the header comment describes that rule instead of the removed pin. - `rust-version` leaves the workspace manifest and the crate manifests, ending the resolver's preference for dependency versions compatible with the old minimum. - `crates/gateway-stt/src/audio.rs` drops the parentheses around the trait object type in an error-chain assertion, clearing the lint stable fires on them. - `crates/workshop/src/bridge.rs` expects `clippy::ref_as_ptr` on the test module in place of `clippy::ptr_as_ptr` and `clippy::borrow_as_ptr`, which stable consolidated into the single lint. - `decode_rendezvous_timeout_releases_a_late_arrival_and_permits_a_follow_up` runs on the real clock again, because a paused clock cannot drive the test's real-time condvar rendezvous. - `.github/workflows/ci.yml` loses the `msrv` job and every `RUSTUP_TOOLCHAIN` override, and `ci-green` no longer lists `msrv` among its needs. - `tools/validate-rust-1.89.0.ps1` is deleted along with `tools/check-stt-native-workflow.test.mjs`, the test file that existed only to exercise it. - `README.md` loses the minimum-version section, the version badge, and the version number in the build instructions; `AGENTS.md` loses the bullet warning that CI lints on a newer toolchain than local. Design: removes feature-flag @ .github/workflows/ci.yml Plan: vibe/2026-09-11-2-unify-toolchain.md
State the timeout durations in the units the values use. The change rewrites four timeout constants in `artifacts.rs`, `live.rs`, `backoff.rs`, and `cuda.rs` from `Duration::from_secs` products to `Duration::from_mins` and `Duration::from_hours`. - The rewritten constants keep their previous values; no test or call site changes.
Speech synthesis becomes a fourth routed model kind beside chat, embedding, and classifier, so a catalog entry can declare a text-to-speech model and clients can discover it before shaping requests. A model may now declare the voices it offers, and the declaration is validated at load: entries must be non-empty and unique, and a non-empty list is rejected on any non-speech kind, symmetric with the existing chat-only field discipline. An empty or absent voice list stays valid, so providers with no fixed voice catalog keep working. The configuration interface gains the new kind as a dropdown choice and a chips editor for the voices that renders only for speech models. - `ModelKind::Speech` joins the kind enum with the serde and Display spelling "speech", a fourth routed catalog kind whose spelling round-trips through TOML and JSON. - `voices` is a private, serde-defaulted string list on the capabilities record, documented as speech-only and surfaced verbatim on the models listing; empty means the model exposes no fixed voice list. - `validate_capabilities` rejects empty voice entries and duplicate voices at load, the error naming the voices field or the duplicated entry. - `validate_kind_scope` rejects a non-empty voices list on chat, embedding, and classifier models, the symmetric half of the chat-only field discipline, so a misplaced list fails loudly naming the field. - `settings-registry.ts` declares a chips editor for voices that renders only when the model kind is speech, and `models-view.ts` gains speech as a fourth kind dropdown option. - `model-detail.test.mjs` pins the four-kind dropdown and a pick-speech, add-chip, save round-trip into the PUT body, while the new config validation tests pin each acceptance and rejection. Design: new surface-growth @ crates/gateway-config/src/config.rs::ModelKind::Speech boundary: wire Design: new surface-growth @ crates/gateway-config/src/config.rs::Capabilities::voices boundary: wire Design: new encapsulated-invariant @ crates/gateway-config/src/config.rs::Capabilities Design: new surface-growth @ crates/gateway-config/src/config/accessors.rs::Capabilities::voices boundary: pub Deferred: the speech kind exists one commit before launch_options becomes fallible and refuses it for local models Plan: vibe/2026-09-07-2-gateway-tts-phase-1.md
Launching a local model now fails loudly when the model's kind has no local serve mode. The kind-to-mode mapping previously defaulted every unmatched kind to chat, so a speech model declared for local launch would have started as a chat server; it now returns a dedicated error naming the kind, and any kind added to the enum later fails the same way instead of launching misconfigured. Both local launch paths check every configured model's kind before provisioning anything, so a profile containing an unsupported kind refuses it before any download, server provisioning, or metadata write, and a profile with no launchable model does nothing at all.
- `serve_mode_for` extracts the kind-to-serve-mode mapping as a side-effect-free fallible function, so both launch paths can preflight a kind before any provisioning; its wildcard arm refuses kinds added after the mapping because `ModelKind` is `#[non_exhaustive]`.
- `LocalError::UnsupportedKind` carries the offending `ModelKind` and renders "local {kind} models are not yet supported", a public variant downstream callers can match.
- `start_impl` and `provision_artifacts_impl` check every configured model's kind before the shared server or any model provisions; an all-speech profile produces per-model refusals with zero side effects, a mixed profile provisions only supported models, and FailFast returns the first refusal.
- `launch_options` returns `Result<LaunchOptions, LocalError>` and `launch_options_for` propagates, with existing callers and tests unwrapping.
- `tool_dialect` keeps its chat-default wildcard; that default is deliberate and untouched.
Design: swallowed-exception -> pure-function @ crates/gateway-local/src/runtime.rs::serve_mode_for deps: ModelKind
Design: new surface-growth @ crates/gateway-local/src/error.rs::LocalError::UnsupportedKind boundary: pub
Repairs: A14 @ crates/gateway-local/src/runtime.rs::launch_options - a local speech model launched as a chat server through the catch-all kind arm
Plan: vibe/2026-09-07-2-gateway-tts-phase-1.md
Speech synthesis requests now have a wire type carrying boundary validation. It rejects an empty model, empty or over-length input, an out-of-range playback speed, unknown audio encodings and stream framings, and reserved keys smuggled into the passthrough map, while every field the gateway does not name is preserved verbatim for the provider. The default audio encoding is pinned in the type itself, because providers disagree on the default and an unpinned one would silently change the wire. - `SpeechVoice` is an untagged enum accepting a plain name or the OpenAI object form; catalog membership is deliberately not checked in the type, because voice sets are per-checkpoint. - `SpeechResponseFormat` is a closed set with a structural default: an omitted field resolves to mp3 at deserialization, and provider-only spellings such as raw stay unrepresentable until the enum is deliberately widened. - `SpeechRequest` keeps every field the gateway does not name in a flattened `rest` map for verbatim passthrough, and `RESERVED` names the seven known keys so a duplicate smuggled into `rest` fails validation. - `validate` rejects an empty model, empty input, input past 4096 characters, and speed outside 0.25 to 4.0, returning a static reason string for each failure. - `speech_request_validation_table` pins the cap in characters rather than bytes: 4096 two-byte characters pass and 4097 fail. - `SpeechRequest` has no production caller in this change; only the new tests construct it. Design: new bag-of-state @ crates/shared-protocol/src/wire.rs::SpeechRequest Design: new surface-growth @ crates/shared-protocol/src/wire.rs::SpeechRequest boundary: wire Design: new value-object @ crates/shared-protocol/src/wire.rs::SpeechVoice boundary: wire Plan: vibe/2026-09-07-2-gateway-tts-phase-1.md
The upstream abstraction can now forward a speech synthesis request to a backend and return the audio as a stream of opaque bytes, and an upstream with no speech implementation declines the workload by default rather than fabricating a response. The forwarding substitutes the backend's model name for the caller's, sends the endpoint credential exactly as the other workloads do, and passes the audio through untransformed. Time to the first response bytes and per-read idle on the opened body are bounded as two separate budgets, because the HTTP client library arms its read timeout during the header wait and would otherwise cap slow but legitimate generations at the body-idle budget. A mid-stream failure or stall surfaces as one error item and ends the stream, so a truncated response is never mistaken for a complete one. - `Upstream::send_speech` defaults to `ProtocolError::ModelUnavailable` naming the caller's model, so an upstream without speech refuses the workload; a test drives the default through `Arc<dyn Upstream>` to pin object safety. - `audio_streaming_client()` carries only a connect timeout and a 60 s TCP keepalive: reqwest arms `read_timeout` during the wait for response headers, so both speech deadlines live in `send_speech` rather than on the client. - `FIRST_RESPONSE_TIMEOUT` (120 s) bounds the header wait and `AUDIO_READ_TIMEOUT` (30 s) bounds each read of the opened body; both are `cfg(test)`-scaled so the deadline-separation tests run in milliseconds. - `send_speech` rewrites the request model to `upstream_model`, posts to `audio/speech`, and forwards the body otherwise verbatim; tests pin the model substitution, the bearer forwarding, and byte passthrough including invalid UTF-8. - `StreamedAudio` carries the upstream `Content-Type` verbatim, empty when the upstream omits it, beside the raw byte stream; a transport failure or read stall yields one `Err` item and then ends the stream. - `with_client` now backs all three clients from the one injected client, so a test client built with a `read_timeout` also arms the audio path. - `streaming_client()` and the chat SSE path are unchanged; the audio path is a third client field, not a modification of the existing two. - `ProtocolError::UpstreamStatus` keeps its existing shape for upstream 429 and 503, status plus capped body; this change adds no error variants and no envelope mapping. Design: new pure-function @ crates/shared-protocol/src/http_util.rs::audio_streaming_client boundary: pub Design: new surface-growth @ crates/shared-protocol/src/upstream.rs::Upstream boundary: pub Plan: vibe/2026-09-07-2-gateway-tts-phase-1.md
The gateway gains a speech synthesis route for speech-kind models: an OpenAI-shaped endpoint that authenticates before parsing the body, validates the request and the voice before queue admission, and then streams the provider's audio bytes back unread. Audio frames are opaque bytes that cannot be re-validated per chunk, so the reply is a byte passthrough rather than the usual typed relay, and a bounded background task owns the upstream body, the queue permit, and the cancellation guard so neither a stalled provider nor a stalled client can hold a queue slot indefinitely. Upstream rate-limit and unavailability statuses map to speech-only 429 and 503 envelopes, while every other route keeps its existing error mapping.
- `audio_speech` runs `check_auth` on the ungated `Caller` parts-extractor before extracting `Json<SpeechRequest>` from the raw `Request`, so an unauthorized caller never makes the gateway parse a body. The kind guard and the voice check also run before dominion queue admission, so a 400 never burns a queue slot.
- `relay_audio` spawns `relay_speech_stream`, a background task that owns the upstream body, the dominion `Permit`, and the `InFlightGuard` cancellation guard and feeds a bounded channel (four data slots plus one reserved for the terminal error) to the HTTP body. `SPEECH_RELAY_TOTAL_LIFETIME` (60 min), `SPEECH_RELAY_BYTE_CEILING` (1 GiB), `SPEECH_RELAY_UPSTREAM_IDLE` (30 s), and `SPEECH_RELAY_DOWNSTREAM_BLOCKED` (60 s) cap every stream, each test-scaled under `cfg(test)`/`test-fixtures`.
- `GatewayError::UpstreamRateLimited` and `GatewayError::UpstreamUnavailable` give the speech route its own 429 `upstream_rate_limited` and 503 `upstream_unavailable` envelopes, matched out of `ProtocolError::UpstreamStatus` at the handler. The shared protocol mapping is untouched, so every other route's envelopes stay bit-identical.
- `GatewayError::InvalidVoice` rejects a voice outside the model's catalog list with 400 `invalid_voice`, and the message names the valid voices. An empty `voices` list skips the check and forwards any voice verbatim.
- `speech_fallback_mime` resolves the response `Content-Type` when the upstream omits or garbles it: the framing selector wins first, so an SSE stream is labeled `text/event-stream` and never an audio type, and only then does the requested format's MIME mapping apply.
- `gateway = { path = ".", features = ["test-fixtures"], default-features = false }` makes the crate dev-depend on itself so every test target builds the library with the relay's test-scaled bounds and no `--features` flag. A recorded side effect: the `not(test-fixtures)` rendezvous test in `boot.rs` now compiles out under every current runner, and its comment says so and names the re-enablement conditions.
- `build_router` registers `POST /v1/audio/speech`, and `admin_status` gains a "Speech synthesis" row through a table-driven refactor of the endpoint list.
- `crates/gateway/tests/it/speech.rs` pins byte-identical passthrough with the `<laugh>` emotion tag intact, the structural mp3 default reaching the provider, and no caller-bearer leak upstream; `kind_mismatch` in both directions (the reverse rows ride along in `chat.rs`, `embeddings.rs`, and `rerank.rs`); voice-before-admission under a full pool; queue-full 503; the permit held until the stream ends; client disconnect aborting the upstream stream; and profile-switch cancellation and mid-stream upstream error each failing the client's body read, with permit release proven by a later request's admission. The relay boundary tests cover the exact byte ceiling, one byte over, the total deadline, a sub-idle drip, an idle upstream, a blocked downstream, and delayed-but-accepted headers.
- `INFERENCE_DRAIN_TIMEOUT` names the profile-switch drain deadline (30 s) and test-scales it to 1 s, so a switch cancellation beats the relay's scaled 2 s total lifetime to the terminal error item.
- `Content-Length` is never set on the speech response, so hyper emits `Transfer-Encoding: chunked`.
- No JSON error envelope can follow 200 plus audio bytes: every terminal path (a bound tripped, an upstream error, a cancellation, the downstream gone) emits exactly one `Err` item, so the client's body read fails instead of seeing a clean EOF.
- `GET /v1/audio/voices` is advertised by the crate-doc header this change writes, but this change registers no such route and adds no handler for it.
Design: new surface-growth @ crates/gateway/src/lib.rs::audio_speech
boundary: wire
Design: new pure-function @ crates/gateway/src/lib.rs::speech_fallback_mime deps: Option<SpeechStreamFormat>,SpeechResponseFormat
Design: new oversized-unit @ crates/gateway/src/lib.rs::relay_speech_stream
Design: new oversized-unit @ crates/gateway/src/error.rs::gateway_error_classify_is_table_driven
Plan: vibe/2026-09-07-2-gateway-tts-phase-1.md
The gateway gains a read-only route that answers the union of every configured speech model's voices, deduplicated and sorted, as identifier-first entries under a single voices key. OpenAI has no voice-list route, but the OpenAI-compatible ecosystem converged on this shape and clients read the identifier key, so the entry shape is a compatibility surface pinned by the integration suite. Authentication runs before the union is computed, so an unauthenticated request is refused without any voice entry leaving the handler. Each entry's display name mirrors its identifier because the catalog configures voices as bare strings.
- `audio_voices` collects the active profile's speech models' configured voices into a `BTreeSet`, so the union is deduplicated and sorted by construction, then maps each voice to an `{"id", "name"}` object with `name` mirroring `id`. The doc comment records the ecosystem rationale: Kokoro-FastAPI, vLLM-Omni, and Fish Audio converged on this route, and clients such as Open WebUI read the `id` key.
- `build_router` registers `/v1/audio/voices` with `get(audio_voices)` between the speech and models routes, so the route is unconditional like its neighbors.
- `check_auth` runs before the switch lock and the live-state read, so authentication precedes any catalog access in the handler.
- `voices_route_deduplicates_and_sorts_across_speech_models` pins cross-model deduplication and ordering on the raw body; companion tests pin the id-first shape, the empty union without speech models, and non-speech exclusion.
- `unauthenticated_voices_request_is_refused_before_the_union_is_computed` pins the 401 `unauthorized` envelope and asserts no `voices` key is present, so the refusal happens before any union is computed.
- `catalog_gateway` exists only to satisfy config validation: its doc comment states the voices route never calls an upstream, and the fixture's recorder is unused.
Design: new surface-growth @ crates/gateway/src/lib.rs::audio_voices deps: Caller,State<AppState> boundary: wire
Plan: vibe/2026-09-07-2-gateway-tts-phase-1.md
A dev-only Node script can now build the gateway fresh, boot it with a throwaway speech profile on loopback, and assert the speech and voices surfaces through the gateway's own responses, so the wire contract can be checked against a live provider on demand. The vendor key comes from the process environment only and is ferried to the gateway subprocess, whose configuration interpolation is what ultimately receives it; the script never contacts a provider directly and never prints or persists the key. Every wait is bounded, and the gateway subprocess is terminated on success and on failure. With no key in the environment the probe prints a skip and exits successfully without building or touching the network. A verification note records the probe's contract and assertion set now, defers the live run's provenance and observed dialect to the phase 3 rerun, and nine offline tests drive the probe against a fake gateway double.
- `tools/gateway-tts-live.mjs` always builds the gateway with `cargo build -p gateway` before booting it, so a stale binary can never be probed against a recorded current commit. It boots the fresh binary on an ephemeral loopback port with a throwaway Together-backed profile written into a temp directory that the run deletes afterward.
- `runProbe` takes its collaborators as injected parameters (environment, build, spawn, fetch, and every timeout), so the offline suite drives the full probe against a fake gateway with no build and no network. With no key in the environment it logs a skip and returns 0 before anything builds.
- `renderConfig` writes the vendor credential as the `api_key = "${TOGETHER_API_KEY}"` interpolation reference only, and the probe ferries the key solely into the gateway subprocess environment. The key is never printed, never written to a file, and never sent anywhere except through the gateway.
- `GATEWAY_KEY` is a throwaway shared secret that exists only so the booted gateway serves with authentication enabled; it never leaves the loopback listener and is not a real credential.
- `runSpeechSurface` asserts the speech contract through the gateway's responses: the default call returns `audio/mpeg` with mp3 magic bytes, which proves the structural format pin reached the provider because the provider's own documented default is wav; the response carries no `Content-Length`; a wav request returns `audio/wav` with a RIFF body; and an input carrying `<laugh>` returns 200 with audio.
- `VOICES` names the eight configured Orpheus voices, and the voices assertion requires the union answer to list them sorted as `{"id", "name"}` entries with `name` mirroring `id`.
- `instructions` is probed as the named optional wire field, while `sample_rate` and the bogus `promptforge_probe` ride the verbatim passthrough; all three must come back 2xx.
- `stopGateway` escalates from SIGTERM to SIGKILL inside a bounded grace window, and the suite asserts every spawned gateway child is dead after success, boot failure, and assertion failure alike.
- `tools/gateway-tts-live.test.mjs` runs nine tests against a fake gateway double whose behavior modes arrive through its environment, exactly where the real gateway would find the vendor key. A canary credential and a sink file prove the key reaches the subprocess unmodified while never appearing in the probe's output or in the rendered config.
- `design/note-gateway-tts-phase-1-verification.md` records the probe's run boundary and assertion set now and leaves the commit, tree hash, and observed-dialect lines as placeholders for the phase 3 rerun.
- `UpstreamRateLimited` and `UpstreamUnavailable` envelopes are not provoked on the paid provider unless a cost-free trigger exists; those mappings stay covered by the gateway integration suite.
- `design/note-gateway-tts-phase-1-verification.md` also names what the probe never exercises live: voice rejection, kind mismatch, and mid-stream disconnect, all left to the integration suite.
Design: new pure-function @ tools/gateway-tts-live.mjs::renderConfig deps: port
Design: new oversized-unit @ tools/gateway-tts-live.mjs::runSpeechSurface
Design: new oversized-unit @ tools/gateway-tts-live.mjs::runProbe
Deferred: the live run and its observed-dialect record (commit, tree hash, dialect observations) to the Phase 3 rerun
Plan: vibe/2026-09-07-2-gateway-tts-phase-1.md
The gateway's operator documentation now covers remote speech synthesis: how to declare a speech model, how to call the synthesis route, and how to list voices. An omitted audio format pin resolves to mp3 so a Together-backed model does not silently answer wav, and a missing or invalid content type is labeled as an event stream before any audio type. Together's event-stream mode cannot be requested because the wire type rejects the format it requires, so Together speech stays non-streaming and event-stream forwarding is only forward-looking for providers that already speak that dialect. Later guide chapters shift down by one to make room, the remote and local kind lists include speech, and a commented catalog example shows a remote speech model. Local speech models are documented as refused at launch because no local speech runtime exists yet.
- `crates/gateway/README.md` now lists `POST /v1/audio/speech` and `GET /v1/audio/voices` in the served-route paragraph and adds a Speech synthesis models section with a Together-backed catalog example.
- `guide/src/gateway/06-speech-synthesis.md` is a new chapter inserted after speech-to-text. Chapters 06 through 10 become 07 through 11, `guide/src/SUMMARY.md` and the gateway index list the new chapter, and the assembled gateway guide regenerates with the same shift; the design report's serving-chapter citation follows to `11-serving-and-observing.md`.
- `gateway.local.example.toml` adds a commented Together-backed `[[endpoint]]` and speech `[[model]]` with `voices`, beside the existing commented chat block.
- `guide/src/gateway/03-remote-models.md` adds `speech` to the kind list and to the enumerated spelling vocabulary.
- `POST /v1/audio/speech` is documented as the OpenAI-shaped route: `model`, `input` (non-empty, capped at 4096 characters), and `voice` (a name or `{"id": "..."}`) are required; omitted `response_format` resolves to `mp3` in the wire type; `speed`, `instructions`, and `stream_format` are optional, and unnamed fields pass through verbatim.
- `text/event-stream` is the documented Content-Type fallback when the upstream type is missing or invalid and the framing selector is SSE, so an event stream is never labeled as an audio type; only then does the requested format's MIME type apply, and `Content-Length` is never set.
- `stream_format` is forwarded, but Together's `stream=true` mode requires `response_format = "raw"`, which the wire enum rejects, so Together SSE cannot be requested and Together speech is non-streaming; the forwarding is forward-looking for SSE-capable OpenAI-compatible providers.
- `GET /v1/audio/voices` is documented as the active profile's deduplicated, sorted union of id-first `{"id", "name"}` entries, with `name` mirroring `id`.
- `guide/src/gateway/04-local-models.md` states a `speech` kind has no local serving mode and is refused at launch: no local speech runtime exists yet.
- `stream_format = "sse"` is forwarded with no reframing or decoding: whatever framing the provider answers with passes through untouched.
Design: new clone-block @ crates/gateway/README.md
Design: new clone-block @ guide/src/gateway/06-speech-synthesis.md
Design: new shotgun-surgery @ guide/src/gateway/06-speech-synthesis.md
Plan: vibe/2026-09-07-2-gateway-tts-phase-1.md
The gateway now carries a standalone record of the speech synthesis design that actually shipped, so later work can navigate the as-built choices rather than the original report. Speech is a routed catalog kind with remote passthrough, inline handlers, authentication before body parse, voice checks before queue admission, a bounded background relay for unread audio, split header and body deadlines, and speech-only upstream rate-limit envelopes. Local launch refuses the kind before any provision. A gateway-only Node probe takes the vendor key from the process environment and ferries it only to the gateway subprocess. Catalog examples use the shipped interpolation form, and live-provider verification remains for a later rerun.
- `design/design-gateway-tts-phase-1.md` records sixteen as-built choices, naming preceding commits `05dd8eda`, `5392c357`, `ffc535eb`, `6506f4ca`, `b32c64c6`, `738cfc7c`, `7f3cf607`, and `9153cd7c` as the series this document closes.
- `ModelKind::Speech` is recorded as a fourth routed catalog kind with `audio_speech` and `audio_voices` inline in the gateway crate. An engine-slot catalog and a `gateway-tts` service crate are recorded as rejected; branch `gate-way-tts-phase-1` was discarded, not ported.
- `relay_speech_stream` is recorded as a bounded background relay that owns the upstream body, the dominion permit, and the cancellation guard, with named ceilings on total lifetime, response bytes, upstream idle after headers, and blocked downstream delivery. Every terminal path emits one error item so cancellation never surfaces as a clean end of stream.
- `send_speech` is recorded as owning both the first-response deadline and the per-read body idle deadline; `audio_streaming_client` carries connect timeout and keepalive only.
- `ProtocolError::classify` is recorded as frozen; speech maps upstream 429 and 503 through `GatewayError::UpstreamRateLimited` and `GatewayError::UpstreamUnavailable` only.
- `serve_mode_for` is recorded as a side-effect-free preflight so a local speech model fails before any server, model, companion, or cache side effect.
- `check_auth` is recorded as running before body extraction, and catalog voice membership as checked before queue admission so a client error never occupies a queue slot.
- `tools/gateway-tts-live.mjs` is recorded as a gateway-only Node probe: the vendor key comes from the process environment and is ferried only to the gateway subprocess, whose throwaway profile uses `api_key = "${TOGETHER_API_KEY}"` interpolation and never a direct vendor call.
- `ServeMode::Speech` is absent, as are a local engine, encoder and WAV-header policy, sampling pins, Together SSE reframing, Workshop playback, the config-UI voice picker, text conditioning, and ElevenLabs and Baseten adapters. Live-provider verification remains for the Phase 3 rerun; this document names no live commit or tree hash.
Plan: vibe/2026-09-07-2-gateway-tts-phase-1.md
close cppalliance#21 A kind switch away from speech hides the voices control but keeps its value in the draft, and the server rejects a non-speech model that carries voices, so the save failed with no visible field to clear. The new stripKindBoundFields runs in buildSavePayload and deletes voices from the payload when the effective kind is not speech. - \stripKindBoundFields\ runs on both payload paths: the draft branch that pushes \entry.data\ and the edit branch that writes the edits map into the loaded entry. - The value stays in the draft after the strip, so toggling the kind back to speech before Save restores the voices control with its chips intact. - The new model-detail test picks speech, adds a chip, switches back to chat, and pins that the PUT body carries kind chat and no voices key.
c4ec890 to
b9b7c50
Compare
2 participants
Closes #20