Skip to content

fix: expose safe composer prediction diagnostics - #437

Merged
iililiiilili merged 1 commit into
mainfrom
fix/composer-prediction-diagnostics
Nov 2, 2026
Merged

iililiiilili merged 1 commit into
mainfrom
fix/composer-prediction-diagnostics

Conversation

@iililiiilili

Copy link
Copy Markdown
Collaborator

Related: #418

What Problem This Solves

An enabled composer can quickly return {"text":null} when inference is skipped, a provider request fails, or output is rejected. These distinct outcomes were indistinguishable in browser developer tools and had no safe terminal log. Increasing the timeout cannot diagnose a response that finishes in about one second.

Why This Change Was Made

The admin HTTP plugin now records a fixed reason at the inference/admission boundary. The authenticated Web BFF adds its elapsed time and reports the same diagnostic fields in one terminal log. Numeric providerStatus is included only for an unsuccessful provider HTTP response; raw provider bodies and exception messages remain excluded.

The BFF keeps the final transcript/Space ACL recheck before publishing or logging the outcome. A single terminal latch prevents timeout, disconnect, and late completion from producing contradictory or duplicate records. Incoming diagnostic strings are allowlisted; arbitrary fields never reach the log. Existing prediction text remains usable during rolling Gateway updates, with an explicit diagnostics_unavailable reason when the producer does not supply a recognized code.

User Impact

In browser developer tools, open Network before sending a new chat message, filter composer-predictions, select Payload action: "predict", then inspect Response. A skipped prediction can now report:

{"text":null,"reason":"model_busy","elapsedMs":1180}

This is an example, not a diagnosis of the reported deployment. model_busy describes the Gateway's ordinary-call tracking, not measured GPU utilization. Provider failures instead show a code such as provider_http_error and a numeric providerStatus.

The deployment needs the updated Web ingress and Gateway. No new configuration or environment variable is required. Browser presentation, server/account defaults, model selection, private-address/DNS pinning, no-retry policy, ordinary-call priority, timeout budgets, and context/output limits stay the same. Logs contain no conversation text, user/session identifiers, destination URLs, credentials, or raw errors.

Evidence

  • Focused repository wrappers on the trusted Windows source: 72 tests across three files, including the actual HTTP BFF boundary, plugin diagnostics, and existing endpoint/config contracts.
  • The only correction after that pass was a synthetic URL fixture: its password is explicitly redacted while retaining the same userinfo-rejection assertion. No scanner suppression, baseline, ignore, or gate was changed.
  • Final Codex autoreview (--mode uncommitted --engine codex): exit 0, no accepted/actionable P0 findings. TruffleHog passed. The sparse review copy's full-index patch and all five changed file hashes match the implementation checkout.
  • Linux Node24.18.0/pnpm11.15.1: real HTTP BFF14 + plugin48 focused tests passed. After the fixture-only correction, native admin-http-rpc quick passed typed lint (zero warnings/errors), production/test tsgo, and95tests across8files including finalplugin48/index10. Control-plane lint and production tsgo passed; both mapped package builds passed. The whole owner quick exited124 at its600-second validation bound while the broad control-plane suite was still running (122completedfiles/1785individualpasses/no assertion-failure markers); that suite is incomplete, not counted as passing. No known-failing full SDK preparation loop was repeated.
  • Existing hosted CI must provide full SDK/type/lint/build evidence at the pushed head. Local mandatory SDK declaration preparation has a documented 300-second resource limit; no timeout extension, fabricated freshness stamp, or raw-linter bypass is used.
  • The actual company provider and deployed image are not accessible from this environment. The reported text:null is not claimed to be fixed or attributed to a specific provider failure; this patch exposes the missing diagnostic facts.

Production delta: +190/-56 lines (net +134), justified by observable fixed outcomes, the HTTP diagnostic contract, and safe single-terminal logging. Tests: +338/-41 (net +297); docs: +38. Three additive HTTP diagnostic fields (reason, elapsedMs, optional providerStatus); zero new config/env/dependency/storage/version surfaces. Codex review isolation flags were checked directly in sibling codex-rs/exec/src/cli.rs:36.

Named follow-ups, not asserted causes of the incident:

  • Reliable Composer admission lifecycle. Model-call notifications use a bounded hook queue that can drop a completion (src/agents/embedded-agent-runner/run/attempt.model-diagnostic-events.ts:525, src/hooks/fire-and-forget.ts:154). Investigate the authoritative lifecycle owner before changing admission semantics; this patch only reports the current tracking decision.
  • Composer API selection versus ordinary model defaults. The ordinary configured-provider resolver can select openai-completions for a base-URL-only provider (src/agents/embedded-agent-runner/model.configured-overrides.ts:81); Composer currently requires an explicit supported adapter. Verify intent and the actual unsupported_api diagnostic before changing that policy.

@iililiiilili
iililiiilili marked this pull request as ready for review November 2, 2026 06:06
@iililiiilili
iililiiilili merged commit 318b75d into main Nov 2, 2026
107 of 108 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant