fix(runtime): LLM + embedding drivers set no HTTP timeout — hangs make the fallback chain unreachable

[Nideesh1]

Root cause

The LLM driver HTTP clients never set a timeout. reqwest applies no timeout by default, so a provider that accepts the TCP connection and then never answers hangs the agent forever.

Observed in production: an agent sat 15 minutes on a single request and made zero tool calls, while the same model answered a direct probe in seconds.

Why this is worse than a slow request

A hang never returns Err. FallbackDriver only advances on Err. So the fallback chain is unreachable precisely when it is needed — the mechanism built to survive a bad provider is disabled by the exact failure mode it exists for.

Converting a hang into an error is what makes fallback work at all. That is the real fix here; the timeout is just the means.

Audit: every non-LLM client already had timeouts

Client	File	Timeout before this PR
openai	drivers/openai.rs (2 builders)	❌ none
anthropic	drivers/anthropic.rs	❌ none
gemini	drivers/gemini.rs	❌ none
bedrock	drivers/bedrock.rs	❌ none
vertex	drivers/vertex.rs	❌ none (also no USER_AGENT)
embedding	embedding.rs	❌ none (also no USER_AGENT)
web_search	web_search.rs	✅ timeout(15s)
web_fetch	web_fetch.rs	✅ timeout(config.timeout_secs)
a2a	a2a.rs	✅ timeout(30s)
provider_health	provider_health.rs	✅ connect_timeout + timeout
copilot (token exchange)	drivers/copilot.rs	✅ timeout(10s)
copilot (device flow)	drivers/copilot.rs	✅ timeout(30s)
tool_runner	tool_runner.rs	✅ timeout(30s) / timeout(15s)

The only clients without a timeout were the ones that gate every single agent turn.

The last two were nearly missed

vertex.rs and embedding.rs construct their client with bare reqwest::Client::new() rather than Client::builder(), so a search for builder() sites did not surface them. Same bug, same consequence — vertex.rs in particular is a full chat-completion driver with the identical agent-hang exposure, so leaving it for a follow-up would have shipped a fix that still left the hang open. Both are folded in here.

The fix

Three constants in openfang-runtime/src/lib.rs:

• LLM_CONNECT_TIMEOUT_SECS = 15 — TCP + TLS establishment only. An unreachable provider should fail over immediately rather than stalling a turn.
• LLM_READ_TIMEOUT_SECS = 300 — stall detector for chat completions.
• EMBEDDING_READ_TIMEOUT_SECS = 60 — stall detector for embeddings (new; see below).

Applied to all 7 client constructions (openai ×2, anthropic, gemini, bedrock, vertex, embedding). CopilotDriver delegates completions to OpenAIDriver (make_inner_driver), so it inherits the fix.

vertex.rs was additionally the only driver not sending .user_agent(crate::USER_AGENT). That constant's doc comment records that some providers (Moonshot, Qwen) reject requests arriving without one — a separate latent bug, fixed in the same edit. embedding.rs was likewise missing it and now sets it too.

Why embeddings get a shorter read timeout

300s is calibrated for a chat completion, which has a generation phase and can legitimately go quiet for a while mid-stream. An embedding call has no generation phase, returns a fixed-size vector, and normally completes in well under a second — a 60s silence already means the provider is not coming back.

Failing fast is also cheap here, unlike a chat completion. Embeddings only upgrade memory recall to vector similarity search. agent_loop already handles embed_one returning Err by logging "Embedding recall failed, falling back to text search" and continuing with keyword recall. Waiting the full 300s would stall the turn for five minutes before doing the exact work the existing fallback would have done immediately.

Why read_timeout and not timeout

timeout caps total request duration. Setting it would kill long legitimate generations and long streams — a reasoning model or a large streamed response would be terminated mid-flight for doing nothing wrong.

read_timeout bounds the gap between successive bytes. A connection that keeps producing output can run as long as it likes; a connection that goes silent fails. That is exactly the distinction between "slow" and "dead", and only the second one should be an error.

300s is deliberately under DEFAULT_AGENT_TURN_TIMEOUT_SECS (600) so the driver gives up first, not the turn supervisor. That keeps the error attributable to the provider and leaves the fallback chain enough remaining budget to actually run.

⚠️ Not compiled locally — CI must verify

This was not built or tested locally. The only available host is a live Kubernetes control-plane node (15 GB RAM, no swap) where a previous cargo run OOM-killed kube-apiserver. Running a workspace build there risks taking down the cluster.

What was verified without compiling:

• cargo fmt --all runs clean and made no changes to the inserted code
• builder / user_agent / connect_timeout / read_timeout counts match exactly per file — openai 2/2/2/2, anthropic 1/1/1/1, gemini 1/1/1/1, bedrock 1/1/1/1, vertex 1/1/1/1, embedding 1/1/1/1
• no reqwest::Client::new() remains anywhere under drivers/ or in embedding.rs
• all three constants are pub const in crates/openfang-runtime/src/lib.rs, so crate:: paths resolve from both the drivers submodule and embedding
• Cargo.lock pins reqwest 0.12.28; ClientBuilder::read_timeout exists on the non-wasm client since 0.12.5

CI is the source of truth for build/test/clippy on this PR.

🤖 Generated with Claude Code