[faresobeid]

When unset, inference.vllm.api_server_count now defaults to max(4, local DP size), the same rule everywhere. LoRA still forces 1.

Why: with one API server, all request parsing and response serialization runs in a single process, and its GC pauses stall /health and /metrics. Measured on GLM-4.5-Air TP8, 1 node, vLLM 0.31, 130 closed-loop /inference/v1/generate requests:

arm	API procs	gen tok/s	/health p99 / max s
main	1	5395	4.97 / 29.3
main	4	5487	0.19 / 1.8
#3900	1	5524	0.22 / 1.7
#3900	4	5497	0.02 / 0.1
#3900	8	5540	0.03 / 0.1

8 adds nothing over 4.

/metrics with 4 API servers (vLLM multiprocess Prometheus) was checked against scheduler stats recorded inside EngineCore, while the orchestrator's InferenceMetricsCollector + ConcurrencyController drove the load (600 s per arm):

• One series per family (engine="0"), no per-process duplicates.
• num_requests_running / num_requests_waiting match engine truth within 2 (p99 1); kv_cache_usage_perc within 0.05 (p99 0.025). With 1 API server the max error was 56 / 42, from stale reads during GC stalls.
• num_preemptions_total and prefix-cache counters equal engine truth; request_success_total, TTFT count and generation_tokens_total are counted once per request.
• The controller made the same decisions in both arms.
• Only vllm:iteration_tokens_total (W&B passthrough, not used by the controller) changes: its _count is ~3x because each API server observes its own share of every engine step.

Admin routes (/pause, /resume, /update_weights, /init_broadcaster) act on EngineCore, so they work from any API server.

Config changes:

• VllmConfig.auto_setup_api_server_count: unset -> max(4, local DP size). The RL/SFT single-node DP fills (api_server_count < dp -> dp) then also resolve to max(4, dp).
• Multi-node wide-EP fill: api_server_count = max(4, inferred_dp_local).
• RL/SFT set api_server_count = 1 where they turn on inference.vllm.enable_lora (that happens after VllmConfig validation).
• Multi-node / disaggregated fills now raise api_server_count to dp_per_node when it is lower, instead of only when it equals 1. The per-rank launchers still pass --vllm.api-server-count 1, so multi-node runtime is unchanged.

Resolved api_server_count over all configs/ + examples/ TOMLs: 28 configs change: 27 single-engine non-LoRA configs (1 -> 4) and one DP=2 config (2 -> 4). LoRA, DP>=4 and disaggregated configs are unchanged.

Tests: pytest tests/unit/orchestrator tests/unit/test_configs.py tests/unit/inference -m "not gpu" (316 passed), ruff check, ruff format --check.