feat(agentx): aggregate full-response ITL and interactivity - #2504
Conversation
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline finding, I checked two other candidate issues in the new aggregation and ruled both out: stats dropping when request start/end ns metadata is missing, and degenerate zero/non-positive values not being filtered — neither holds up against the actual aggregation logic.
Extended reasoning...
Two additional candidate issues around the new full_response_output_token_throughput_per_user aggregation were examined this run and ruled out: (1) concern that records missing request_start_ns/request_end_ns metadata would silently drop the new stat — this is not the case, extract_per_record_floats/stats_for operate independently of the start/end-ns filtering used for duration, so the new metric's per-record extraction is unaffected by that gate; (2) concern that degenerate zero/non-positive full_response_output_token_throughput_per_user values would skew mean/percentiles unfiltered — to_float only excludes None/non-numeric values, but this mirrors how input_tokens/output_tokens and other existing throughput stats in this same file are handled (no zero-filtering there either), so this is consistent with existing conventions rather than a new defect.
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
cquil11
changed the title
cquil11
added
agentx-fast
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
d368c20 to
1985ac7
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31045023380 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31045242272 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31052700355 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31129833729 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31129988234 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31133509890 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31133509890 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31143552329 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31143750490 |
|
/stage-results 31143750490 |
|
@cquil11 |
|
/stage-results 31143750490 |
|
@cquil11 staged run 31143750490: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-07~r31143750490 This run remains available across future @cquil11 已将运行 31143750490 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-07~r31143750490 后续的 |
cquil11
removed
sweep-enabled
full-sweep-enabled
agentx-fast
Resolves the perf-changelog.yaml conflict by keeping both sides: the branch's dsv4-fp4-mi355x-vllm-agentic-mtp DEP8 entry and main's dsr1/dsv4-b200/gb300/glm5.2/qwen3.5 entries are independent appends at the same list position. Brings in the AIPerf submodule bump past agentx-v1.0.1 (b7b16cf, #2484) to main's current 743fb82 (#2504, full-response ITL + interactivity aggregation).
1 participant
Summary
agentx-v1.0.2at the merged full-response implementationfull_response_inter_token_latencyfrom AgentX per-request profile recordsrequest_metrics.latency.full_response_itlrequest_metrics.latency.full_response_intvtyWhy
AgentX can receive a server-reported raw token count even when a structured-output parser stops emitting visible content before generation completes. The existing ITL then mixes the raw token count with a shorter parsed-content interval.
The full-response ITL uses the client-observed interval through HTTP response completion. InferenceX derives each interactivity statistic as the reciprocal of its matching ITL statistic, consistent with the existing
itlandintvtyfields. For example, full-response p75 interactivity is1 / p75(full-response ITL), preserving slow-tail percentile semantics.Validation
PYTHONPATH=. uv run pytest -q utils/agentic/aggregation: 43 passedgit diff --check: passedDependencies and merge order
agentx-v1.0.2)