feat(sbd-v6): declare and record the AgentX grading axis

[luochen-amd]

What

Makes an AgentX session's grading axis visible in session_breakdown.json, and records it when the run resolves it rather than re-deriving it at export.

Why

#1362 made an AgentX replay rank on total token throughput under an interactivity veto; a synthetic run still ranks on output throughput alone. On the canonical corpus the two differ by roughly two orders of magnitude — a measured Kimi-K3 baseline read 25978 tok/s total against 183 tok/s output.

Every throughput field in the breakdown is the output axis by construction, and nothing in the document said which axis the gains were on. A consumer had no way to keep an AgentX session out of a synthetic leaderboard: it would sort one against the other and every number would look plausible.

The block

"metadata": {
  "grading": {
    "benchmark_mode": "agentx",
    "objective": "total_throughput",
    "intvty_veto": {"enabled": true, "noise_pct": 5.0},
    "degrade_reason": null
  }
}

objective is what the gains in this document are actually on, not what was requested. A session that asked for the total axis but never measured it graded on output, and reads output_throughput here with degrade_reason naming the gap. Reading it the other way would let a degraded run pass as a comparable AgentX result.

outcome carries the four AgentX axes at both ends — total and input throughput, interactivity p90, TPOT p90 — plus graded_on on final and on validation. The reconciliation has to be single-axis: an attributed figure on one axis against an unattributed figure on another makes the gap meaningless.

Two deliberate choices:

• Purely additive. throughput_tok_s_per_gpu, output_throughput and every other existing key keep their output-axis meaning. No existing consumer changes.
• Unmeasured axes are explicit nulls. Absent would be indistinguishable from an axis the framework failed to report; zero reads as "measured, and it was zero". (TPOT p90 was already parsed out of the benchmark report and discarded at five call sites — it now lands.)

outcome resolves the axis through the same helper metadata.grading uses, so the two blocks cannot drift into naming different axes for one session. There is a test that locks it.

Recorded, not re-derived

Resolving the axis reads HYPERLOOM_PERF_METRIC / HYPERLOOM_PERF_NOISE_PCT. The breakdown is exported from CLOSE — frequently a subprocess that never inherited them — so deriving it there can name an axis the session never graded on. An operator re-exporting an old session later would get whatever their shell happens to hold. The failure is silent: the block looks well-formed and states the wrong axis.

So the axis is resolved once at seed and recorded in SharedState.grading, next to benchmark_mode and agentx_epoch, which live there for exactly the same reason. The collector prefers what was recorded. Its fallback for older sessions derives from recorded facts only — benchmark_mode and the framework — and reports a null veto band, because the band was never recorded and the current environment is not evidence of what that session applied. The override still works in both directions; it is resolved where the run can see it.

This removes the last os.environ read from the V6 collectors.

What stays a projection is the part genuinely derived from recorded facts: whether the session degraded because its baseline never carried the axis pair. That reads state.baseline_perf, so it is stable at any export time.

Test

14 cases in test_sbd_v6_grading.py (13 pass, 1 skipped where fcntl is missing — importing the CLI package pulls it in, and the rest of the module stays importable on any platform). They cover both axes, the scriptable-framework carve-out, the degrade path, recorded-wins-over-env, and the metadata/outcome agreement lock.

Verified against the same suites on a clean tree: identical results, so the failures present locally are the pre-existing POSIX-only fcntl imports rather than anything here. ruff check src/hyperloom/inference_optimizer and ruff format are clean.

Rollout

Additive to the wire shape, so nothing downstream has to change to keep working. Cortex cannot store the new axes yet — there is no column for total throughput or interactivity anywhere — so the immediate value is that an AgentX session becomes identifiable, which is the prerequisite for keeping it out of a leaderboard it is not comparable on.

[github-actions]

CI E2E report — ❌ Timeout

item	value
result	❌ Timeout
model	Qwen/Qwen3-0.6B (dense)
resources	1× GPU, TP=1
PR branch	feat/sbd-agentx-grading-axes
commit	acaf3af96ae52fff95c913be6d02b5e3badc047a
session_id	23d64dce-c359-459c-bccb-d3855b0c23d0
queue → dispatch	24m 1s
run time	67m 41s
total	91m 42s
reason	Timed out — the run never reached a terminal state in time (task stuck, or the GPU stayed queued too long).
detail	not terminal after 13200s

details