[H3 CI]: add video serving metrics and GPU energy [DO NOT MERGE] / 新增视频服务指标与 GPU 能耗（暂勿合并）

.github/workflows/e2e-tests.yml

@@ -7,6 +7,21 @@ permissions:
 on:
     workflow_dispatch:
         inputs:
+            h3-video:
+                description: "Run only the configured H3 smoke; manual dispatch, no LLM sweep"
+                required: false
+                type: boolean
+                default: false
+            h3-reuse-run-ids:
+                description: "Reprocess one or two accepted H3 CI runs (comma-separated); no new H3 generation"
+                required: false
+                type: string
+                default: ""
+            h3-inventory-run-id:
+                description: "Reuse a successful H3 hardware inventory; export on CPU without allocating GPUs"
+                required: false
+                type: string
+                default: ""
             generate-cli-command:
                 description: "Command passed to generate matrix script"
                 required: false
@@ -102,6 +117,21 @@ on:
                 default: "[]"
     workflow_call:
         inputs:
+            h3-video:
+                description: "Run only the configured H3 smoke; manual dispatch, no LLM sweep"
+                required: false
+                type: boolean
+                default: false
+            h3-reuse-run-ids:
+                description: "Reprocess one or two accepted H3 CI runs (comma-separated); no new H3 generation"
+                required: false
+                type: string
+                default: ""
+            h3-inventory-run-id:
+                description: "Reuse a successful H3 hardware inventory; export on CPU without allocating GPUs"
+                required: false
+                type: string
+                default: ""
             generate-cli-command:
                 description: "Command passed to generate matrix script"
                 required: false
@@ -197,7 +227,18 @@ on:
                 default: "[]"
 
 jobs:
+    h3-video:
+        if: ${{ inputs.h3-video }}
+        permissions:
+            contents: read
+            actions: read
+        uses: ./.github/workflows/h3-video.yml
+        with:
+            source-run-ids: ${{ inputs.h3-reuse-run-ids }}
+            inventory-run-id: ${{ inputs.h3-inventory-run-id }}
+
     get-jobs:
+        if: ${{ !inputs.h3-video }}
         runs-on: ubuntu-latest
         outputs:
             single-node-config: ${{ steps.get-jobs.outputs.single-node-config }}
@@ -753,7 +794,7 @@ jobs:
 
     calc-success-rate:
         needs: [collect-results, collect-evals]
-        if: ${{ always() }}
+        if: ${{ always() && !inputs.h3-video }}
         runs-on: ubuntu-latest
 
         env:

.github/workflows/h3-video.yml

@@ -0,0 +1,212 @@
+name: H3 Video Smoke
+run-name: H3 video smoke - ${{ github.ref_name }}
+
+on:
+  workflow_dispatch:
+    inputs:
+      inventory-run-id:
+        description: "Reuse a successful hardware inventory for CPU-only export"
+        required: false
+        type: string
+        default: ""
+      source-run-ids:
+        description: "One or two accepted H3 run IDs to reprocess; empty runs new generation"
+        required: false
+        type: string
+        default: ""
+  workflow_call:
+    inputs:
+      inventory-run-id:
+        required: false
+        type: string
+        default: ""
+      source-run-ids:
+        required: false
+        type: string
+        default: ""
+
+permissions:
+  contents: read
+  actions: read
+
+concurrency:
+  group: h3-video-${{ github.repository }}
+  cancel-in-progress: false
+
+jobs:
+  prepare:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    env:
+      H3_SOURCE_SHA: ${{ github.sha }}
+    outputs:
+      priority: ${{ steps.queue.outputs.priority }}
+      queue-token: ${{ steps.queue.outputs.queue-token }}
+    steps:
+      - name: Authorize manual repository execution
+        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+        with:
+          script: |
+            if (context.eventName !== 'workflow_dispatch' ||
+                context.repo.owner !== 'SemiAnalysisAI' || context.repo.repo !== 'InferenceX' ||
+                !context.ref.startsWith('refs/heads/')) {
+              throw new Error('H3 smoke requires manual dispatch of an InferenceX repository branch.');
+            }
+            for (const username of new Set([context.actor, process.env.GITHUB_TRIGGERING_ACTOR])) {
+              const {data} = await github.rest.repos.getCollaboratorPermissionLevel({
+                ...context.repo, username,
+              });
+              if (!['write', 'maintain', 'admin'].includes(data.permission)) {
+                throw new Error(username + ' must have write, maintain, or admin permission.');
+              }
+            }
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        with:
+          ref: ${{ github.sha }}
+          persist-credentials: false
+      - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+      - name: Prepare native queue identity
+        id: queue
+        env:
+          H3_SITE_CONFIG: ${{ vars.H3_SITE_CONFIG }}
+          PRIORITY_ENABLED: ${{ vars.PRIORITY_SCHEDULER_ENABLED }}
+          NODE_SLOTS_ENABLED: ${{ vars.NODE_SLOT_SCHEDULER_ENABLED }}
+          H3_SOURCE_RUN_IDS: ${{ inputs.source-run-ids }}
+          H3_INVENTORY_RUN_ID: ${{ inputs.inventory-run-id }}
+        run: |
+          set -euo pipefail
+          if [[ -n "$H3_INVENTORY_RUN_ID" && ( -z "$H3_SOURCE_RUN_IDS" || ! "$H3_INVENTORY_RUN_ID" =~ ^[1-9][0-9]{0,19}$ ) ]]; then
+            echo 'Hardware reuse needs one inventory run ID and explicit source executions.' >&2
+            exit 1
+          fi
+          [[ "$H3_SITE_CONFIG" = /* ]] || { echo 'Set H3_SITE_CONFIG to the reviewed runner-local JSON path.' >&2; exit 1; }
+          if [[ -n "$H3_SOURCE_RUN_IDS" && ! "$H3_SOURCE_RUN_IDS" =~ ^[1-9][0-9]{0,19}(,[1-9][0-9]{0,19})?$ ]]; then
+            echo 'Expected one or two comma-separated source run IDs.' >&2
+            exit 1
+          fi
+          if [[ "$PRIORITY_ENABLED" == true && "$NODE_SLOTS_ENABLED" != true ]]; then
+            echo 'H3 priority scheduling requires node-slot admission for nodes:1.' >&2
+            exit 1
+          fi
+          scored=$(printf '%s' '[{"runner":"cluster:h200-dgxc","framework":"sglang","node-count":1}]' |
+            uv run --no-project --with pyyaml --python 3.12 utils/ci_priority.py)
+          echo "priority=$(jq -r '.[0].priority' <<<"$scored")" >> "$GITHUB_OUTPUT"
+          python3 - <<'PY'
+          import hashlib
+          import os
+          identity = f"{os.environ['GITHUB_RUN_ID']}:{os.environ['GITHUB_RUN_ATTEMPT']}:h3-video"
+          token = hashlib.sha256(identity.encode()).hexdigest()[:32]
+          with open(os.environ['GITHUB_OUTPUT'], 'a') as output:
+              output.write(f"queue-token={token}\n")
+          PY
+
+  smoke:
+    needs: prepare
+    outputs:
+      mode: ${{ steps.execute.outputs.mode }}
+    if: ${{ inputs.inventory-run-id == '' }}
+    name: p${{ needs.prepare.outputs.priority }} | ${{ inputs.source-run-ids != '' && 'H3 H200 hardware inventory' || 'H3 video H200 smoke' }}
+    runs-on: >-
+      ${{ fromJSON(
+        vars.PRIORITY_SCHEDULER_ENABLED == 'true' &&
+        format('["self-hosted","cluster:h200-dgxc","nodes:1",{0},{1}]',
+          toJSON(format('ci-job-{0}-{1}', needs.prepare.outputs.priority, needs.prepare.outputs.queue-token)),
+          toJSON(format('ci-attempt-{0}', github.run_attempt))) ||
+        '["cluster:h200-dgxc"]'
+      ) }}
+    timeout-minutes: 105
+    env:
+      H3_SITE_CONFIG: ${{ vars.H3_SITE_CONFIG }}
+      H3_SOURCE_SHA: ${{ github.sha }}
+      H3_REPOSITORY: ${{ github.repository }}
+      H3_RUN_ID: ${{ github.run_id }}
+      H3_RUN_ATTEMPT: ${{ github.run_attempt }}
+      H3_WORKFLOW_REF: ${{ github.workflow_ref }}
+      H3_WORKFLOW_SHA: ${{ github.workflow_sha }}
+      H3_ACTOR: ${{ github.actor }}
+      H3_TRIGGERING_ACTOR: ${{ github.triggering_actor }}
+    defaults:
+      run:
+        working-directory: h3-video-source-${{ github.run_id }}-${{ github.run_attempt }}
+    steps:
+      - name: Checkout exact dispatched source
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        with:
+          ref: ${{ github.sha }}
+          path: h3-video-source-${{ github.run_id }}-${{ github.run_attempt }}
+          persist-credentials: false
+      - name: Execute configured H3 smoke
+        id: execute
+        if: ${{ inputs.source-run-ids == '' }}
+        run: |
+          set -euo pipefail
+          python3 - <<'PY'
+          import json, os, sys
+          sys.path.insert(0, 'experimental/video-generation')
+          from ci import validate_config
+          with open(os.environ['H3_SITE_CONFIG']) as stream:
+              config = validate_config(json.load(stream))
+          with open(os.environ['GITHUB_OUTPUT'], 'a') as stream:
+              stream.write('mode=' + config['mode'] + '\n')
+          PY
+          python3 experimental/video-generation/ci.py \
+            --config "$H3_SITE_CONFIG" \
+            --output "$RUNNER_TEMP/h3-video-$H3_RUN_ID-$H3_RUN_ATTEMPT"
+      - name: Inspect current H200 hardware and power configuration
+        if: ${{ inputs.source-run-ids != '' }}
+        env:
+          GH_TOKEN: ${{ github.token }}
+          H3_SOURCE_RUN_IDS: ${{ inputs.source-run-ids }}
+        run: |
+          set -euo pipefail
+          export PYTHONPATH="$PWD"
+          python3 experimental/video-generation/inventory_ci.py \
+            --config "$H3_SITE_CONFIG" \
+            --source-run-ids "$H3_SOURCE_RUN_IDS" \
+            --output "$RUNNER_TEMP/h3-video-$H3_RUN_ID-$H3_RUN_ATTEMPT"
+      - name: Preserve H3 evidence, including incomplete attempts
+        if: ${{ always() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: ${{ inputs.source-run-ids != '' && 'h3-hardware' || 'h3-video' }}-${{ github.run_id }}-${{ github.run_attempt }}
+          path: ${{ runner.temp }}/h3-video-${{ github.run_id }}-${{ github.run_attempt }}/
+          if-no-files-found: error
+          compression-level: 0
+          retention-days: 14
+
+  export:
+    needs: [prepare, smoke]
+    if: ${{ always() && needs.prepare.result == 'success' && needs.smoke.outputs.mode != 'serving-smoke' && (needs.smoke.result == 'success' || (inputs.inventory-run-id != '' && needs.smoke.result == 'skipped')) }}
+    name: Verify and export retained H3 results
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    env:
+      PYTHONPATH: ${{ github.workspace }}
+      GH_TOKEN: ${{ github.token }}
+      H3_SOURCE_RUN_IDS: ${{ inputs.source-run-ids || github.run_id }}
+      H3_REUSE: ${{ inputs.source-run-ids != '' }}
+      H3_INVENTORY_RUN_ID: ${{ inputs.inventory-run-id || github.run_id }}
+    steps:
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        with:
+          ref: ${{ github.sha }}
+          persist-credentials: false
+      - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+      - name: Verify original execution and publish versioned measurements
+        run: |
+          set -euo pipefail
+          hardware_args=()
+          if [[ "$H3_REUSE" == true ]]; then hardware_args=(--hardware-run-id "$H3_INVENTORY_RUN_ID"); fi
+          uv run --no-project --python 3.12 --with 'av==16.1.0' --with 'numpy==2.3.5' \
+            python experimental/video-generation/export_ci.py \
+            --source-run-ids "$H3_SOURCE_RUN_IDS" "${hardware_args[@]}" \
+            --output "$RUNNER_TEMP/h3-results"
+      - name: Preserve exported results and failure evidence
+        if: ${{ always() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: h3-results-${{ github.run_id }}-${{ github.run_attempt }}
+          path: ${{ runner.temp }}/h3-results/
+          if-no-files-found: error
+          compression-level: 0
+          retention-days: 14

.github/workflows/test-h3-video.yml

@@ -0,0 +1,47 @@
+name: Test H3 Video
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]
+    paths:
+      - 'experimental/video-generation/**'
+      - 'utils/aggregate_power.py'
+      - '.github/workflows/h3-video.yml'
+      - '.github/workflows/test-h3-video.yml'
+      - '.github/workflows/e2e-tests.yml'
+  push:
+    paths:
+      - 'experimental/video-generation/**'
+      - 'utils/aggregate_power.py'
+      - '.github/workflows/h3-video.yml'
+      - '.github/workflows/test-h3-video.yml'
+      - '.github/workflows/e2e-tests.yml'
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    if: ${{ github.event.pull_request.draft != true }}
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        with:
+          persist-credentials: false
+      - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+      - name: Test CPU harness and CI control
+        working-directory: experimental/video-generation
+        env:
+          PYTHONPATH: ${{ github.workspace }}
+        run: >-
+          uv run --no-project --python 3.12
+          --with 'av==16.1.0' --with 'numpy==2.3.5'
+          --with 'pytest>=8,<9' --with 'jsonschema>=4,<5' python -m pytest -q
+      - name: Check entry shell syntax
+        run: bash -n experimental/video-generation/runtime-entry.example.sh
+      - name: Check workflow wiring
+        run: >-
+          go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.12 -shellcheck=
+          .github/workflows/h3-video.yml .github/workflows/test-h3-video.yml
+          .github/workflows/e2e-tests.yml

docs/configuration-procedures.md

@@ -257,6 +257,20 @@ Sources: [`AGENTS.md#non-negotiable-benchmark-invariants`](../AGENTS.md#non-nego
 6. If the file conflicts with `main`, restore the current `main` version and re-append only this branch's entries. Do not hand-merge reordered history.
 7. Parse the file and confirm the generated changelog selection includes the intended keys before requesting a sweep.
 
+### Separately dispatched experimental workflows
+
+An experimental benchmark outside the LLM master configs can record its change without selecting LLM jobs:
+
+```yaml
+- config-keys: []
+  workflow-dispatch: h3-video.yml
+  description:
+    - "Add a manually dispatched H3 video benchmark"
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX
+```
+
+`workflow-dispatch` must be a local `.yml` or `.yaml` basename under `.github/workflows/`, and that file must declare a `workflow_dispatch` trigger. It requires empty `config-keys` and rejects active LLM eval, scenario, or `append-only` modifiers. Ordinary entries still require real master-config keys. The changelog processor retains the manual entry in metadata but generates no LLM throughput or eval rows for it; ordinary entries in the same diff keep their normal selection. This field does not dispatch the workflow or establish benchmark success. Run the named workflow explicitly and inspect its artifacts. Replace the `XXX` PR-link placeholder when the PR exists.
+
 ## Stop conditions
 
 Stop before dispatching GPU work or claiming the configuration complete when any condition below holds. Obtain the missing fact or fix the source mismatch. Do not guess.

docs/configuration-procedures_zh.md

@@ -257,6 +257,20 @@ python -m pytest utils/matrix_logic/ -v
 6. 如果文件与 `main` 冲突，恢复当前 `main` 版本，只重新追加本分支条目。不要手动合并已经重排的历史。
 7. 请求 sweep 前解析文件，并确认生成的 changelog 选择包含预期 key。
 
+### 单独手动派发的实验工作流
+
+不属于 LLM master config 的实验基准可以记录变更，而不选择任何 LLM 任务：
+
+```yaml
+- config-keys: []
+  workflow-dispatch: h3-video.yml
+  description:
+    - "Add a manually dispatched H3 video benchmark"
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX
+```
+
+`workflow-dispatch` 必须是 `.github/workflows/` 下本地 `.yml` 或 `.yaml` 文件的名称，文件必须声明 `workflow_dispatch` 触发器。该模式要求 `config-keys` 为空，不允许启用 LLM eval、scenario 或 `append-only` 选项。普通条目仍须使用真实的 master-config key。changelog 处理器会将手动条目保留在元数据中，但不会为它生成 LLM 吞吐或 eval 任务；同一 diff 中的普通条目仍按原规则选择任务。该字段不会自动派发工作流，也不能证明基准运行成功。请显式运行所列工作流并检查产物。PR 创建后，将链接中的 `XXX` 占位符替换为实际编号。
+
 ## 停止条件
 
 出现以下任何条件时，在派发 GPU 工作或宣称配置完成前停止。取得缺失事实或修复来源不一致；不要猜测。