From 2ec031e118a4525024736d8cc70a45da17489a5c Mon Sep 17 00:00:00 2001 From: Wenyao Gao Date: Tue, 8 Sep 2026 17:39:46 -0700 Subject: [PATCH 1/3] feat: add H3 video artifact viewer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 中文：新增 H3 视频产物查看器，直接读取 CI manifest、原始媒体和遥测；复用隐藏导航，区分执行、回归及测量窗口，并提供明确计费口径的收入与利润估算。 --- docs/h3-video.md | 156 +++ docs/index.md | 2 + packages/app/cypress/e2e/navigation.cy.ts | 29 + .../app/src/app/(dashboard)/video/page.tsx | 11 + .../app/src/app/zh/(dashboard)/video/page.tsx | 11 + packages/app/src/components/tab-nav.tsx | 1 + .../video-benchmark/VideoBenchmark.tsx | 924 ++++++++++++++++++ .../components/video-benchmark/bundle.test.ts | 170 ++++ .../src/components/video-benchmark/bundle.ts | 257 +++++ packages/app/src/lib/dashboard-routes.ts | 16 + packages/app/src/lib/tab-meta-zh.ts | 6 + packages/app/src/lib/tab-meta.ts | 4 + scripts/serve-h3-artifact.py | 42 + 13 files changed, 1629 insertions(+) create mode 100644 docs/h3-video.md create mode 100644 packages/app/src/app/(dashboard)/video/page.tsx create mode 100644 packages/app/src/app/zh/(dashboard)/video/page.tsx create mode 100644 packages/app/src/components/video-benchmark/VideoBenchmark.tsx create mode 100644 packages/app/src/components/video-benchmark/bundle.test.ts create mode 100644 packages/app/src/components/video-benchmark/bundle.ts create mode 100644 scripts/serve-h3-artifact.py diff --git a/docs/h3-video.md b/docs/h3-video.md new file mode 100644 index 000000000..6cb3d4a0a --- /dev/null +++ b/docs/h3-video.md @@ -0,0 +1,156 @@ +# H3 video artifact viewer + +`/video` and `/zh/video` consume the existing H3 CI artifact from +[InferenceX PR #2894](https://github.com/SemiAnalysisAI/InferenceX/pull/2894). +They do not launch inference or define a new backend result schema. + +## Review locally or in a PR preview + +Download the complete `h3-video--` GitHub Actions artifact, unzip +it, and choose **Open artifact folder**. Select the directory containing +`manifest.json` and `SHA256SUMS`. All files stay in browser memory. Nothing is +uploaded, persisted in localStorage, or bundled into the website. Reload or +Clear results releases the import. The 1 GiB bundle / 512 MiB file limits keep +this small smoke viewer bounded; larger runs need streaming storage. + +The reference download link is the real +[run 34291306687, artifact 10081961245](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/34291306687/artifacts/10081961245) +at backend commit `45c9d055dd17d2f023ce42ab7a6024fe2351b23a`. +The default reference link now points to the newer [eight-second clockwork-fox run 34293342829, artifact 10082823150](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/34293342829/artifacts/10082823150) at `65699f7c6f1c4d69a3226793251e6d2739ab60f4`; it uses the same contract. Both are identical-revision A/A runs, not optimized candidates. GitHub may +require sign-in and expires these artifacts after 14 days. Download before +expiry. No fixture is preloaded as a result. + +For a localhost URL that loads that downloaded bundle automatically: + +```sh +bun install --frozen-lockfile +bun run dev --hostname 127.0.0.1 --port 3000 +# In another terminal, use your extracted artifact's actual path: +python3 scripts/serve-h3-artifact.py /path/to/h3-video-34291306687-1 +``` + +Open `http://127.0.0.1:3000/video?manifest=http%3A%2F%2F127.0.0.1%3A8769%2Fmanifest.json`. +The helper binds loopback, serves only inventoried files under that directory, +and allows CORS only from the local app on port 3000. The H3 route does not need +a database. `E2E_FIXTURES=1` can supply unrelated dashboard routes for repository +tests; it never supplies video results. Other ports can use folder import. + +A PR preview opens the same viewer; collaborators use their own authorized +artifact download. **Sharing the page does not grant access to any media.** +An explicit HTTPS manifest-directory input is also supported for approved, +CORS-enabled public storage. No credentials are forwarded, redirects are rejected, +and URLs with embedded credentials/query tokens are rejected. Private or signed +single-object URLs should be downloaded and opened locally, not converted into a +public bucket. Deployed URLs never auto-fetch a manifest query parameter. + +## Existing feature gate and deployment boundary + +`use-feature-gate.ts` listens for **↑ ↑ ↓ ↓**, sets +`localStorage['inferencex-feature-gate'] = '1'`, and broadcasts +`inferencex:feature-gate:unlocked`. `tab-nav.tsx` derives hidden tabs from +`dashboard-routes.ts`. Video uses that registry with no global data providers, +English/Chinese routes, and `indexable: false`. Both routes send noindex metadata +and are excluded from the sitemap. Direct route access works while navigation is +locked, exactly because the feature gate is visibility, **not authorization**. + +This change prepares the integration and PR preview. It does not authorize a +production merge/deployment, change deployment protections, or publish private +media. The import surface masks PostHog text and excludes autocapture; its only +explicit analytics event carries no bundle values. + +## Backend contract and interpretation + +| Source | UI use | +| ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ | +| `manifest.json` (`schema_version: 1`) | Run/attempt, backend commit, frozen workload, model revision, declared resources and evidence hashes | +| `SHA256SUMS` | Verify every listed file before displaying any results; missing, duplicate, unsafe or mismatched entries fail the import | +| `ci.json` | Final execution/regression/qualification status; startup Slurm snapshots do not override final execution state | +| `report/evidence.json` (`controlled_gpu_report`) | Role summaries, original media identities, observations/warmups and comparison checks | +| `gpu//run.json` | Full completion accounting and recorded video/audio integrity checks | +| `gpu/gpu-job.json` | Observed GPU identities, launch flags, telemetry qualification and measured-window memory samples | +| `gpu/supervisor//telemetry.jsonl` | Recorded board power, monotonic times, phase and owned GPU observations | +| `gpu/comparison.json` | Normalized video MAE, audio waveform MAE and paired coverage | +| `report/index.html` and content-addressed assets | Original report in a sandbox; scripts, external resources and forms disabled; original files downloadable | + +Checksums verify internal consistency, not source authenticity or independent +hardware attestation. Confirm the exact CI run and manifest digest. All displays +are recorded results or explicitly labeled arithmetic, not a fresh decode or +re-evaluation. Missing fields remain unavailable; a failed attempt without a +report still shows its final status and downloadable evidence. + +- Latency is client submit → downloaded, validated media. It includes polling, + transfer and local analysis; it is not GPU kernel latency. +- Clips/sec is valid clips / serial measured-block wall seconds, with warmup + excluded. Counts retain failed/invalid and scheduled requests. It is not + demonstrated saturated capacity, and a one-clip median is one observation. +- Power is the sum of observed participating-board watts, not TDP. The viewer + integrates adjacent complete, owned `measurement` samples by trapezoids and + divides by covered time for mean watts. It does not extrapolate endpoints or + bridge missing/duplicate/out-of-order samples or gaps above 3× requested cadence. + Backend telemetry qualification must be true. **This window includes warmup** + and is shown with UTC endpoints and sample count. Energy is a sampled integral + estimate (kJ), not facility energy or measured-only J/clip. +- Memory uses the backend's measurement-window per-GPU maximum in MiB, including + warmup. Sampling can miss peaks; it is not framework allocator high-water usage. +- Video MAE is normalized RGB difference (0–1). Identical decoded frames have + `video_identical: true` and null finite PSNR; null is not a failed comparison. + Audio RMS ratio is candidate/baseline. Similarity and integrity are not + perceptual quality, prompt adherence, lip sync, or release qualification. +- The top-level regression remains inconclusive when uncalibrated, even if + individual fidelity checks pass and execution completes. + +## Optional economics + +The fields start without an assumed market price or cost. Enter USD/clip, a source +or explanation, and an as-of date. Billed GPU count defaults from Slurm `AllocTRES` +and is editable when a billing agreement differs; participating GPUs come from +recorded telemetry. With measured rate `q`, price `p`, billed count `B`, +participating count `P`, and all-in billed GPU-hour cost `c`: + +- Total revenue/hour = `q × 3600 × p`. +- Revenue per participating/billed GPU-hour = revenue divided by `P` / `B`. +- Total profit/hour = revenue − `B × c`; divide by `P` / `B` for the two views. + +Blank cost means unavailable profit, not zero cost. The cost must include rental +or amortization, electricity, cooling, host/network, storage, labor, licensing and +other overhead without double-counting included rental costs. Negative profit is +preserved. This is a full-demand extrapolation of serial throughput, excludes +startup/warmup, and does not claim realized revenue or sustainable serving capacity. +The four-second reference uses four participating GPUs and eight allocated GPUs. + +## Persistent storage: remaining configuration + +The existing app has public JSON Vercel Blob caching (`blob-cache.ts`, +`BLOB_READ_WRITE_TOKEN` plus `BLOB_CACHE_PREFIX`) and public GCS artifact-backup +readers. Those patterns do not establish approved private H3 media access. +No R2 client/configuration or R2 credential names were found in the app source, +example environment, current worktree environment, or readable GitHub secret-name +inventory on 2026-09-09 UTC. This does not inspect or assert the contents of Vercel's +remote environment. + +A private persistent upload is blocked on: the approved bucket/account and S3 +endpoint, an object-scoped write credential (R2 access-key ID and secret or an +approved worker binding), collaborator access policy/identity provider or signed +read service, and CORS origins. No bucket policy or credential was changed. +Reuse `report/assets/.mp4` identities. A future approved publisher can +namespace the immutable full bundle by repository/run/attempt/manifest SHA256, +verify existing bytes before reusing objects, and preserve `SHA256SUMS` plus the +CI artifact reference. Do not overwrite an object with different bytes or treat +an unlisted public URL as private storage. + +## Verification + +`bundle.test.ts` covers checksum/path failures, partial attempts, missing data, +sampled power integration, invalid windows and both economics denominators. +To additionally read actual downloaded backend bytes: + +```sh +H3_ARTIFACT_DIR=/path/to/extracted-artifact bun --cwd packages/app vitest run \ + src/components/video-benchmark/bundle.test.ts +``` + +`navigation.cy.ts` covers the real shared keyboard gate, English/Chinese noindex +routes, empty state and recoverable import errors. Verify actual media in a browser: +play each side with sound, switch to warmup and back, check original report playback, +download manifest/media and compare hashes, clear/reload, and retry after a bad URL. +Tests' synthetic contract data is test-only and is never a default demo result. diff --git a/docs/index.md b/docs/index.md index 96ac83a45..b76232f09 100644 --- a/docs/index.md +++ b/docs/index.md @@ -4,6 +4,8 @@ Design rationale and non-obvious conventions. See [CLAUDE.md](../CLAUDE.md) for ## Docs +- [H3 Video Viewer](./h3-video.md) — CI artifact contract, local/preview access, measurements and storage boundaries + - [API Skill Examples](./inferencex-api-examples.md) — Install the public skill, query benchmarks, export measured PowerX data, and explain empty results - [API Skill Releases](./inferencex-skills-release.md) — Prepare an immutable package, verify clean installations and agent exports, and publish through the package-specific workflow - [API Skill Discovery](./inferencex-skills-discovery.md) — Accept or reject implicit skill discovery in fresh Codex and Claude Code projects diff --git a/packages/app/cypress/e2e/navigation.cy.ts b/packages/app/cypress/e2e/navigation.cy.ts index 3e5ee3e0c..b46326952 100644 --- a/packages/app/cypress/e2e/navigation.cy.ts +++ b/packages/app/cypress/e2e/navigation.cy.ts @@ -321,3 +321,32 @@ describe('TPUv7 launch banner', { testIsolation: true }, () => { cy.get('[data-testid="tco-basis-toggle"]').should('not.exist'); }); }); + +describe('H3 video artifact viewer', () => { + it('uses the shared unlock for navigation and keeps an empty viewer free of sample results', () => { + cy.viewport(1440, 1000); + cy.visit('/video', { + onBeforeLoad(win) { + win.localStorage.removeItem('inferencex-feature-gate'); + }, + }); + cy.get('[data-testid="video-benchmark"]').should('contain', 'Open a real CI result to begin'); + cy.get('video[data-role]').should('not.exist'); + cy.get('[data-testid="tab-trigger-hidden"]').should('not.exist'); + cy.get('body').type('{upArrow}{upArrow}{downArrow}{downArrow}'); + cy.get('[data-testid="tab-trigger-hidden"]').click(); + cy.contains('a', 'Video').should('have.attr', 'href', '/video'); + cy.get('head meta[name="robots"]').should('have.attr', 'content', 'noindex, nofollow'); + }); + it('shows a recoverable load error and the Chinese empty state', () => { + cy.visit('/video'); + cy.get('[data-testid="video-benchmark"]').contains('summary', 'Manifest URL').click(); + cy.get('input[aria-label="Manifest URL"]').type('https://example.com/wrong.json'); + cy.contains('button', 'Load manifest').click(); + cy.get('[role="alert"]').should('contain', 'Could not load this bundle'); + cy.get('video[data-role]').should('not.exist'); + cy.visit('/zh/video'); + cy.get('[data-testid="video-benchmark"]').should('contain', '打开真实 CI 结果开始查看'); + cy.get('head meta[name="robots"]').should('have.attr', 'content', 'noindex, nofollow'); + }); +}); diff --git a/packages/app/src/app/(dashboard)/video/page.tsx b/packages/app/src/app/(dashboard)/video/page.tsx new file mode 100644 index 000000000..0624873c2 --- /dev/null +++ b/packages/app/src/app/(dashboard)/video/page.tsx @@ -0,0 +1,11 @@ +import type { Metadata } from 'next'; +import VideoBenchmark from '@/components/video-benchmark/VideoBenchmark'; +import { tabMetadata } from '@/lib/tab-meta'; + +export const metadata: Metadata = { + ...tabMetadata('video'), + robots: { index: false, follow: false }, +}; +export default function VideoPage() { + return ; +} diff --git a/packages/app/src/app/zh/(dashboard)/video/page.tsx b/packages/app/src/app/zh/(dashboard)/video/page.tsx new file mode 100644 index 000000000..3550a1019 --- /dev/null +++ b/packages/app/src/app/zh/(dashboard)/video/page.tsx @@ -0,0 +1,11 @@ +import type { Metadata } from 'next'; +import VideoBenchmark from '@/components/video-benchmark/VideoBenchmark'; +import { tabMetadataZh } from '@/lib/tab-meta-zh'; + +export const metadata: Metadata = { + ...tabMetadataZh('video'), + robots: { index: false, follow: false }, +}; +export default function VideoPage() { + return ; +} diff --git a/packages/app/src/components/tab-nav.tsx b/packages/app/src/components/tab-nav.tsx index 4732315eb..d93012ea0 100644 --- a/packages/app/src/components/tab-nav.tsx +++ b/packages/app/src/components/tab-nav.tsx @@ -56,6 +56,7 @@ const TAB_LABELS_EN: Record = { 'ai-chart': 'AI Chart', 'gpu-metrics': 'PowerX', 'current-inferencex-image': 'Images', + video: 'Video', feedback: 'Feedback', }; diff --git a/packages/app/src/components/video-benchmark/VideoBenchmark.tsx b/packages/app/src/components/video-benchmark/VideoBenchmark.tsx new file mode 100644 index 000000000..dc3ae90be --- /dev/null +++ b/packages/app/src/components/video-benchmark/VideoBenchmark.tsx @@ -0,0 +1,924 @@ +'use client'; + +import { useEffect, useRef, useState } from 'react'; +import { Card } from '@/components/ui/card'; +import { Button } from '@/components/ui/button'; +import { Input } from '@/components/ui/input'; +import { Heading } from '@/components/ui/heading'; +import { useLocale } from '@/lib/use-locale'; +import { track } from '@/lib/analytics'; +import { + at, + entries, + estimateEconomics, + folderReader, + httpReader, + loadBundle, + number, + ROLES, + rows, + sampledPower, + text, + type Bundle, + type Json, +} from './bundle'; + +const STRINGS = { + en: { + title: 'H3 Video Benchmark', + subtitle: 'Original media. Measured execution. Traceable comparisons.', + aa: 'Same-build A/A', + sameWorkload: 'Same workload', + statsNote: + 'The metrics below summarize the measured block. Selected warmup clips are excluded.', + open: 'Open artifact folder', + load: 'Load manifest', + source: 'Manifest URL', + clear: 'Clear results', + loading: 'Loading and verifying checksums…', + empty: 'Open a real CI result to begin', + instructions: + 'Download the H3 artifact from GitHub Actions, extract it, then choose the folder containing manifest.json. Files stay in this browser; nothing is uploaded. Reloading clears imported files.', + sample: 'Download reference CI artifact', + remote: + 'Alternatively, load an HTTPS artifact directory with CORS enabled. Private bundles should use the local folder option; hidden navigation is not access control.', + error: 'Could not load this bundle', + retry: + 'Choose a complete artifact folder or correct the manifest URL and retry. Existing results have been cleared.', + verified: 'File checksums verified', + trust: + 'Checksums establish bundle consistency, not independent source or hardware attestation. Confirm the CI run and manifest hash before sharing conclusions.', + execution: 'Execution', + comparison: 'Regression', + calibration: 'Calibration', + qualification: 'Release qualified', + unavailable: 'Unavailable', + yes: 'Yes', + no: 'No', + baseline: 'Baseline', + candidate: 'Candidate', + slot: 'Clip / request', + prompt: 'Prompt', + settings: 'Generation settings', + seed: 'Seed', + warmup: 'Warmup', + measured: 'Measured', + noMedia: 'No generated media for this request', + mediaError: + 'This browser could not decode the video/audio. Download the original media to inspect it.', + download: 'Download original MP4', + audio: 'Audio is enabled. Play one side at a time to compare the sound.', + latency: 'Median end-to-end latency', + throughput: 'Valid clips / second', + counts: 'Completion accounting', + memory: 'Sampled GPU memory', + timing: + 'Latency: submission through downloaded and validated media, including polling, transfer and analysis. Throughput: valid measured clips ÷ serial measured-block seconds; warmup excluded. These are point estimates, not saturated serving capacity.', + window: 'Measured block', + completed: 'Completed', + valid: 'Valid', + scheduled: 'Scheduled', + failed: 'Failed / invalid', + power: 'Measured power & sampled energy', + powerNote: + 'Board power summed across participating GPUs. Time-weighted power and trapezoidal energy cover only the first-to-last complete telemetry samples tagged measurement, including warmup. No endpoint extrapolation; gaps over 3× the requested interval invalidate the estimate. This is not facility energy or energy per measured clip.', + meanPower: 'Time-weighted mean', + energy: 'Integrated sampled energy (estimate)', + coverage: 'Covered telemetry window', + samples: 'Samples', + memoryNote: + 'Maximum observed device-used VRAM per GPU during the client workload, including warmup. MiB = 2²⁰ bytes. Sampling can miss peaks; these are not allocator peaks.', + integrity: 'Video & audio integrity', + fidelity: 'Paired fidelity', + fidelityNote: + 'Pixel and waveform similarity detect implementation drift. They do not establish prompt adherence, perceptual quality, lip sync or human preference. Thresholds remain uncalibrated unless the backend says otherwise.', + identical: 'Identical frames; finite PSNR undefined', + checks: 'Recorded checks', + hardware: 'Hardware & revisions', + participating: 'Participating GPUs', + allocated: 'Allocated GPUs', + provenance: 'Provenance & downloads', + ciRun: 'Exact CI run', + commit: 'Backend commit', + manifest: 'Manifest SHA256', + report: 'Open original report', + reportNote: + 'Original report rendered in a sandbox, with verified local media. Scripts and external requests are disabled.', + raw: 'Raw artifacts', + economics: 'Revenue & profit scenario', + economicsNote: + 'User-entered USD assumptions. Revenue extrapolates measured serial throughput to one hour at 100% demand; it excludes startup/warmup and is not demonstrated capacity. Profit requires all-in cost per billed GPU-hour, including rental or amortization, electricity, cooling, host/network, storage, labor, licensing and other overhead. Allocation count defaults from Slurm; change billed count only if your contract differs.', + price: 'Selling price (USD / generated clip)', + cost: 'All-in cost (USD / billed GPU-hour)', + billed: 'Billed GPUs', + assumption: 'Price and cost source / assumptions', + date: 'Assumptions as of', + revenueParticipant: 'Revenue / participating GPU-hour', + revenueBilled: 'Revenue / billed GPU-hour', + profitParticipant: 'Profit / participating GPU-hour', + profitBilled: 'Profit / billed GPU-hour', + currency: 'USD', + estimate: 'Estimate', + needAssumptions: + 'Enter a price, source, date and billed GPU count. Leave cost blank to keep profit unavailable.', + missing: 'Backend fields & limits', + missingNote: + 'Missing values stay unavailable. This contract has no measured-only per-request power boundaries, facility energy, pricing or full cost model. Sensor calibration and independent hardware attestation are not supplied. No private object-storage access is configured by this viewer.', + }, + zh: { + title: 'H3 视频基准测试', + subtitle: '查看原始媒体、实测执行数据与可追溯的比较结果。', + aa: '同构建 A/A', + sameWorkload: '相同工作负载', + statsNote: '以下指标汇总正式测量时段；所选 warmup 视频不计入其中。', + open: '打开产物文件夹', + load: '加载 manifest', + source: 'Manifest URL', + clear: '清除结果', + loading: '正在加载并校验文件…', + empty: '打开真实 CI 结果开始查看', + instructions: + '从 GitHub Actions 下载 H3 产物并解压，选择包含 manifest.json 的文件夹。文件仅在当前浏览器中读取，不会上传；刷新页面后需要重新导入。', + sample: '下载参考 CI 产物', + remote: + '也可加载启用 CORS 的 HTTPS 产物目录。私有文件请通过本地文件夹打开；隐藏导航不提供访问控制。', + error: '无法加载此产物包', + retry: '请选择完整的产物文件夹，或修正 manifest URL 后重试。此前的结果已清除。', + verified: '文件校验和通过', + trust: + '校验和仅确认产物包内部一致，不构成独立的来源或硬件认证。分享结论前请核对 CI run 与 manifest 哈希。', + execution: '执行状态', + comparison: '回归结论', + calibration: '校准状态', + qualification: '发布验收通过', + unavailable: '暂无数据', + yes: '是', + no: '否', + baseline: '基线', + candidate: '候选', + slot: '视频 / 请求', + prompt: 'Prompt', + settings: '生成配置', + seed: 'Seed', + warmup: 'Warmup', + measured: '正式测量', + noMedia: '此请求没有生成媒体', + mediaError: '浏览器无法解码此视频或音频，请下载原始媒体检查。', + download: '下载原始 MP4', + audio: '音频已启用。建议逐个播放，对比两侧声音。', + latency: '端到端延迟中位数', + throughput: '每秒有效视频数', + counts: '完成情况', + memory: 'GPU 显存采样', + timing: + '延迟从提交请求计时，直到媒体下载并验证完成，包含轮询、传输与分析。吞吐量 = 有效测量视频数 ÷ 串行测量时段秒数，不含 warmup。这些是点估计，不代表饱和服务容量。', + window: '正式测量时段', + completed: '已完成', + valid: '有效', + scheduled: '计划请求', + failed: '失败 / 无效', + power: '实测功耗与采样能耗', + powerNote: + '功率为参与计算的 GPU 板卡功率之和。时间加权平均功率和梯形积分能耗仅覆盖 measurement 阶段首末完整采样之间的时段，包含 warmup。端点不外推；间隔超过请求采样周期的 3 倍时不提供估算。这不是设施能耗，也不是每个正式测量视频的能耗。', + meanPower: '时间加权平均功率', + energy: '采样积分能耗（估算）', + coverage: '遥测覆盖时段', + samples: '采样数', + memoryNote: + '各 GPU 在客户端工作负载期间采到的显存占用最大值，包含 warmup。MiB = 2²⁰ 字节。采样可能漏掉峰值，不等同于分配器峰值。', + integrity: '视频与音频完整性', + fidelity: '配对保真度', + fidelityNote: + '像素与波形相似度用于发现实现变化造成的输出漂移，不能证明 prompt 遵循度、感知质量、口型同步或人类偏好。除非后端明确标注，否则阈值仍未经校准。', + identical: '帧完全一致，有限 PSNR 无定义', + checks: '已记录的检查', + hardware: '硬件与版本', + participating: '参与计算的 GPU', + allocated: '分配的 GPU', + provenance: '来源记录与下载', + ciRun: '对应 CI run', + commit: '后端 commit', + manifest: 'Manifest SHA256', + report: '打开原始报告', + reportNote: '报告在沙盒中显示并引用已校验的本地媒体，脚本与外部请求均禁用。', + raw: '原始产物', + economics: '收入与利润情景估算', + economicsNote: + '金额采用用户输入的美元假设。收入按实测串行吞吐量外推至满需求的一小时，不含启动与 warmup，不代表已验证的服务容量。利润需要每个计费 GPU 小时的完整成本，涵盖租赁或折旧、电力、冷却、主机与网络、存储、人力、许可费及其他开销。分配数量默认来自 Slurm；仅在合同计费方式不同时修改计费数量。', + price: '售价（USD / 生成视频）', + cost: '完整成本（USD / 计费 GPU 小时）', + billed: '计费 GPU 数', + assumption: '价格与成本来源 / 假设', + date: '假设日期', + revenueParticipant: '每个参与 GPU 小时的收入', + revenueBilled: '每个计费 GPU 小时的收入', + profitParticipant: '每个参与 GPU 小时的利润', + profitBilled: '每个计费 GPU 小时的利润', + currency: 'USD', + estimate: '估算', + needAssumptions: '请填写售价、来源、日期与计费 GPU 数。成本留空时，利润显示为暂无数据。', + missing: '后端字段与局限', + missingNote: + '缺失值显示为暂无数据。当前协议不提供逐请求且排除 warmup 的功耗边界、设施能耗、售价或完整成本模型，也未提供传感器校准与独立硬件认证。此查看器未配置私有对象存储访问。', + }, +}; + +interface Loaded { + bundle: Bundle; + urls: Map; + power: Record>; + html: string; +} +const REF_RUN = 'https://github.com/SemiAnalysisAI/InferenceX/actions/runs/34293342829'; +const REF_ARTIFACT = `${REF_RUN}/artifacts/10082823150`; +const parseInput = (value: string) => (value.trim() === '' ? null : Number(value)); + +const field = (label: string, value: string, change: (value: string) => void, type = 'number') => ( + +); + +export default function VideoBenchmark() { + const s = STRINGS[useLocale()]; + const [loaded, setLoaded] = useState(null); + const [loading, setLoading] = useState(false); + const [loadError, setError] = useState(''); + const [source, setSource] = useState(''); + const [slot, setSlot] = useState(''); + const [price, setPrice] = useState(''); + const [cost, setCost] = useState(''); + const [billed, setBilled] = useState(''); + const [assumption, setAssumption] = useState(''); + const [date, setDate] = useState(''); + const generation = useRef(0); + const input = useRef(null); + const activeUrls = useRef([]); + const fmt = (value: Json | undefined, digits = 2): string => { + if (typeof value === 'number') + return Number.isFinite(value) + ? value.toLocaleString('en-US', { maximumFractionDigits: digits }) + : s.unavailable; + if (value === null || value === undefined || value === '') return s.unavailable; + if (typeof value === 'boolean') return value ? s.yes : s.no; + return typeof value === 'string' ? value : JSON.stringify(value); + }; + function clear() { + generation.current++; + activeUrls.current.forEach(URL.revokeObjectURL); + activeUrls.current = []; + setLoaded(null); + setError(''); + setLoading(false); + setSlot(''); + } + async function open(read: () => (path: string) => Promise) { + clear(); + const current = generation.current; + setLoading(true); + const urls = new Map(); + try { + const bundle = await loadBundle(read()); + const power: Loaded['power'] = {}; + for (const role of ROLES) { + const file = bundle.files.get(`gpu/supervisor/${role}/telemetry.jsonl`); + const telemetry = at(bundle.job, 'roles', role, 'telemetry_summary'); + const uuids = rows(at(telemetry, 'gpu_identity')).map((g) => text(at(g, 'uuid'))); + const interval = number(at(telemetry, 'requested_interval_seconds')); + const telemetryText = file ? await file.text() : ''; + power[role] = + file && interval !== null && at(telemetry, 'qualified') === true + ? sampledPower( + telemetryText + .trim() + .split('\n') + .filter(Boolean) + .map((line) => JSON.parse(line)), + uuids, + interval * 3, + ) + : null; + } + if (current !== generation.current) return; + for (const [path, file] of bundle.files) { + urls.set( + path, + URL.createObjectURL( + new Blob([file], { + type: path.endsWith('.mp4') ? 'video/mp4' : 'application/octet-stream', + }), + ), + ); + } + const rawHtml = await bundle.files.get('report/index.html')?.text(); + let html = ''; + if (rawHtml) { + const doc = new DOMParser().parseFromString(rawHtml, 'text/html'); + doc + .querySelectorAll('script,base,iframe,object,embed,link,meta[http-equiv],form') + .forEach((n) => n.remove()); + doc.querySelectorAll('[src], [href]').forEach((n) => { + for (const attr of ['src', 'href']) { + const value = n.getAttribute(attr); + if (value) { + const url = urls.get(`report/${value}`); + if (url) n.setAttribute(attr, url); + else n.removeAttribute(attr); + } + } + }); + const csp = doc.createElement('meta'); + csp.httpEquiv = 'Content-Security-Policy'; + csp.content = + "default-src 'none'; media-src blob:; img-src blob: data:; style-src 'unsafe-inline'; form-action 'none'; base-uri 'none'"; + doc.head.prepend(csp); + html = doc.documentElement.outerHTML; + } + if (current !== generation.current) { + urls.forEach(URL.revokeObjectURL); + return; + } + activeUrls.current = [...urls.values()]; + setLoaded({ bundle, urls, power, html }); + const first = + rows(at(bundle.report, 'roles', 'baseline', 'observations'))[0] ?? + rows(at(bundle.report, 'roles', 'candidate', 'observations'))[0]; + setSlot(text(at(first, 'slot_id'))); + setBilled( + /(?:^|,)gres\/gpu=(?\d+)(?:,|$)/u.exec(text(at(bundle.ci, 'slurm_job', 'AllocTRES'))) + ?.groups?.count ?? '', + ); + track('video_bundle_loaded'); + } catch (error) { + urls.forEach(URL.revokeObjectURL); + if (current === generation.current) + setError(error instanceof Error ? error.message : String(error)); + } finally { + if (current === generation.current) setLoading(false); + } + } + useEffect(() => { + // A shared preview link must not trigger requests to a URL supplied by its sender. + if (['127.0.0.1', 'localhost'].includes(location.hostname)) { + const initial = new URLSearchParams(location.search).get('manifest'); + if (initial) { + setSource(initial); + void open(() => httpReader(initial)); + } + } + return () => { + generation.current++; + activeUrls.current.forEach(URL.revokeObjectURL); + }; + // eslint-disable-next-line react-hooks/exhaustive-deps + }, []); + + const b = loaded?.bundle; + const participating = rows( + at(b?.job, 'roles', 'baseline', 'telemetry_summary', 'gpu_identity'), + ).length; + const observations = (role: string) => [ + ...rows(at(b?.report, 'roles', role, 'observations')), + ...rows(at(b?.report, 'roles', role, 'warmups')), + ]; + const selected = (role: string) => observations(role).find((o) => at(o, 'slot_id') === slot); + const clip = selected('baseline') ?? selected('candidate'); + const slots = [ + ...new Map( + ROLES.flatMap((role) => observations(role)).map((o) => [text(at(o, 'slot_id')), o]), + ).values(), + ]; + const ciUrl = /^\d+$/u.test(text(at(b?.manifest, 'run_id'))) + ? `https://github.com/SemiAnalysisAI/InferenceX/actions/runs/${text(at(b?.manifest, 'run_id'))}` + : null; + const pair = at(b?.report, 'slot_comparisons', slot); + const rawPair = rows(at(b?.comparison, 'slots')).find((o) => at(o, 'slot_id') === slot); + const table = (data: [string, Json | undefined][]) => ( +
+ {data.map(([label, value]) => ( +
+
{label}
+
{fmt(value, 4)}
+
+ ))} +
+ ); + + return ( +
+
+
+ + {s.title} + +

{s.subtitle}

+
+
+ + {b && ( + + )} +
+
+ { + const files = [...(e.target.files ?? [])]; + e.target.value = ''; + if (files.length > 0) void open(() => folderReader(files)); + }} + /> +
+ {s.source} +
{ + e.preventDefault(); + void open(() => httpReader(source)); + }} + > + setSource(e.target.value)} + placeholder="https://…/manifest.json" + required + /> + +
+

{s.remote}

+
+ {loading && {s.loading}} + {loadError && ( + + {s.error} +

{loadError}

+

{s.retry}

+ + )} + {!b && !loading && !loadError && ( + + {s.empty} +

{s.instructions}

+ + {s.sample} · #34293342829 + + + )} + {b && loaded && ( + <> + +
+ + GitHub #{text(at(b.manifest, 'run_id'))} · {fmt(at(b.manifest, 'run_attempt'))} ·{' '} + {fmt(at(b.ci, 'finished_at'))} + + + {s.verified} · {b.checksums.size} + +
+ {at(b.report, 'same_workload') === true && + at(b.report, 'same_gpu_uuid_set') === true && + at( + b.documents.get('gpu/baseline/run.json'), + 'configuration', + 'configuration_sha256', + ) !== null && + at( + b.documents.get('gpu/baseline/run.json'), + 'configuration', + 'configuration_sha256', + ) === + at( + b.documents.get('gpu/candidate/run.json'), + 'configuration', + 'configuration_sha256', + ) && + at(b.report, 'roles', 'baseline', 'source_identity', 'source_sha256') !== null && + at(b.report, 'roles', 'baseline', 'source_identity', 'source_sha256') === + at(b.report, 'roles', 'candidate', 'source_identity', 'source_sha256') && ( +

{s.aa}

+ )} +
+ {[ + [s.execution, at(b.ci, 'phase')], + [s.comparison, at(b.ci, 'regression_status')], + [s.calibration, at(b.report, 'policy', 'calibration_status')], + [s.qualification, at(b.ci, 'release_qualified')], + ].map(([label, value]) => ( +
+

{fmt(label)}

+

{fmt(value)}

+
+ ))} +
+

+ {s.sameWorkload}: {fmt(at(b.report, 'same_workload'))}. {s.trust} +

+ + {slots.length > 0 && ( + + +
+ + {s.prompt} · {s.seed} {fmt(at(clip, 'seed'))} + +

{fmt(at(clip, 'prompt'))}

+
+
+ {s.settings} +
+ {table(entries(at(b.manifest, 'workload_plan', 'generation')))} +
+
+ + )} +
+ {ROLES.map((role) => { + const o = selected(role); + const r = at(b.report, 'roles', role); + const summary = at(r, 'summary'); + const mediaUrl = loaded.urls.get(`report/${text(at(o, 'artifact_path'))}`); + const run = b.documents.get(`gpu/${role}/run.json`); + const rawSummary = at(run, 'summary'); + return ( + +
+ {s[role]} + {fmt(at(o, 'status'))} +
+ {mediaUrl ? ( + <> + + ); + })} +
+

{s.timing}

+ + {s.power} +

{s.powerNote}

+
+ {ROLES.map((role) => { + const power = loaded.power[role]; + return ( +
+ {s[role]} + {table([ + [`${s.meanPower} (W)`, power?.watts], + [`${s.energy} (kJ)`, power ? power.joules / 1000 : null], + [`${s.coverage} (s)`, power?.seconds], + [s.samples, power?.sampleCount], + ])} + {power && ( +

+ {power.start} → {power.end} +

+ )} +
+ ); + })} +
+ {s.memory} +

{s.memoryNote}

+
+ {ROLES.map((role) => ( +
+ {s[role]} + {entries( + at( + b.job, + 'roles', + role, + 'telemetry_summary', + 'measurement_observed_memory_peak_mib_by_gpu', + ), + ).length > 0 + ? table( + entries( + at( + b.job, + 'roles', + role, + 'telemetry_summary', + 'measurement_observed_memory_peak_mib_by_gpu', + ), + ).map(([gpu, value]) => [gpu, `${fmt(value)} MiB`]), + ) + : s.unavailable} +
+ ))} +
+ + + {s.fidelity} +

{s.fidelityNote}

+ {table([ + [ + 'Video PSNR (dB)', + at(pair, 'metrics', 'video_identical') === true + ? s.identical + : at(pair, 'metrics', 'video_psnr_db'), + ], + ['Video MAE (normalized 0–1)', at(pair, 'metrics', 'video_mae')], + ['Audio spectral cosine', at(pair, 'metrics', 'audio_spectral_cosine')], + ['Audio RMS ratio (candidate / baseline)', at(pair, 'metrics', 'audio_rms_ratio')], + ['Audio waveform MAE (PCM amplitude)', at(rawPair, 'metrics', 'audio_waveform_mae')], + [ + 'Video coverage (fraction)', + at(rawPair, 'metrics', 'video_sample_coverage_fraction'), + ], + [ + 'Audio coverage (fraction)', + at(rawPair, 'metrics', 'audio_sample_coverage_fraction'), + ], + ])} +
+ {s.checks} + {rows(at(pair, 'checks')).map((c, i) => ( +

+ {text(at(c, 'name'))} · {fmt(at(c, 'status'))} · {text(at(c, 'reason'))} +

+ ))} +
+ + + {s.hardware} + {table([ + [s.participating, participating > 0 ? participating : null], + [ + s.allocated, + /(?:^|,)gres\/gpu=(?\d+)(?:,|$)/u.exec( + text(at(b.ci, 'slurm_job', 'AllocTRES')), + )?.groups?.count, + ], + ['Slurm', at(b.manifest, 'slurm_allocation', 'identity', 'JobId')], + ['Node', at(b.ci, 'slurm_job', 'NodeList')], + ['Model', at(b.manifest, 'workload_plan', 'model_id')], + ['Model revision', at(b.manifest, 'workload_plan', 'model_revision')], + ])} +
+ {ROLES.map((role) => ( +
+ {s[role]} + {table([ + [ + 'Runtime revision', + at(b.report, 'roles', role, 'configuration', 'runtime_revision'), + ], + [ + 'Source SHA256', + at(b.report, 'roles', role, 'source_identity', 'source_sha256'), + ], + [ + 'GPU', + rows(at(b.job, 'roles', role, 'telemetry_summary', 'gpu_identity')) + .map((g) => text(at(g, 'name'))) + .join(', '), + ], + ])} +
+ {s.settings} + 
+                      {JSON.stringify(at(b.job, 'roles', role, 'launch_argv'), null, 2)}
+
+
+
+ ))} +
+ + + {s.economics} +

{s.economicsNote}

+
+ {field(s.price, price, setPrice)} + {field(s.cost, cost, setCost)} + {field(s.billed, billed, setBilled)} + {field(s.assumption, assumption, setAssumption, 'text')} + {field(s.date, date, setDate, 'date')} +
+

{s.needAssumptions}

+
+ {ROLES.map((role) => { + const summary = at(b.report, 'roles', role, 'summary'); + const count = rows( + at(b.job, 'roles', role, 'telemetry_summary', 'gpu_identity'), + ).length; + const e = + assumption.trim() && date + ? estimateEconomics( + number(at(summary, 'valid_clips_per_second')), + count || null, + parseInput(billed), + parseInput(price), + parseInput(cost), + ) + : null; + return ( +
+ + {s[role]} · {s.estimate} (USD) + + {table([ + [s.revenueParticipant, e?.revenuePerParticipating], + [s.revenueBilled, e?.revenuePerBilled], + [s.profitParticipant, e?.profitPerParticipating], + [s.profitBilled, e?.profitPerBilled], + ])} +
+ ); + })} +
+ + + {s.provenance} + {ciUrl && ( + + {s.ciRun} · #{text(at(b.manifest, 'run_id'))} + + )} + + {s.commit} · {text(at(b.manifest, 'git_commit'))} + + {table([[s.manifest, b.manifestSha256]])} + {loaded.html && ( +
+ {s.report} +

{s.reportNote}

+