[swizzley]

What does this PR do?

Adds an opt-in on-device WebGPU playground at /webgpu. It turns a phone, tablet or laptop browser into an on-device inference target. The flag is off by default (ENABLE_WEBGPU_PLAYGROUND=True turns it on), the same pattern the Builder uses with ENABLE_BUILDER.

Open /webgpu on any device that can reach the server. The page:

1. Checks WebGPU and reads the adapter: vendor and architecture, shader-f16 support and buffer limits. It also flags software renderers (SwiftShader, lavapipe, WARP).
2. Measures the GPU with two short compute shaders:
   • memory bandwidth, from a vec4 copy over a 64 MB buffer;
   • fp32 throughput, from a chain of fused multiply-adds.
3. Estimates the decode speed of each model from the measured bandwidth. 4-bit weights are read once per token, so bandwidth sets the pace.
4. Downloads a small instruction-tuned LLM (SmolLM2 360M, Qwen2.5 0.5B or Llama 3.2 1B, prebuilt 4-bit WebLLM builds) and chats with it on the device's own GPU. Tokens stream, a Stop button cancels, and the page reports the decode speed WebLLM measured for each answer. The page picks the fp16 or fp32 build from shader-f16.

Prompts and answers stay on the device. The server hands out one static HTML page and adds no inference endpoints: both routes are include_in_schema=False, and the OpenAPI schema is unchanged. The page is also a quick way to see what the GPU in someone's pocket can do before you target it for edge deployment.

Files

File	Change
inference/core/interfaces/http/webgpu/playground.html	The page: self-contained and mobile-first, with no build step. The only third-party code is WebLLM, pinned to @mlc-ai/web-llm@0.2.78 (Apache-2.0) and imported from jsDelivr only when the visitor starts a model.
inference/core/interfaces/http/webgpu/routes.py	mount_webgpu_playground(app, prefix) registers GET /webgpu and GET /webgpu/.
inference/core/env.py	ENABLE_WEBGPU_PLAYGROUND, default False.
inference/core/interfaces/http/http_api.py	Mounts the page when the flag is set, next to the Builder.
tests/.../test_webgpu_playground.py	Absent when off; served at both paths; still served under Prometheus instrumentation; adds no API routes; loads only the pinned WebLLM build.
docs/webgpu_playground.md, mkdocs.yml	Enabling it, the HTTPS requirement, supported browsers, models, network access.

Browsers expose WebGPU only to secure contexts. localhost works as is. From a phone, serve the page over HTTPS or a tunnel. When the page is opened over plain HTTP, it says so and explains the fix.

Type of Change

• New feature (non-breaking change that adds functionality)

Testing

• pytest tests/inference/unit_tests/core/interfaces/http/test_webgpu_playground.py test_builder_availability.py: 9 passed.
• black, isort and flake8 at the versions in requirements.code_analysis.txt: clean on the changed Python files.
• End to end against the real server. I ran docker/config/cpu_http.py under uvicorn with ENABLE_WEBGPU_PLAYGROUND=True, then drove the page with headless Chromium at phone size (390 × 844, mobile, touch). WebGPU came from SwiftShader because the test machine has no GPU. Results:
  • /webgpu and /webgpu/ return 200, and /metrics counts both under their own handler names.
  • Neither path appears in /openapi.json.
  • Detect: 0.1 s. Measure: about 3 s.
  • SmolLM2 360M (fp32 build) loaded in 30 s.
  • The question "In one sentence, what is computer vision?" got: "Computer vision is the process of teaching computers to recognize and interpret visual information."
  • Measured speed: 0.3 tok/s, which is the CPU emulating the GPU. That run is why the page now detects software adapters and drops the bandwidth estimate for them.

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code where necessary, particularly in hard-to-understand areas
• My changes generate no new warnings or errors
• I have updated the documentation accordingly (if applicable)

Additional Context

A finding on main, outside this PR: with the dependency versions a fresh install resolves today (FastAPI 0.141.1, inside fastapi>=0.134,<0.142, and prometheus-fastapi-instrumentator 6.0.0), the Builder returns 500 at /build while metrics are on, which is the default (METRICS_ENABLED=True).

The cause: FastAPI 0.141 wraps a router added with include_router in an _IncludedRouter object that has no .path attribute. The instrumentator's _get_route_name reads route.path on every route while it names a request, so it raises:

AttributeError: '_IncludedRouter' object has no attribute 'path'

To reproduce: ENABLE_BUILDER=True, start cpu_http:app, then curl localhost:9001/build.

This PR keeps its own page clear of it: mount_webgpu_playground registers its routes with app.add_api_route, and a test covers the instrumented app. The Builder is left as it is, to keep this PR to one feature. Pinning fastapi<0.141 or upgrading the instrumentator would fix it there.

Packaging: the page ships the same way as the Builder's editor.html. The Docker images copy the source tree (COPY inference inference), so it needs no package_data change.

Network access: the page itself comes from the inference server. Model weights come from Hugging Face (mlc-ai/*), and WebLLM comes from jsDelivr. On an air-gapped network, the GPU check and measurement still work; loading a model needs those two hosts.

[dkosowski87]

In order to proceed with the review, pleas sign the CLA.