Skip to content

fix(ggml-cuda): require nonzero graph ID for property bypass - #676

Open
cheese-cakee wants to merge 2 commits into
Luce-Org:mainfrom
cheese-cakee:fix/ggml-graph-generation-uid-20260828
Open

fix(ggml-cuda): require nonzero graph ID for property bypass#676
cheese-cakee wants to merge 2 commits into
Luce-Org:mainfrom
cheese-cakee:fix/ggml-graph-generation-uid-20260828

Conversation

@cheese-cakee

@cheese-cakee cheese-cakee commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Prevent a direct GGML graph from replaying an older graph's CUDA/HIP executable when its metadata arena is reused and forced property-check bypass is requested.

Direct graphs have an unset generation ID (uid = 0). The old bypass condition accepted that unset ID as a match. Rebuilding a different operation at the same first-node address could therefore replay the old operation: the regression changes NEG to SQR, but the old libraries return -3 instead of 9 for input 3.

The bypass now requires a matching, nonzero generation ID. Zero-ID graphs retain ordinary property inspection, capture, and replay. This is a correctness fix, not a throughput claim.

Changes

  • Tighten the bypass predicate in ggml-cuda.cu and document the requirement in ggml-cuda.h.
  • Add a model-free direct-backend regression with persistent device buffers and a reused metadata arena.
  • Register the regression with CTest for CUDA/HIP graph-enabled builds, with a 60-second timeout and exit-77 handling for an unavailable GPU or disabled runtime graph support.
  • Check all 16 output elements on every submission and assert the existing graph telemetry. Eager-only execution cannot count as a pass.

Verification

Base: 5c775e31dc700ed2d5fd9dabbe1cb20bd083d949 (current main when prepared).

Runtime fix: standalone hardware validation

The pre-publication validation report records the following results for the same runtime fix shipped in this PR. Adding the registered regression test and CTest wiring did not change the production predicate or invalidate these earlier hardware results.

Hardware Before fix, forced bypass With fix Subsequent capture/replay
RTX 4050 Laptop, CUDA sm_89 Stale NEG: -3 instead of 9 Correct SQR: 9 Correct SQR with changed input: 400
Lucebox6 R9700, HIP gfx1201 Stale NEG: -3 instead of 9 Correct SQR: 9 Correct SQR with changed input: 400
Lucebox6 Strix Halo, HIP gfx1151 Stale NEG: -3 instead of 9 Correct SQR: 9 Correct SQR with changed input: 400

Normal property inspection also produced the correct result on all three targets. The HIP runs used ROCm 7.2.4 / HIP 7.2.53211. These are the recorded standalone-validation results, not a claim that the later CTest packaging was rerun on HIP.

Registered regression: additional CUDA validation

Fresh local validation: RTX 4050 Laptop (sm_89), WSL Ubuntu, CUDA 12.6, Release build.

Check Result
Registered focused CTest PASS, 1/1
Normal inspection and forced-bypass request PASS, 192 exact output-element comparisons across 12 submissions
Rebuild fallback, recapture, and replay with changed inputs PASS, checked through graph telemetry in both cases
Same new regression against preserved pre-fix libraries Normal-inspection control passes; forced bypass fails with -3 instead of 9, exit 134
git diff --check PASS

Reproduce with a graph-enabled server build:

cmake --build server/build --target test_direct_zero_uid_graph_aba -j4
ctest --test-dir server/build -R '^test_direct_zero_uid_graph_aba$' --output-on-failure --no-tests=error
# Select a different visible device when needed:
server/build/test_direct_zero_uid_graph_aba --device 1

For HIP, configure with -DDFLASH27B_GPU_BACKEND=hip -DGGML_HIP_GRAPHS=ON.

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

You’re at about 90% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread server/CMakeLists.txt
Comment thread server/test/test_direct_zero_uid_graph_aba.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant