perf(ds4): accelerate long context and add PFlash

[davide221]

Summary

• use sparse fused verifier attention with direct F16 KV at long context
• add Vulkan-inspired packed small-CM rocWMMA indexer dispatch for verifier widths 2 through 5 on RDNA3.5 and RDNA4
• enable DS4 PFlash with Qwen3-0.6B, cross-tokenizer conversion, and drafter residency lifecycle
• fix batched verifier visibility masks

Matched Strix Halo result

• 122,879 prompt tokens, 256 output tokens, all 6 experts
• 17.513 to 22.513 client decode tok/s: +28.6%
• output SHA unchanged; zero swap
• Vulkan reference is 32.60 tok/s, so this reaches 69% and does not claim parity

PFlash smoke test

• 8,192 DS4 tokens became 1,739 effective tokens
• Qwen scoring took 2.18 seconds
• PFlash is lossy prompt compression and is disabled for matched true-context results

Verification

• HIP build passes on gfx1151
• feature-gate suite passes
• full DeepSeek4 unit and GPU suite passes
• packed q2 through q5 indexer results are bit-identical to the generic kernel

[cubic-dev-ai]

4 issues found across 10 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/deepseek4/deepseek4_backend.cpp">

<violation number="1" location="server/src/deepseek4/deepseek4_backend.cpp:2648">
P1: When the previous request leaves GPU work in flight, `park()` frees DeepSeek weights and cache before synchronization runs. Synchronize both backends before calling `park()` to prevent queued kernels from accessing freed buffers.</violation>

<violation number="2" location="server/src/deepseek4/deepseek4_backend.cpp:2661">
P2: When `load_drafter()` fails after creating its backend, `pflash_drafter_loaded_` remains false and the context backend is never freed. Free `pflash_drafter_ctx_` on every failed load before restoring the target.</violation>
</file>

<file name="server/src/common/model_capabilities.h">

<violation number="1" location="server/src/common/model_capabilities.h:79">
P2: With a DeepSeek4 `--target-devices` split, this row admits `--prefill-compression`, but the request reaches `DeepSeek4LayerSplitAdapter::compress()`. That inherited default returns `{}`, so `apply_pflash_compression()` returns HTTP 500; gate split placements or implement the adapter path before setting this flag.</violation>
</file>

<file name="server/src/deepseek4/deepseek4_fused_verify.inc">

<violation number="1" location="server/src/deepseek4/deepseek4_fused_verify.inc:545">
P1: When fused verification uses `q > 1` after the SWA ring wraps, this predicate enables sparse attention even though the graph appends preserved raw rows after the compressed rows. The sparse kernel classifies that suffix as compressed and can drop visible preserved rows, so keep sparse attention off for batched lanes or add an explicit preserved-row boundary to the kernel layout.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: When the previous request leaves GPU work in flight, park() frees DeepSeek weights and cache before synchronization runs. Synchronize both backends before calling park() to prevent queued kernels from accessing freed buffers.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/deepseek4/deepseek4_backend.cpp, line 2648:

<comment>When the previous request leaves GPU work in flight, `park()` frees DeepSeek weights and cache before synchronization runs. Synchronize both backends before calling `park()` to prevent queued kernels from accessing freed buffers.</comment>

<file context>
@@ -2599,17 +2607,150 @@ GenerateResult DeepSeek4Backend::restore_and_generate_impl(
+    if (load_request == nullptr) return results;
+
+    const bool was_parked = parked_;
+    if (!load_request->skip_park && !parked_ &&
+        !park(ParkTarget::TargetModel)) {
+        return results;
</file context>

Suggested change

	if (!load_request->skip_park && !parked_ &&
	!park(ParkTarget::TargetModel)) {
	return results;
	}
	if (backend_) ggml_backend_synchronize(backend_);
	if (spec_backend_) ggml_backend_synchronize(spec_backend_);
	if (backend_) ggml_backend_synchronize(backend_);
	if (spec_backend_) ggml_backend_synchronize(spec_backend_);
	if (!load_request->skip_park && !parked_ &&
	!park(ParkTarget::TargetModel)) {
	return results;
	}

[cubic-dev-ai]

P1: When fused verification uses q > 1 after the SWA ring wraps, this predicate enables sparse attention even though the graph appends preserved raw rows after the compressed rows. The sparse kernel classifies that suffix as compressed and can drop visible preserved rows, so keep sparse attention off for batched lanes or add an explicit preserved-row boundary to the kernel layout.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/deepseek4/deepseek4_fused_verify.inc, line 545:

<comment>When fused verification uses `q > 1` after the SWA ring wraps, this predicate enables sparse attention even though the graph appends preserved raw rows after the compressed rows. The sparse kernel classifies that suffix as compressed and can drop visible preserved rows, so keep sparse attention off for batched lanes or add an explicit preserved-row boundary to the kernel layout.</comment>

<file context>
@@ -534,13 +534,30 @@ static bool ds4_build_fused_verify_graph(
+        // top-k boundary, preserving fused-graph replay.
+        const bool sparse_attention =
+            ds4_env_flag("DFLASH_DS4_SPARSE_DECODE_FLASH") && ratio > 0 &&
+            padded > 2 * w.n_indexer_top_k;
+        const DeepSeek4AttentionImpl attention_impl = sparse_attention
+            ? DeepSeek4AttentionImpl::SparseFlash
</file context>

Suggested change

	padded > 2 * w.n_indexer_top_k;
	padded > 2 * w.n_indexer_top_k && lane_q == 1;

[cubic-dev-ai]

P2: When load_drafter() fails after creating its backend, pflash_drafter_loaded_ remains false and the context backend is never freed. Free pflash_drafter_ctx_ on every failed load before restoring the target.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/deepseek4/deepseek4_backend.cpp, line 2661:

<comment>When `load_drafter()` fails after creating its backend, `pflash_drafter_loaded_` remains false and the context backend is never freed. Free `pflash_drafter_ctx_` on every failed load before restoring the target.</comment>

<file context>
@@ -2599,17 +2607,150 @@ GenerateResult DeepSeek4Backend::restore_and_generate_impl(
+        release_pflash_drafter();
+    }
+    if (!pflash_drafter_loaded_) {
+        if (!load_drafter(load_request->drafter_path, 999,
+                          load_request->drafter_gpu,
+                          pflash_drafter_ctx_)) {
</file context>

Suggested change

	if (!load_drafter(load_request->drafter_path, 999,
	load_request->drafter_gpu,
	pflash_drafter_ctx_)) {
	std::fprintf(stderr, "[deepseek4-pflash] load failed: %s\n",
	dflash27b_last_error());
	if (!load_request->skip_park && !was_parked) {
	unpark(ParkTarget::TargetModel);
	}
	return results;
	}
	if (!load_drafter(load_request->drafter_path, 999,
	load_request->drafter_gpu,
	pflash_drafter_ctx_)) {
	std::fprintf(stderr, "[deepseek4-pflash] load failed: %s\n",
	dflash27b_last_error());
	dflash::common::free_drafter(pflash_drafter_ctx_);
	if (!load_request->skip_park && !was_parked) {
	unpark(ParkTarget::TargetModel);
	}
	return results;
	}

[cubic-dev-ai]

P2: With a DeepSeek4 --target-devices split, this row admits --prefill-compression, but the request reaches DeepSeek4LayerSplitAdapter::compress(). That inherited default returns {}, so apply_pflash_compression() returns HTTP 500; gate split placements or implement the adapter path before setting this flag.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/common/model_capabilities.h, line 79:

<comment>With a DeepSeek4 `--target-devices` split, this row admits `--prefill-compression`, but the request reaches `DeepSeek4LayerSplitAdapter::compress()`. That inherited default returns `{}`, so `apply_pflash_compression()` returns HTTP 500; gate split placements or implement the adapter path before setting this flag.</comment>

<file context>
@@ -76,7 +76,7 @@ inline constexpr ArchCapabilities kArchCapabilities[] = {
     {"qwen3",      false, false, true,  false,   kNever, kNever, kNever, kNever, kNever, kNever, kNever},
     {"gemma4",     true,  false, false, false,   kMono, kNever, kNever, kNever, kBoth, kNever, kNever},
-    {"deepseek4",  true,  false, false, false,   kNever, kNever, kNever, kNever, kNever, kNever, kNever},
+    {"deepseek4",  true,  false, true,  false,   kNever, kNever, kNever, kNever, kNever, kNever, kNever},
 };
 
</file context>