Skip to content

fix(pflash): score the user query and reject non-finite scores - #679

Draft
Graffioh wants to merge 3 commits into
Luce-Org:mainfrom
Graffioh:codex/pflash-query-nonfinite
Draft

fix(pflash): score the user query and reject non-finite scores#679
Graffioh wants to merge 3 commits into
Luce-Org:mainfrom
Graffioh:codex/pflash-query-nonfinite

Conversation

@Graffioh

@Graffioh Graffioh commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • maps the final user message into the drafter-rendered token stream and uses its last eight tokens as the scorer query
  • fails closed if that user query cannot be mapped, rather than falling back to the assistant/chat suffix
  • keeps scorer-query selection separate from DFLASH_COMPRESS_QUERY_TOKENS, which remains the lexical-anchor window
  • propagates the query window through local Qwen3/Qwen3.5, layer-split, and IPC paths
  • assembles scorer-Q windows that cross a 4,096-token forward chunk boundary
  • checks graph execution and score readback, rejecting NaN/Inf values before top-k reduction

Why

The previous fixed lookahead=8 window landed on the rendered assistant suffix rather than on the question. On Strix Halo/gfx1151, corrupted HIP forward values could also reach the scorer; NaN comparisons then left earlier maxima in place and silently produced a selection.

Validation

  • rebased onto Luce-Org/lucebox@ef82fadf
  • HIP Release build for gfx1151: dflash_server, test_tokenizer_harness, and test_server_unit
  • HIP_VISIBLE_DEVICES=1 ./build-hip-gfx1151/test_server_unit: 439 passed, 0 failed
  • live 16K Strix Halo smoke:
    • uncompressed control: 2/2 correct
    • PFlash: fails closed with explicit non-finite score diagnostics
    • query mapping log confirms the actual user-query window, e.g. [16374, 16382), rather than the rendered assistant suffix

The underlying gfx1151 numeric corruption is intentionally not hidden by this PR; this makes it observable and prevents quality results from being reported from invalid scores.

Paired long-context benchmark campaign: Luce-Org/luce_box#85.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant