Skip to content

perf(cpu): isolate Raspberry Pi 5 polling contention - #361

Draft
richiejp wants to merge 11 commits into
mudler:mainfrom
richiejp:row/KERNEL-GEMM-CPU-ELEM-A76
Draft

perf(cpu): isolate Raspberry Pi 5 polling contention#361
richiejp wants to merge 11 commits into
mudler:mainfrom
richiejp:row/KERNEL-GEMM-CPU-ELEM-A76

Conversation

@richiejp

@richiejp richiejp commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Row

KERNEL-GEMM-CPU-ELEM-A76 — tracked by #284. The benchmark-control prerequisite is tracked by #293.

Scope

This draft contains the reviewed concurrency-one blocking-c1 benchmark control and the Raspberry Pi 5 W0/C1 records. It preserves poll as the default and rejects blocking-c1 above concurrency one.

It also carries the latest-main integration repairs needed to preserve #206 prompt pretokenization, exact prompt IDs, and atomic ordered wave admission after rebasing.

This PR does not contain the general C2 event wait, the F16 neon-zip / neon-p8 candidates, a default kernel change, or new assembly.

What changed

  • Added the explicit poll / blocking-c1 output-wait selector, CLI validation, JSON/text reporting, and exact-token tests.
  • Bound the real unconditional get_output_nowait evaluation before the C1 blocking call.
  • Preserved current-main pretokenization and wave-admission behavior through the rebase.
  • Added mutation-sensitive prompt-ID and checker tests required by the exact PR-size gate.
  • Updated STATUS, BENCHMARKS, USAGE, NOW, the owning specs/matrices, and indexed Pi evidence. The llama.cpp floor remains unmet.

Evidence

  • Current-head native benchmark suite: 15/15 cases, 124/124 assertions.
  • Current-head scoped native CTest: 5/5 (test_engine_core_proc, test_output_processor, test_async_llm, test_bench, test_cpu_threadpool).
  • Prompt-ID assignment deletion, nowait bypass, blocking dispatch inversion, report lies, special-token removal, and wave-publication mutations turn RED and restore byte-for-byte.
  • Exact PR-size semantic-evidence gate passes in an environment where its sanitized PATH can resolve Git.
  • Historical same-binary Pi C1 A/B: 12/12 clean, token-identical, unthrottled. At four compute threads, blocking improved prefill 1.973x, per-stream decode 2.149x, and output throughput 2.151x; T3 was neutral. Four-thread polling was unstable, so this is a causal discriminator rather than a new binding floor.
  • Exact-current-head AArch64/QEMU and Pi rebenchmark are owed after the next main rebase.

Review state

  • Latest-main C1 conflict integration and its repair passed fresh mutation-driven review. The reviewed repair tree is byte-identical to this PR head.
  • Full repository preflight still reports inherited stale live rows and environment/history fixture failures; all change-specific record, document, trailer, registration, PR-size, and scoped test gates pass.

Follow-up work not in this PR

  • C2 event wait candidate 518316436: product, native gates, and nine mutations passed review; a record-only anchor correction is required before it can pass final review.
  • F16 C++ candidate b120a71b5: native/cross-build/disassembly checks passed; review found that the physical-selection fixture needs an independent correctness golden before Pi benchmarking.
  • After those repairs: combine reviewed heads, build under AArch64 QEMU, run same-binary Pi baseline / neon-zip / neon-p8 A/B, retain only a token-exact physical winner, and compare the full model against llama.cpp. Assembly remains forbidden until the C++ path clears that floor.

Rebase status

This branch was rebased through upstream 11d45330b; upstream main is now 6dbedf9f3, and GitHub currently reports merge conflicts. Rebase, fresh conflict review, and exact-head QEMU/Pi evidence remain required before marking the PR ready.

FOLLOWING_AGENTS_PROTOCOL

@richiejp
richiejp force-pushed the row/KERNEL-GEMM-CPU-ELEM-A76 branch from c53700e to cc4b195 Compare August 11, 2026 13:25
FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Transplant the reviewed Pi profile onto current upstream without treating the
historical binary or T4/T3 arm as a one-variable polling control. Record the
prompt-content mismatch and link the separately tracked mudler#293 discriminator.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Record the isolated current-upstream implementation worktree and move the
scoped polling discriminator from READY to ACTIVE without moving the broad
SERVE-CLI-BENCH row.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Reapply the same-binary concurrency-one polling discriminator unchanged onto
current upstream. Keep poll as the default, reject blocking-c1 above c1, and
record both the selected mode and blocking-call count in text and JSON output.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Record the hash-verified current-source Pi control: blocking the sole collector
restores the four-thread arm while the three-thread control stays neutral and
exact tokens hold. Keep the accepted llama.cpp floor unchanged because the
pathological polling arm exceeds the binding spread limit, and leave C2 to
fresh review and implementation.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Add a test-only wait trace that proves each blocking-c1 wait follows an empty
nowait probe, cover the human-readable report, and repair the row anchors. The
trace compiles out of production; native and AArch64 vllm-bench artifacts retain
the reviewed executable bytes.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Make the output-wait test observer inspect the result only after the same
unconditional AsyncLLM call used by production. Replace the source-text guard
with a deterministic runtime probe whose bypass mutation fails normally, while
preserving the measured AArch64 Release binary byte-for-byte.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Record the reviewed post-rebase operator gate without replacing the older binary's provenance. The current-main same-binary Pi control confirms four-core polling contention while T3 remains neutral, but the unstable poll arm still forbids a binding floor change.

Correct the low-severity evidence precision finding by reporting the raw perf-stat T3 CPU-utilization medians to two decimal places.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Require both benchmark admission arms to expose populated, exact per-request prompt IDs, so dropping the production assignment fails normally. Add semantic mutation evidence for the A76 inventory and STATUS ratchet changes, re-anchor the SERVE-CLI-BENCH row, and project the confirmed C1 polling cause without changing the llama.cpp floor.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
@richiejp
richiejp force-pushed the row/KERNEL-GEMM-CPU-ELEM-A76 branch from cc4b195 to 858560f Compare August 13, 2026 12:33
@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

@richiejp This branch currently conflicts with main. The C1 control and evidence look coherent, but the PR itself notes that exact-current-head AArch64/QEMU and Pi remeasurement are still owed. Please rebase onto current main and refresh those gates before merge.

@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

Reviewed as part of an external-contributor sweep. This branch is from 2026-08-11 and main has moved a long way underneath it, so it needs a rebase before it can be considered.

Nine conflicts, and two of them are against files that no longer exist:

modify/delete  docs/STATUS.md                          (deleted on main)
modify/delete  tests/scripts/test_check_public_doc_tables.py  (deleted on main)
content        .agents/coordination.md
content        .agents/kernel-matrix.md
content        .agents/roadmap_v1.md
content        docs/BENCHMARKS.md
content        docs/USAGE.md
content        scripts/check-agent-record.py
content        tests/scripts/test_agent_record.py

What changed under you:

  • docs/STATUS.md and its doc-checkpoint gate were retired. Drop those edits rather than restoring the file.
  • scripts/check-agent-record.py and its test have been reworked since. Please rebase onto the current versions rather than merging your copies over them, since a merged checker can end up enforcing something neither side intended.

For the record-file conflicts (.agents/*.md), the repository's rule is to take the complete target-branch version and then re-apply your scoped edit, rather than accepting an automatic three-way merge. Verify unrelated keys stay byte-for-byte equal afterwards.

The measurement work itself is not in question here; this is purely about the branch's age. windows-msvc-cpu and windows-msvc-vulkan are red tree-wide and are not caused by this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants