Repository navigation
Quick-check baseline refreshed on an idle machine - #243
Merged
Merged
Conversation
…ed, both runs measured a loaded machine; Lagan features read is machine, not code (@Perf) Two runs of tools/bench_quick.py run on master c317c7e (AC, 15:19 and 15:30 local) read every case 6 to 15 % slower than the stored baseline (357808b, 2026-10-11), the single-thread C++ refine phases included (tile t=1 refine +15.5 %) on byte-identical meshes and a _core.so byte-identical to the one within band at 11:15 today. The machine was in interactive use (load average 3 to 4.5, WindowServer 30 %). The record run 2 saved is kept under docs/benchmarks/2026-10-12/baseline-refresh/ as baseline-ac-candidate-loaded-1530.json and the stored baseline is restored: a baseline that slow on unchanged code would hide a regression of that size. A waiter sampled the load for 45 minutes (1.50 to 5.11) and never found an idle minute to save a clean one. Lagan features read: the baseline's own commit and .so, run back to back with master three times each, reads 1.107 s against master's 1.103 s (both against 0.922 s stored); under cProfile open_features, read_source and query_features have the same call counts and times on both trees. Not a code change in io/repository.py or io/models.py (neither touches the read path); the machine today against the machine on 2026-10-11. 7 % of the run, not a hotspot by the 40 % rule. Evidence: README.md (method, versions, the two verdict tables, the features-read finding), the two run logs and the waiter's, the stored baseline as found, run 2's record, raw/ (delta table, plain runs, profile text, scripts). No code changed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… increment 38 (8f9289c) quick check NO CHANGE against it (@Perf) Run 4, 20:29 local, after a 14-minute wait for the 1-minute load average to read under 2.0 twice: tools/bench_quick.py run --save-baseline on this tree (master f09b986 merged in; _core Release .so 55bb3993..., byte-identical to the afternoon's). The tool printed NO BASELINE: input $RASPUTIN_DATA/sweden_glo30_cache changed, because the Swedish batch's fetch added GLO-30 tiles to the Lagan case's cache directory; the saved baseline carries the new fingerprint. Judged against the stored 357808b record with the old inputs substituted: no mesh changed, max_error equal on every case, every refine phase within 1.5 %, totals +0.7 to +4.7 % (within band); one line beyond its band, lagan features read 0.92 -> 1.03 s (+12.1 %, band 5.5 %), shown this afternoon to be machine-side (the baseline's own commit reads the same), cause within the machine not measured. The afternoon's +6 to +15 % on every phase was the loaded machine. Run 5, 20:31: tools/bench_quick.py run --tree .claude/worktrees/threads (increment 38, --threads and --memory-budget as settings, Python only, no C++ or CMake change against its merge base 5d2e706; same .so): NO CHANGE, exit 0; no MESH CHANGED, SLOWER, FASTER, BROKEN or NOT JUDGED line, so every mesh SHA-256 equals the baseline's and no phase moved beyond its band. Evidence: docs/benchmarks/2026-10-12/baseline-refresh/README.md (evening section), quick-run-4-idle-save.log, quick-run-5-increment-38.log, baseline-ac-new-969c49e2.json, raw/delta-old-357808b1-vs-new-969c49e2.md, raw/evening_idle_save_then_38.sh. The installed baseline: docs/benchmarks/quick/baseline-ac.json. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… sentence fixed, by the main session Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refreshes the quick-check baseline (
docs/benchmarks/quick/baseline-ac.json) on masterf09b9863, measured by @Perf on an idle machine. An earlier attempt on a loaded machine was measured and declined; its record is kept. Increment 38 (#239) was checked against the new baseline: no change.Prose and benchmark data only. Review round 1 approved.
🤖 Generated with Claude Code