Repository navigation
test(move): bound move timing relative to the runner, not absolutely - #19
Merged
Merged
Conversation
The share of block windows under 1 ms and the accessors' 99th percentile are properties of the machine as much as of a move: a runner three times slower than usual moved a third as many blocks and had 88% and 92% of its windows under 1 ms, with every grain whole and every PMD kept, and failed. The bound that always applies is now relative to the same guest: half of the kernel's windows must end within 16 times the median time of a cache-cold 2 MiB copy from node 0 to node 1 in user space, which a window is plus a zap and a page cache swap. An added 5 ms per window fails it; a guest on one or two host CPUs does not. STRICT=1 adds the absolute bounds back for a quiet local machine (WINDOW_UNDER_1MS_PERMILLE, STALL_P99_US); otherwise they are printed. Stale and torn grains, blocks left behind, lost PMD mappings and THP splits still fail every run. A missed grain is one a reader could not check before the writer reused its slot 600 ms later, which a guest starved of CPU shows with or without moves; more than 5% of the grains checked still fails. A block refused BUSY, because a fault still held a reference on a starved guest, is moved by up to three further calls, as a caller would, before the move counts as leaving it.
…tries A fixed 5% of missed grains was loose enough for a starved guest and equally loose for a move that held readers up: a grain is missed when its reader is held 600 ms, whatever holds it. The bound is now relative to the same job's run without moves, passed as MISSING_BASE_PERMILLE: twice its share plus 0.5%. A starved guest misses grains in both runs; a move path that stalls readers misses them only in the moving one. A block refused BUSY is retried, which hides BUSY becoming the rule; at most 1% of the move calls may now be such retries. The moves are paced to half of the writer's CPU, on which they run, so moving does not take the readers' CPU time and a reader that misses grains is one a move held up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tests/move.shfailed on a slow CI runner on timing alone: it moved 383 passes where an ordinary runner moves about 1300, 91.7% and 88.1% of the kernel's block windows were under 1 ms (the bound was 99%), and the accessors' 99th percentile was 1.50 and 1.23 ms, while no grain was stale, torn or missing and every PMD mapping was kept.What fails a run now:
MISSING_BASE_PERMILLE, passed likeSTALL_P99_US) plus 0.5%. A missed grain is one a reader could not check before the writer reused its slot 600 ms later; a starved guest misses grains in both runs, a move that holds readers up only in the moving one;WINDOW_COPY_FACTOR(16) times the median of 64 cache-cold user-space copies of 2 MiB from a folio on node 0 to one on node 1 (copy_2m_us_p50, printed). A window is that copy, page by page and slower under KASAN, plus a zap and a page cache swap;STRICT=1, for a quiet local machine: 99% of windows under 1 ms (WINDOW_UNDER_1MS_PERMILLE) and the accessors' 99th percentile under 1 ms or twice the no-move run's (STALL_P99_US). Otherwise both are printed only.A move that a fault's reference made leave a block (BUSY) is retried up to three times, as a caller would;
busy_retriesis printed. One such refusal showed on a guest given two host CPUs.After review: a module that stalls read faults 600 ms (one in four, within 100 ms of a move) fails on missed grains (4034 missed, 271 checked; 0 without moves); one that refuses every 10th block BUSY fails on retries (3615 of 14463 calls). Ordinary guest: 0 missed, 0 retries; guest on 2 host CPUs: 0 missed, 1 retry in 11585 calls; both 17/17. The moves are paced to half of the writer's CPU they run on, so moving does not take the readers' CPU.
Local runs before review (6.18.42 debug kernel, two-node guest),
move.shin default mode:mdelay(5)added to every block windowThe ordinary run would also have failed the old 99% rule (98.7%). The 1-CPU case passes everything about the moves; its readers fell behind. With
STRICT=1the same 1-CPU guest fails the absolute bounds, as intended.numa.sh: 19/19.🤖 Generated with Claude Code