Repository navigation
test(rust): drop the cold-run wall-clock floor from the persist cache test - #1029
Merged
Merged
Conversation
… test The floor asserted the cold run took at least 3 seconds, which measures the machine rather than the corpus: 4,000 files parse in ~2.2s on a fast box, so `cargo test --workspace --features cli` went red locally while staying green on slower CI runners. The guarantee it stood in for is already carried machine-independently. The parse counters assert the cold run spawned the parser for every file and the warm run for none, which is the cache-reuse claim itself, and the warm/cold ratio bound of 10% is exactly the statement that per-file parsing dominated the cold run. Nothing the floor added survives its removal.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1021
unchanged_second_run_reuses_the_cache_instead_of_reparsinggated its ratioassertion behind
cold >= Duration::from_secs(3). That floor measures themachine, not the corpus: on this box 4,000 files parse in ~2.2s, so the test
failed locally in the direction of "your hardware is fast" while passing on
slower CI runners.
Approach
Removed the floor and kept everything else.
The floor stood in for "the corpus is big enough that the warm/cold ratio is
signal, not noise". Two assertions already in the test carry that guarantee
without reference to wall-clock speed:
fx.parses() == CORPUSafter the cold run and== 0after the warm one. That is cache reuse measured directly, one count per
parser process, immune to how fast those processes run. It is the counter
option the issue prefers, and it needed no new instrumentation — the fixture
already counts parser invocations by appending a byte per run.
warm / cold <= 0.10is the statement that per-fileparse work was at least 90% of the cold run. A corpus too small for the ratio
to mean anything is a corpus whose fixed overhead dominates, and that fails
the ratio directly. The absolute floor added nothing on top.
Rejected the other two options. A timer-resolution multiple would be dead code:
Instantresolves to nanoseconds on Linux, so any honest multiple of it sitsorders of magnitude below a run that spawns 4,000 processes, and the assertion
would never fire. Raising
CORPUSis worse than the status quo — it slows thetest for everyone and only relocates the machine-speed threshold.
CORPUSstays at 4,000. It is no longer sized against a stopwatch, but theratio still needs per-file work to dominate; measured margin below is wide.
Proof the remaining assertions can fail
Broke cache reuse locally by making
parsed_cache::is_freshreturnfalseunconditionally (nothing is ever served from the cache). The test went red on
the counter, not on timing:
Restored
is_fresh; working tree clean apart from the test file.Separately confirmed the ratio assertion is live by temporarily setting
WARM_RATIOto0.0, which is also where the margin numbers come from:1.8% against a 10% bound. The ratio is self-normalizing — a slower runner
inflates both terms — so a machine where the cold run takes the 3s+ that CI
sees lands in the same place. The removed floor was the only absolute time in
the test.
Tests
cargo test -p dirsql --test persist_parsed_path_table— 8 passed (was 1failed before this change, on this machine).
cargo test --workspace --features cli— green, exit 0.cargo clippy --workspace --all-targets --features cli— no new warnings;zero mentions of the edited file.
cargo benchnot run: test-only change, no hot path touched.E2E Verification
N/A - test-only change (
packages/rust/tests/), which AGENTS.md lists under"Not substantial". No
packages/rust/srcchange, so neither package's[e2e].extra_scopeputs a receipt in scope.cargo test --workspace --features cli, greenpackages/python/e2e-attestations/<branch>.jsonreceipt written ifpackages/pythonchanged — N/A, unchangedpackages/ts/e2e-attestations/<branch>.jsonreceipt written ifpackages/tschanged — N/A, unchangedcargo test --workspace --features cliChangelog / Migrations
Not required.
changelog_gate's_is_exempttreats anything under<pkg>/tests/as a test tier, sopackages/rust/tests/is not source and thegate demands no fragment. No user-visible behavior changes, so no migration
fragment either.
<root>/<pkg>/changelog.d/for each changed package — N/A, test-only path is gate-exempt<root>/<pkg>/migrations.d/— not required, no behavior changeParity
No SDK public API touched;
PARITY.mdunchanged.Note
Previously two checks failed here for reasons that predated this branch, both
on the
Check one function per filestep:python-sdk / Static checks (python)and
typescript-sdk / Static checks (typescript). Both were the sharedtesting-conventions 0.0.111 breakage that reddened
mainand every open PR,not a defect in this change. #1023 carried the fix and has merged;
mainat612bcf1 shows both static-check jobs green. This branch has been merged up from
main, so those two checks now run against the fixed configuration.