Skip to content

test(rust): drop the cold-run wall-clock floor from the persist cache test - #1029

Merged
thekevinscott merged 2 commits into
mainfrom
claude/1021-cold-floor-r7q2m
Sep 4, 2026
Merged

thekevinscott merged 2 commits into
mainfrom
claude/1021-cold-floor-r7q2m

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Sep 2, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1021

unchanged_second_run_reuses_the_cache_instead_of_reparsing gated its ratio
assertion behind cold >= Duration::from_secs(3). That floor measures the
machine, not the corpus: on this box 4,000 files parse in ~2.2s, so the test
failed locally in the direction of "your hardware is fast" while passing on
slower CI runners.

Approach

Removed the floor and kept everything else.

The floor stood in for "the corpus is big enough that the warm/cold ratio is
signal, not noise". Two assertions already in the test carry that guarantee
without reference to wall-clock speed:

  • The parse counters: fx.parses() == CORPUS after the cold run and == 0
    after the warm one. That is cache reuse measured directly, one count per
    parser process, immune to how fast those processes run. It is the counter
    option the issue prefers, and it needed no new instrumentation — the fixture
    already counts parser invocations by appending a byte per run.
  • The ratio bound itself: warm / cold <= 0.10 is the statement that per-file
    parse work was at least 90% of the cold run. A corpus too small for the ratio
    to mean anything is a corpus whose fixed overhead dominates, and that fails
    the ratio directly. The absolute floor added nothing on top.

Rejected the other two options. A timer-resolution multiple would be dead code:
Instant resolves to nanoseconds on Linux, so any honest multiple of it sits
orders of magnitude below a run that spawns 4,000 processes, and the assertion
would never fire. Raising CORPUS is worse than the status quo — it slows the
test for everyone and only relocates the machine-speed threshold.

CORPUS stays at 4,000. It is no longer sized against a stopwatch, but the
ratio still needs per-file work to dominate; measured margin below is wide.

Proof the remaining assertions can fail

Broke cache reuse locally by making parsed_cache::is_fresh return false
unconditionally (nothing is ever served from the cache). The test went red on
the counter, not on timing:

assertion `left == right` failed: an unchanged tree must not re-run the parser for any file
  left: 4000
 right: 0

Restored is_fresh; working tree clean apart from the test file.

Separately confirmed the ratio assertion is live by temporarily setting
WARM_RATIO to 0.0, which is also where the margin numbers come from:

the warm run must cost at most 0% of the cold one; cold 2.220608991s, warm 39.204276ms (1.8%)

1.8% against a 10% bound. The ratio is self-normalizing — a slower runner
inflates both terms — so a machine where the cold run takes the 3s+ that CI
sees lands in the same place. The removed floor was the only absolute time in
the test.

Tests

  • cargo test -p dirsql --test persist_parsed_path_table — 8 passed (was 1
    failed before this change, on this machine).
  • cargo test --workspace --features cli — green, exit 0.
  • cargo clippy --workspace --all-targets --features cli — no new warnings;
    zero mentions of the edited file.
  • cargo bench not run: test-only change, no hot path touched.

E2E Verification

N/A - test-only change (packages/rust/tests/), which AGENTS.md lists under
"Not substantial". No packages/rust/src change, so neither package's
[e2e].extra_scope puts a receipt in scope.

  • Ran e2e suites locally for every affected SDK — N/A
  • Python SDK e2e: N/A
  • TypeScript SDK e2e: N/A
  • Rust core e2e (if applicable): covered by cargo test --workspace --features cli, green
  • packages/python/e2e-attestations/<branch>.json receipt written if packages/python changed — N/A, unchanged
  • packages/ts/e2e-attestations/<branch>.json receipt written if packages/ts changed — N/A, unchanged
  • Command(s) run: cargo test --workspace --features cli
  • Result summary: all suites pass, exit 0

Changelog / Migrations

Not required. changelog_gate's _is_exempt treats anything under
<pkg>/tests/ as a test tier, so packages/rust/tests/ is not source and the
gate demands no fragment. No user-visible behavior changes, so no migration
fragment either.

  • Changelog fragment added under <root>/<pkg>/changelog.d/ for each changed package — N/A, test-only path is gate-exempt
  • Migration fragment added under <root>/<pkg>/migrations.d/ — not required, no behavior change

Parity

No SDK public API touched; PARITY.md unchanged.

Note

Previously two checks failed here for reasons that predated this branch, both
on the Check one function per file step: python-sdk / Static checks (python)
and typescript-sdk / Static checks (typescript). Both were the shared
testing-conventions 0.0.111 breakage that reddened main and every open PR,
not a defect in this change. #1023 carried the fix and has merged; main at
612bcf1 shows both static-check jobs green. This branch has been merged up from
main, so those two checks now run against the fixed configuration.

… test

The floor asserted the cold run took at least 3 seconds, which measures the
machine rather than the corpus: 4,000 files parse in ~2.2s on a fast box, so
`cargo test --workspace --features cli` went red locally while staying green
on slower CI runners.

The guarantee it stood in for is already carried machine-independently. The
parse counters assert the cold run spawned the parser for every file and the
warm run for none, which is the cache-reuse claim itself, and the warm/cold
ratio bound of 10% is exactly the statement that per-file parsing dominated
the cold run. Nothing the floor added survives its removal.
@thekevinscott
thekevinscott merged commit 06ef38a into main Sep 4, 2026
77 checks passed
@thekevinscott
thekevinscott deleted the claude/1021-cold-floor-r7q2m branch September 4, 2026 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

persist_parsed_path_table's cold-run floor fails on fast machines

2 participants