Skip to content

Close the four effectful mutation survivors, and the deadlock behind the three timeouts - #1014

Merged
thekevinbot merged 5 commits into
mainfrom
claude/tackle-1012-6lbxz4
Sep 2, 2026
Merged

thekevinbot merged 5 commits into
mainfrom
claude/tackle-1012-6lbxz4

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Aug 21, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1012.

The four mutants #975's audit left unresolved, plus the three scanner.rs timeouts it flagged as worth a look. Three of the four were real coverage at a tier that could always reach them; the fourth is not a gap at all, and the proof is short. Tests and docs only -- no packages/rust/src change.

RootedFs::hash -> None and -> Some([0; 32])

Both are observable through the parsed path-table's racy window, so packages/rust/tests/persist_parsed_path_table.rs now drives it on purpose. pin_inside_the_racy_window sets a file's mtime ahead of the cache's snapshot_ns, which is exactly the condition is_trusted cannot settle on the stat tuple: the reuse decision falls through to the content hash.

Each mutant then has its own counter-example.

  • A corpus pinned into the window and left alone must still parse nothing. A hash that cannot be computed cannot confirm, so -> None re-parses all of it (parses(): 8 != 0). The test asserts the pinned mtimes are still in the future after the cold run, so it cannot pass vacuously by having missed the window.
  • An in-place edit of the same length with the mtime restored leaves the stat tuple byte-identical. fs::write reuses the inode, and FileStat's ctime_ns is created() -- a birth time no rewrite moves -- so size, mtime, ctime, inode and dev all match and only a hash that varies with content re-parses the file. That kills both constants; Some([1; 32]), which the audit did not list, dies with them. Size, mtime and creation time are asserted unchanged before the second run, so a filesystem that did disturb the tuple fails the test loudly instead of passing for the wrong reason.

load_state's delete !

The --extension override guard is wrong in both directions, and neither was observed anywhere: with the flag present the mutant drops the launcher's resolved paths and stops suppressing the config's own entries; with the flag absent it suppresses a config's [[dirsql.extension]] entries that nothing overrode.

extensions.rs covers both behaviors at the builder surface, but the choice between them is the CLI's, so the builder tests structurally cannot see it. packages/rust/tests/extensions_e2e.rs spawns the real binary over the testext fixture once per direction. Under the mutant they fail with the two distinct diagnostics you would want:

no such function: dirsql_testext_answer in SELECT dirsql_testext_answer() AS a
failed to load config: failed to load extension '.../does/not/exist.so'

wait_for_shutdown, both arms

cli/run.rs:651 (the unix arm) was already caught -- but only by wait_until_ready timing out after 10s, because a server that returns from its own signal wait exits before any readiness poll can observe it. That is an incidental never-became-ready panic standing in for a contract docs/reference/cli.md states plainly ("runs until it receives SIGINT (Ctrl-C) or SIGTERM"; exit 0 = "Clean shutdown after SIGINT / SIGTERM"). cli_e2e.rs's new test waits on the startup banner instead -- which the mutant still prints -- looks again a beat later, and asserts the process is alive. Same mutant, killed in 0.6s by an assertion that names the contract, then SIGTERM and exit 0.

cli/run.rs:664 is the #[cfg(not(unix))] twin, and no test on this gate can kill it. The gate runs on Linux, where that item is never compiled, so its body cannot be observed by anything. Proof rather than argument -- plant a compile_error! in it:

#[cfg(not(unix))]
async fn wait_for_shutdown() -> std::io::Result<()> {
    compile_error!("this arm is not compiled on unix");
}

cargo build --features cli still succeeds. The e2e test above would kill it on a Windows runner (a shipped target), so this is a platform boundary, not test debt. No exemption added, on the same reasoning as the audit's seven proved equivalents: mutation is diff-scoped, so it only resurfaces if someone edits that line.

The three scanner timeouts were one deadlocked test

walk (two) and scan_directory reported Timeout, which the gate reads as inconclusive rather than as a survivor -- and the mutants were describing the suite accurately. async_sdk.rs's sync_backed_methods_before_ready_error parked background init on a Barrier::new(2) whose second arrival is the extract closure. A scan that matches no file never calls the closure, so the test's own release waited for a party that was never coming and wedged the whole test binary. Running every test binary in the tree under walk -> empty() confirmed it: all of them failed fast, async_sdk was the only hang.

The barrier becomes a Mutex<Gate> + Condvar bounded on both sides. The assertions are untouched, the release is now a flag flip that cannot block, and a starved scan fails in 30s naming the cause:

init never reached the extract closure: the scan matched no file, so this test
never exercised a parked init

The general lesson is in agents/reference/testing-gates.md now: a mutant that times out is worth reading as a claim about the suite's waits before it is written off as engine slowness.

Verification

Every claim above was first checked by hand -- each mutation applied in the working tree, the suite re-run, the specific assertion written for it observed failing -- and then confirmed by re-running cargo mutants over all 11 mutants at those five sites under CI's own scoping (--features cli, no -C --lib, so the whole cargo test suite judges each one):

11 mutants tested in 9m: 1 missed, 9 caught, 1 unviable
site mutation now
parsed_vtab.rs:276 hash -> None caught
parsed_vtab.rs:276 hash -> Some([0; 32]) caught
parsed_vtab.rs:276 hash -> Some([1; 32]) (not in the audit's list) caught
cli/run.rs:558 delete ! in load_state caught
cli/run.rs:651 wait_for_shutdown -> Ok(()) (unix) caught -- now by the contract, in 0.6s, not by a 10s readiness timeout
scanner.rs:18 scan_directory -> vec[] and its two siblings caught, all three
scanner.rs:128 walk -> ::std::iter::empty() caught
scanner.rs:128 walk -> ::std::iter::once(Default::default()) unviable -- walkdir::DirEntry has no Default, so it never compiled; it was never a real mutant
cli/run.rs:664 wait_for_shutdown -> Ok(()) (#[cfg(not(unix))]) missed, by construction -- see above

The run's wall clock is the other half of the timeout finding: 11 mutants in 9 minutes, where the scanner ones used to burn the engine's entire timeout budget apiece and come back inconclusive.

Red/green note: these tests close mutation gaps in behavior that already exists, so they pass on arrival by construction -- there is no new behavior to fail against. The RED they are written for is the mutant, and each was observed failing before and passing after, which is the same shape #1013 used for the seven gaps it closed.

Changes

  • packages/rust/tests/persist_parsed_path_table.rs -- pin_inside_the_racy_window plus the two hash counter-examples
  • packages/rust/tests/extensions_e2e.rs (new) -- the CLI's two extension sources, one test per direction
  • packages/rust/tests/cli_e2e.rs -- the run-until-signalled contract
  • packages/rust/tests/async_sdk.rs -- the parked-init handshake, bounded
  • agents/reference/testing-gates.md -- what closed each survivor, the compile_error! proof for the cfg'd twin, and the timeout lesson

Manual exercise

The e2e tests are themselves real CLI runs (spawned binary, real dlopen, real signals), and beyond them the mutation work was driven by hand: each of the five sites was mutated in the working tree and the suite re-run to watch the specific assertion fail, and the compile_error! probe above was compiled to completion rather than reasoned about.

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: N/A -- no packages/python change
  • TypeScript SDK e2e: N/A -- no packages/ts change
  • Rust core e2e (if applicable): pass -- cargo test --features cli whole suite green (775 unit + every integration/e2e binary)
  • packages/python/e2e-attestations/BRANCH.json receipt written if packages/python changed -- not required, unchanged
  • packages/ts/e2e-attestations/BRANCH.json receipt written if packages/ts changed -- not required, unchanged
  • Command(s) run: cargo test --manifest-path packages/rust/Cargo.toml --features cli; cargo clippy --workspace -- -D warnings; cargo fmt --all -- --check; uv run --project internals/checks dirsql-checks preflight --conventions .github/workflows/dirsql-rust-ci.yml
  • Result summary: all green. The rust lane's gates (colocated-test, unit-lint, integration-lint, mutation) pass; mutation reports no mutatable changed lines — engine not run, the honest verdict for a diff that changes no source (not the ambiguous all-caught line). packaging skips locally, as it needs a built artifact.

The diff touches no packages/rust/src, packages/python, or packages/ts file, so neither binding's e2e attestation is staled and no receipt is owed.

Changelog / Migrations

  • Changelog fragment added under ROOT/PKG/changelog.d/ for each changed package (or: skip-changelog trailer on a commit with reason) -- skip-changelog: tests and docs only, no observable behavior change
  • Migration fragment added under ROOT/PKG/migrations.d/ (or: not required -- additive/bugfix only) -- not required, no API or behavior change

PARITY.md is untouched: no SDK public API changes, so neither drift introduced nor restored.

claude added 5 commits August 21, 2026 15:37
`RootedFs::hash` is the one input to the reuse decision that no tier
observed: two mutants (`-> None`, `-> Some([0; 32])`) survived every
suite in the #975 audit. Both are reachable from the integration tier
once a file is put where the stat tuple cannot settle it.

`pin_inside_the_racy_window` sets a file's mtime ahead of the cache's
`snapshot_ns`, so `is_trusted` falls through to the content hash. From
there each mutant has its own counter-example:

- a corpus pinned into the window and left alone must still parse
  nothing. A hash that cannot be computed cannot confirm, so `None`
  re-parses all of it.
- an in-place edit of the same length with the mtime restored leaves the
  stat tuple byte-identical -- `fs::write` reuses the inode and
  `FileStat`'s `ctime_ns` is `created()`, a birth time no rewrite moves
  -- so only a hash that varies with content re-parses it. That kills
  both constants, `Some([1; 32])` included.

The second test asserts size, mtime and creation time are unchanged
before it re-runs, so a filesystem that did disturb the tuple fails it
loudly rather than passing for the wrong reason.

Refs #1012

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsEEptS8FEY23bGiTXTKKS
`load_state`'s `if !cfg.extension.is_empty()` guard picks which of the
two sources of extensions the builder gets. Deleting the `!` -- a
surviving mutant from the #975 audit -- is wrong in both directions and
neither was observed anywhere: with `--extension` present the launcher's
resolved paths are dropped and the config's own entries stop being
suppressed; with it absent the config's `[[dirsql.extension]]` entries
are suppressed by nothing.

`extensions.rs` covers both behaviors at the builder surface, but the
choice between them is the CLI's, so the builder tests cannot see it.
These spawn the real binary over the `testext` fixture once per
direction: a config-declared extension must load with no flag in sight,
and a flag must replace a config entry rather than add to it -- proved
by pointing the config at a path that does not exist, which would fail
the build if it were still honored.

Refs #1012

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsEEptS8FEY23bGiTXTKKS
`docs/reference/cli.md` promises `dirsql server` runs until it receives
SIGINT or SIGTERM and then exits 0. Nothing asserted it. The unix
`wait_for_shutdown` mutant (`-> Ok(())`) was caught only by
`wait_until_ready` timing out after 10s -- a server that returns from
its own signal wait exits before any readiness poll can observe it, so
the kill was an incidental never-became-ready panic rather than a
statement about shutdown.

This waits on the startup banner instead, which the mutant still prints,
then looks again a beat later and asserts the process is still there.
Same mutant, killed in 0.6s by the assertion that names the contract.

Refs #1012

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsEEptS8FEY23bGiTXTKKS
Three scanner mutants -- `walk` (two) and `scan_directory` -- reported
`Timeout` in the #975 audit, which the gate reads as inconclusive rather
than as survivors. They were describing the suite accurately.

`sync_backed_methods_before_ready_error` parked background init on a
`Barrier::new(2)` whose second arrival is the extract closure. A scan
that matches no file never calls the closure, so the test's own release
waited for a party that was never coming and wedged the whole test
binary. Every other binary in the tree fails fast under the same mutant;
this was the only hang.

The barrier becomes a `Mutex<Gate>` + `Condvar` bounded on both sides.
The assertions are unchanged, the release is now a flag flip that cannot
block, and a starved scan fails in 30s naming the cause -- so the three
mutants become ordinary kills.

Refs #1012

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsEEptS8FEY23bGiTXTKKS
Three of the four are coverage at a tier that could always reach them,
and the doc now says where each one lives and what makes it observable
-- the pinned racy window, the two directions of the `--extension`
guard, the banner-then-liveness shape that replaces a readiness timeout.

The fourth is worth being precise about because it is not a gap: the
`wait_for_shutdown` mutant at `cli/run.rs:664` is the
`#[cfg(not(unix))]` twin, and the gate runs on Linux, where that item is
never compiled. Plant a `compile_error!` in its body and
`cargo build --features cli` still succeeds. No exemption, on the same
reasoning as the seven proved equivalents: mutation is diff-scoped.

Also records what the three scanner timeouts turned out to be, and the
general lesson -- a mutant that times out is worth reading as a claim
about the suite's waits before it is written off as engine slowness.

skip-changelog: tests and docs only, no observable behavior change

Refs #1012

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsEEptS8FEY23bGiTXTKKS

Copy link
Copy Markdown
Collaborator Author

CI Gate is red here for a reason outside this PR, and I am not pushing anything for it.

The gate failed 37s in, before any workflow it monitors had finished, with:

Unresolvable check names: [".github/workflows/dirsql-python-ci.yml :: python-sdk / static (dynamic matrix)", …13 entries…].
willfire sees these jobs but cannot say what checks they will create, so the
predicted set is incomplete and the gate cannot compare against it.

Every one of the 13 is a (dynamic matrix) job in a workflow this diff does not touch — including release-ci.yml :: precheck / build / build, which has nothing to do with a test-only change.

pr-monitor@v1 is a rolling tag and it moved between 15:12 and 15:42 UTC today:

#1013 (04:56) and #1000 (15:12) — both green #1014 (15:42) — red
pr-monitor@v1 resolves to c4cf535 da8181d
willfire 0.1.2 0.1.11
prediction granularity workflow runs (5 predicted workflow runs completed successfully) individual check names

0.1.11 predicts check names rather than workflow runs, and hard-errors on any job whose matrix it cannot statically expand. The reusable testing-conventions matrix (fromJSON(needs.detect.outputs.languages)) and putitoutthere's precheck / build matrix are both computed at run time, so nothing in this repo can make them statically expandable — this will red every PR here until willfire can resolve them.

Two ways out, both yours to pick:

  1. Fix it upstream in willfire — resolve or tolerate a dynamic matrix instead of erroring.
  2. Pin the gate here as a stopgap: thekevinscott/pr-monitor@v1 → @c4cf535ad67e072d17c0dba1709ae1836f72c040 in .github/workflows/pr-monitor.yml, the last SHA observed green. One line, and testable on the PR that makes it, but it walks back the deliberate rolling-@v1 choice, so I have not done it here.

The substantive checks on this PR are unaffected — the rust lane's gates and both binding lanes are running normally. Happy to push the pin if you want it.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Four mutants in effectful core survive every test tier (found by the #975 blind-window audit)

2 participants