[thekevinbot]

Fixes #1008

Problem

prepare_persist dropped the old user tables the moment it opened the cache — before anything had loaded the configured [[dirsql.extension]] entries. Dropping a virtual table needs its module registered on that connection, and a module that arrives with an extension is registered nowhere else, so DROP TABLE notes_vec answered no such module: vec0. Editing the ddl of a --persist cache holding one left it unopenable on every later run until .dirsql was removed by hand.

#956 covered exactly this flow — but with FTS5, whose module is compiled into SQLite and so is registered on every connection whether or not dirsql configured anything. A built-in module stood in for a loadable one, and the asymmetry went unnoticed.

Change

The sweep is deferred, not duplicated. prepare_persist now only flags it (needs_sweep); finish_build performs it after the extensions and functions are on the connection and before the batch that re-creates the tables. That is the invariant the bug was missing: the drop sees exactly the module set the batch that created those tables saw. Registering the declared [[dirsql.function]] workers first comes free from the same placement, so a schema object referencing one is droppable too.

Loading the extensions a second time in prepare_persist was the alternative and is wrong: it is one connection, carried from prepare_resolved into finish_build, so a second load_extension re-enters the init function and sqlite3_create_module rejects a name it already holds.

Nothing between the two phases reads the swept tables: reconcile_scan is pure over the already-read file index, and the warm catalog probe that decides whether to re-run a batch comes after the new sweep site.

Tests

Pushed RED first (c8ba191), confirmed red in CI for the right reason (run 32441733664: a_rebuild_sweeps_a_cache_holding_an_extension_backed_virtual_table ... FAILED — no such module: dirsql_testext_vtab), then implemented in 40ea3cf.

• Integration — tests/ddl_batch.rs::a_rebuild_sweeps_a_cache_holding_an_extension_backed_virtual_table, the sibling Persist rebuild wedges on a cache holding an extension-provided virtual table: no such module: vec0 #1008 asks for: same shape as the FTS5 test, but the module comes from a loaded extension.
• E2E — tests/ddl_batch_e2e.rs::a_persisted_cache_holding_an_extension_virtual_table_survives_a_ddl_edit, the real CLI with --persist across a ddl edit, asserting the next run is clean too (the wedge was permanent, so one rebuild squeaking through would not prove it fixed).
• Unit — prepare_persist_cold_start_reports_rebuild now asserts needs_sweep; prepare_persist_warm_cache_asks_for_no_sweep pins the other side; an_edited_ddl_sweeps_the_cache_before_rebuilding covers the drop from the build side (an un-dropped table reads as warm, so the edited batch never re-runs).

Supporting changes: the tests/fixtures/testext extension now registers a virtual table module (dirsql_testext_vtab) alongside its scalar function — the thing no built-in can substitute for — and build_fixture_extension moves to tests/common/mod.rs so the integration and e2e binaries share one implementation.

Manual verification

Reproduced the issue's shape with the fixture extension in place of sqlite-vec (/tmp/dirsql-1008-manual, .dirsql.toml declaring [[dirsql.extension]] + a notes_vec vtab, on-file = "python3 rows.py {path}"):

$ dirsql query "SELECT count(*) AS n FROM notes_vec" -c ./.dirsql.toml --persist
[{"n":1}]

# add `CREATE INDEX notes_slug ON notes(slug);` to the ddl
$ dirsql query "SELECT count(*) AS n FROM notes_vec" -c ./.dirsql.toml --persist
[{"n":1}]                                    # was: no such module: dirsql_testext_vtab

$ dirsql query "SELECT name FROM pragma_index_list('notes')" -c ./.dirsql.toml --persist
[{"name":"notes_slug"}]                      # the batch really did re-run after the sweep

$ dirsql query "SELECT count(*) AS n FROM notes_vec" -c ./.dirsql.toml --persist
[{"n":1}]                                    # and stays openable

Parity

No parity impact. The fix is entirely inside the shared Rust core with no public API change, so all three SDKs pick it up identically — no drift introduced, none restored. PARITY.md needs no edit.

E2E Verification

• Ran e2e suites locally for every affected SDK
• Python SDK e2e: pass (32 passed, 1 skipped)
• TypeScript SDK e2e: pass (5 files, 16 tests)
• Rust core e2e: pass (--tests covers the *_e2e.rs suites; 40 test binaries green)
• packages/python/e2e-attestations/cc-keen-knuth-f11gyq.json receipt written
• packages/ts/e2e-attestations/cc-keen-knuth-f11gyq.json receipt written
• Command(s) run:
  • cargo test --manifest-path packages/rust/Cargo.toml --features cli --tests
  • cargo test --features cli --lib -p dirsql (770 unit tests)
  • cd packages/python && uvx testing-conventions e2e attest 'uv run python -m pytest tests/e2e/ -x -q'
  • cd packages/ts && uvx testing-conventions e2e attest 'pnpm test:e2e'
  • uvx testing-conventions unit mutation --language rust --base origin/main --config testing-conventions.toml packages/rust → no surviving mutants (1 mutant(s) tested)
  • dirsql-checks preflight --gate colocated-test --gate unit-lint --gate integration-lint and --gate e2e-verify → 0 failing pairs
  • cargo clippy --workspace --features cli -- -D warnings, cargo fmt --all -- --check
• Result summary: green across every suite and gate run locally.

Changelog / Migrations

• Changelog fragment added: packages/rust/changelog.d/2026-08-21-sweep-loads-extensions.md
• Migration fragment — not required; a bugfix restoring intended behavior, no public API change and no behavior change to preserve