Skip to content

refactor(python): resolve extensions through the core's plan contract - #1125

Merged
thekevinscott merged 2 commits into
mainfrom
claude/1120-py-ext
Sep 17, 2026
Merged

thekevinscott merged 2 commits into
mainfrom
claude/1120-py-ext

Conversation

@thekevinbot

Copy link
Copy Markdown
Collaborator

Slice B of #1111: the Python SDK stops carrying its own port of extension resolution and reaches the core contract landed in #1124.

What moved

Python kept three decisions the core already knows how to make:

  • is this entry a package or a file → dirsql._dirsql.is_bare_name
  • which configs parse, does any name a package, what does each literal path resolve to against its own config's directory → dirsql._dirsql.plan_config_extensions
  • which installed file is the loadable → dirsql._dirsql.select_loadable

It keeps only the two halves the core cannot have: reading the config files off disk, and locating an installed package with importlib.util.find_spec.

The PyO3 layer stays pure — Python reads each config and passes (abs_path, contents | None) pairs; the core parses and plans. Adding an fs-reading plan_config_files to the core was the obvious alternative and was rejected: a Rust unit test may call only super:: and pure std, so an fs-reading function cannot be unit-tested, and diff-scoped cargo-mutants would leave a surviving mutant in the untested reader.

Seven internal modules go away (is_bare_name, has_bare_name, load_extension_entries, resolve_entries, platform_patterns, resolve_config_extensions, toml_module) against two added (read_config, plan_entry_path, split apart to satisfy one-function-per-file). resolve_config_extension_specs — the singular resolver — had no production caller and goes with them. The tomli dependency on Python 3.10 goes too: the core parses the TOML now.

Why there is no pushed-RED commit

This is a behavior-preserving refactor plus removals, both exempt from the pushed-RED CI gate. The existing suites cover every path and stayed green throughout: same resolved paths, same intervene-or-not decision, and the core's SelectError renders the Python wording verbatim, so the error messages are byte-identical.

One normalization, which I judged inconsequential — please push back if you disagree. The core treats a backslash as a path separator on every host, where is_bare_name used os.altsep (None on POSIX). is_bare_name checks the loadable-suffix list first, so the only value that reclassifies is one with a backslash and no loadable suffix — e.g. a\b. On the old path that was a bare name, went to importlib.util.find_spec, and raised could not resolve extension package 'a\\b'; on the new path it is a literal path and fails at load. Both error. No migration fragment on that basis.

Platform scoping is preserved: select_loadable is handed a PLATFORM_SUFFIXES list (.dylib on macOS, .dll/.pyd on Windows, .so elsewhere) that mirrors the deleted _platform_patterns, so a package shipping loadables for several platforms still resolves rather than going ambiguous. is_bare_name keeps the full four-suffix list, matching the deleted _LOADABLE_SUFFIXES.

Manual verification

Built the repo's tests/fixtures/testext cdylib (registers dirsql_testext_answer() -> 42), staged it as mypkg/testext0.so in a temp dir on PYTHONPATH, and wrote a .dirsql.toml whose [[dirsql.extension]].path is the bare name mypkg.

$ dirsql query -c .dirsql.toml 'SELECT dirsql_testext_answer() AS answer, (SELECT count(*) FROM files) AS files'
[{"answer":42,"files":1}]

Same through the SDK, both entry points:

DirSQL(config=...)                            -> [{'answer': 42, 'files': 1}]
DirSQL(root, extensions=[{'path': 'mypkg'}])  -> [{'programmatic': 42}]

The bare name resolved through plan_config_extensions → select_loadable to the absolute .so, which SQLite then loaded.

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: pass (37 passed)
  • TypeScript SDK e2e: N/A — no packages/ts change
  • Rust core e2e (if applicable): N/A — no packages/rust change
  • packages/python/e2e-attestations/<branch>.json receipt written if packages/python changed (just e2e-attest-python)
  • packages/ts/e2e-attestations/<branch>.json receipt written if packages/ts changed (just e2e-attest-ts) — N/A
  • Command(s) run: just e2e-attest-python; cargo test --manifest-path packages/python/Cargo.toml --features extension-module; uv run pytest dirsql tests
  • Result summary: 37 e2e passed, 91 colocated + 131 integration passed, 15 binding-crate Rust tests passed.

Changelog / Migrations

  • Changelog fragment added under <root>/<pkg>/changelog.d/ for each changed package (or: skip-changelog trailer on a commit with reason)
  • Migration fragment added under <root>/<pkg>/migrations.d/ (or: not required — additive/bugfix only) — not required: behavior-preserving refactor, every deleted module is private to the package and none is exported from dirsql/__init__.py.

Parity

Restoring parity, with intentional drift that slice C closes. PARITY.md gains a paragraph on where the planning lives. Python now reaches the shared core contract; TypeScript still carries its own port and is tracked by #1121, the next slice.

Gates run

one-function-per-file, unit-lint, colocated-test, integration-lint, unit coverage (100%), unit mutation (7 mutants, 0 survivors), declared-deps, plus the rust-python-binding lane's colocated-test and unit-lint. ruff check, ruff format --check, ty check --error all, cargo fmt, and cargo clippy --workspace --features cli -- -D warnings all clean.

Fixes #1120

The Python SDK carried its own port of extension resolution: whether an
entry names a package or a file, which literal path each entry resolves to
against its own config's directory, and which installed file is *the*
loadable. None of that is host-specific, and since #1119 it lives once in
`dirsql::extension_resolution`.

Python now reaches it through three new PyO3 bindings --
`plan_config_extensions`, `select_loadable`, `is_bare_name` -- and keeps
only the two halves the core cannot have: reading the config files, and
locating an installed package with `importlib.util.find_spec`.

Seven internal modules go away (`is_bare_name`, `has_bare_name`,
`load_extension_entries`, `resolve_entries`, `platform_patterns`,
`resolve_config_extensions`, `toml_module`) against two added
(`read_config`, `plan_entry_path`), and with them the `tomli` dependency
on Python 3.10 -- the core parses the TOML now.

Behavior-preserving: same resolved paths, same error messages (the core's
`SelectError` renders the Python wording verbatim), same intervene-or-not
decision. One inconsequential normalization: the core treats a backslash as
a path separator on every host, where `is_bare_name` used `os.altsep`. Only
a value with a backslash *and* no loadable suffix reclassifies, and such a
value errors on both the old and the new path.

Fixes #1120
@thekevinscott
thekevinscott merged commit 6e50a78 into main Sep 17, 2026
49 checks passed
@thekevinscott
thekevinscott deleted the claude/1120-py-ext branch September 17, 2026 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#1111 Slice B: migrate Python extension resolution to Rust plan

2 participants