feat(rust): add mergify-py-shim with embedded Python fallback#1272
Conversation
|
This pull request is part of a Mergify stack:
|
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 🤖 Continuous IntegrationWonderful, this rule succeeded.
🟢 👀 Review RequirementsWonderful, this rule succeeded.
🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
🟢 🔎 ReviewsWonderful, this rule succeeded.
🟢 📕 PR descriptionWonderful, this rule succeeded.
|
c7daf9b to
b187fb4
Compare
3cfbdbd to
47fd5c0
Compare
Revision history
|
b187fb4 to
8f404aa
Compare
576b4cc to
a5202f3
Compare
8f404aa to
27194e6
Compare
27194e6 to
1823ce5
Compare
|
@jd this pull request is now in conflict 😩 |
1823ce5 to
7617830
Compare
There was a problem hiding this comment.
Pull request overview
Introduces a new Rust “Python shim” crate that embeds the existing mergify_cli/ Python sources, extracts them to a per-user cache on first run, and forwards all CLI invocations to python3 -m mergify_cli so the Rust binary behaves like the Python CLI during the port.
Changes:
- Added
crates/mergify-py-shimto embed, extract (locked + atomic), and invoke the Python CLI, with unit tests for extraction behavior. - Updated
crates/mergify-cli’smain.rsto delegate all args to the shim and propagate the Python exit code. - Wired the new shim crate into the Rust workspace dependency graph (Cargo.toml + Cargo.lock).
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/mergify-py-shim/src/lib.rs | Implements embedding/extraction/invocation logic and adds unit tests. |
| crates/mergify-py-shim/Cargo.toml | Defines the new shim crate and its dependencies. |
| crates/mergify-cli/src/main.rs | Delegates CLI execution to the shim and forwards exit status/errors. |
| crates/mergify-cli/Cargo.toml | Adds dependency on mergify-py-shim. |
| Cargo.lock | Records new Rust dependency resolution for the shim crate. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
7617830 to
6c4294a
Compare
6c4294a to
91c97d0
Compare
Adds the Python shim that lets the Rust binary dispatch every command to the existing Python implementation. Phase 1.1's deliverable: a Rust binary that, from a user's perspective, behaves identically to the Python CLI. ## What the shim does 1. Embeds the full ``mergify_cli/`` Python source at compile time via ``include_dir!``. Today the payload is ~400KB of .py files plus tests; Phase 1.5 will add a build-time filter to drop the tests tree and any ``__pycache__`` directories. 2. On first invocation, extracts to ``~/.cache/mergify/py/<CARGO_PKG_VERSION>/mergify_cli/`` under a file lock. Extraction is atomic: write to a temp dir, rename, then write a ``.complete`` sentinel. Concurrent first-runs serialize safely; the fast path (sentinel present) touches only one ``exists()`` syscall. 3. Invokes ``python3 -m mergify_cli <args>`` with the extracted directory prepended to ``PYTHONPATH`` and ``PYTHONSAFEPATH=1`` so the cwd never shadows the extracted tree. Exit code and stdin/stdout/stderr are forwarded transparently. ## Cache keying Today the cache is keyed on ``CARGO_PKG_VERSION``. Since that's ``0.0.0`` during dev, the cache is shared across builds — clear ``~/.cache/mergify/py/`` by hand when iterating on Python source in a worktree. Phase 1.5 stamps a real version + git SHA, after which every build invalidates cleanly. ## Binary wire-up ``crates/mergify-cli/src/main.rs`` now delegates every argv tail to ``mergify_py_shim::run``. When Phase 1.3+ ports the first command, native Rust dispatch intercepts it before falling back to the shim for un-ported commands. Phase 6 deletes the shim crate entirely. ## Known gap ``python3`` on PATH must have mergify_cli's runtime dependencies (``httpx``, ``rich``, ``pydantic``, etc.) importable — the shim embeds only the source, not site-packages. Locally you get this via ``uv run``; in production the Python deps have to come from somewhere (PyPI install, a bootstrap venv, or a ``pip``-backed first-run step). Phase 1.5 decides the mechanism. For Phase 1.1 the shim mechanics are validated in isolation. ## Tests Three unit tests in the shim crate cover the extraction logic: files get written, nested directories preserved, second call is idempotent (sentinel fast-path). End-to-end smoke: ``uv run ./target/release/mergify --help`` extracts and produces Python's help output. Binary size: 2.6 MB. Well under the 15 MB design target. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Change-Id: I0c73b039c98011716bc72a441e92e57c6fab9926
91c97d0 to
b10a54b
Compare
Merge Queue Status
This pull request spent 6 minutes 46 seconds in the queue, including 6 minutes 24 seconds running CI. Required conditions to merge
|
Adds the Python shim that lets the Rust binary dispatch every
command to the existing Python implementation. Phase 1.1's
deliverable: a Rust binary that, from a user's perspective,
behaves identically to the Python CLI.
What the shim does
mergify_cli/Python source at compile timevia
include_dir!. Today the payload is ~400KB of .py filesplus tests; Phase 1.5 will add a build-time filter to drop the
tests tree and any
__pycache__directories.~/.cache/mergify/py/<CARGO_PKG_VERSION>/mergify_cli/under afile lock. Extraction is atomic: write to a temp dir, rename,
then write a
.completesentinel. Concurrent first-runsserialize safely; the fast path (sentinel present) touches only
one
exists()syscall.python3 -m mergify_cli <args>with the extracteddirectory prepended to
PYTHONPATHandPYTHONSAFEPATH=1so the cwd never shadows the extracted tree. Exit code and
stdin/stdout/stderr are forwarded transparently.
Cache keying
Today the cache is keyed on
CARGO_PKG_VERSION. Since that's0.0.0during dev, the cache is shared across builds — clear~/.cache/mergify/py/by hand when iterating on Python sourcein a worktree. Phase 1.5 stamps a real version + git SHA, after
which every build invalidates cleanly.
Binary wire-up
crates/mergify-cli/src/main.rsnow delegates every argv tailto
mergify_py_shim::run. When Phase 1.3+ ports the firstcommand, native Rust dispatch intercepts it before falling back
to the shim for un-ported commands. Phase 6 deletes the shim
crate entirely.
Known gap
python3on PATH must have mergify_cli's runtime dependencies(
httpx,rich,pydantic, etc.) importable — the shimembeds only the source, not site-packages. Locally you get this
via
uv run; in production the Python deps have to come fromsomewhere (PyPI install, a bootstrap venv, or a
pip-backedfirst-run step). Phase 1.5 decides the mechanism. For Phase 1.1
the shim mechanics are validated in isolation.
Tests
Three unit tests in the shim crate cover the extraction logic:
files get written, nested directories preserved, second call is
idempotent (sentinel fast-path). End-to-end smoke:
uv run ./target/release/mergify --helpextracts and produces Python'shelp output.
Binary size: 2.6 MB. Well under the 15 MB design target.
Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com