Skip to content

feat(rust): add mergify-py-shim with embedded Python fallback#1272

Merged
mergify[bot] merged 1 commit intomainfrom
devs/jd/worktree-rust-port/add-mergify-py-shim-embedded-python-fallback--0c73b039
Apr 24, 2026
Merged

feat(rust): add mergify-py-shim with embedded Python fallback#1272
mergify[bot] merged 1 commit intomainfrom
devs/jd/worktree-rust-port/add-mergify-py-shim-embedded-python-fallback--0c73b039

Conversation

@jd
Copy link
Copy Markdown
Member

@jd jd commented Apr 20, 2026

Adds the Python shim that lets the Rust binary dispatch every
command to the existing Python implementation. Phase 1.1's
deliverable: a Rust binary that, from a user's perspective,
behaves identically to the Python CLI.

What the shim does

  1. Embeds the full mergify_cli/ Python source at compile time
    via include_dir!. Today the payload is ~400KB of .py files
    plus tests; Phase 1.5 will add a build-time filter to drop the
    tests tree and any __pycache__ directories.
  2. On first invocation, extracts to
    ~/.cache/mergify/py/<CARGO_PKG_VERSION>/mergify_cli/ under a
    file lock. Extraction is atomic: write to a temp dir, rename,
    then write a .complete sentinel. Concurrent first-runs
    serialize safely; the fast path (sentinel present) touches only
    one exists() syscall.
  3. Invokes python3 -m mergify_cli <args> with the extracted
    directory prepended to PYTHONPATH and PYTHONSAFEPATH=1
    so the cwd never shadows the extracted tree. Exit code and
    stdin/stdout/stderr are forwarded transparently.

Cache keying

Today the cache is keyed on CARGO_PKG_VERSION. Since that's
0.0.0 during dev, the cache is shared across builds — clear
~/.cache/mergify/py/ by hand when iterating on Python source
in a worktree. Phase 1.5 stamps a real version + git SHA, after
which every build invalidates cleanly.

Binary wire-up

crates/mergify-cli/src/main.rs now delegates every argv tail
to mergify_py_shim::run. When Phase 1.3+ ports the first
command, native Rust dispatch intercepts it before falling back
to the shim for un-ported commands. Phase 6 deletes the shim
crate entirely.

Known gap

python3 on PATH must have mergify_cli's runtime dependencies
(httpx, rich, pydantic, etc.) importable — the shim
embeds only the source, not site-packages. Locally you get this
via uv run; in production the Python deps have to come from
somewhere (PyPI install, a bootstrap venv, or a pip-backed
first-run step). Phase 1.5 decides the mechanism. For Phase 1.1
the shim mechanics are validated in isolation.

Tests

Three unit tests in the shim crate cover the extraction logic:
files get written, nested directories preserved, second call is
idempotent (sentinel fast-path). End-to-end smoke: uv run ./target/release/mergify --help extracts and produces Python's
help output.

Binary size: 2.6 MB. Well under the 15 MB design target.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

@jd
Copy link
Copy Markdown
Member Author

jd commented Apr 20, 2026

This pull request is part of a Mergify stack:

# Pull Request Link
1 feat(rust): add mergify-py-shim with embedded Python fallback #1272 👈
2 feat(rust): add mergify-core foundation types (Phase 1.2a) #1280
3 feat(rust): add mergify-core::http with retry + typed errors (Phase 1.2b) #1281
4 feat(rust): port config validate to native Rust (Phase 1.3) #1282
5 test: add port-inventory guard to catch un-ported Python commands #1296
6 ci: add Rust release workflow targeting GitHub Release assets #1297
7 feat(rust): port config simulate to native Rust (Phase 1.3b) #1298
8 feat(rust): port ci scopes-send to native Rust (Phase 1.4) #1300
9 feat(rust): port queue pause and unpause to native Rust (Phase 1.5) #1301
10 feat(rust): port ci git-refs and ci queue-info to native Rust (Phase 1.6) #1302

@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 20, 2026

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 🤖 Continuous Integration

Wonderful, this rule succeeded.
  • all of:
    • check-success=ci-gate

🟢 👀 Review Requirements

Wonderful, this rule succeeded.
  • any of:
    • #approved-reviews-by>=2
    • author = dependabot[bot]
    • author = mergify-ci-bot
    • author = renovate[bot]

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|ui)(?:\(.+\))?:

🟢 🔎 Reviews

Wonderful, this rule succeeded.
  • #changes-requested-reviews-by = 0
  • #review-requested = 0
  • #review-threads-unresolved = 0

🟢 📕 PR description

Wonderful, this rule succeeded.
  • body ~= (?ms:.{48,})

@jd jd force-pushed the devs/jd/worktree-rust-port/add-mergify-py-shim-embedded-python-fallback--0c73b039 branch from c7daf9b to b187fb4 Compare April 21, 2026 08:00
@jd jd force-pushed the devs/jd/worktree-rust-port/scaffold-cargo-workspace-rust-ci--4b278cf7 branch from 3cfbdbd to 47fd5c0 Compare April 21, 2026 08:00
@jd
Copy link
Copy Markdown
Member Author

jd commented Apr 21, 2026

Revision history

# Type Changes Date
1 initial c7daf9b 2026-04-21 08:00 UTC
2 rebase c7daf9b → b187fb4 2026-04-21 08:00 UTC
3 rebase b187fb4 → 8f404aa 2026-04-21 09:10 UTC
4 rebase 8f404aa → 27194e6 2026-04-21 13:13 UTC
5 rebase 27194e6 → 1823ce5
6 rebase 1823ce5 → 7617830
7 content 7617830 → 6c4294a
8 rebase 6c4294a → 91c97d0
9 rebase 91c97d0 → b10a54b 2026-04-23 19:02 UTC

@mergify mergify Bot had a problem deploying to Mergify Merge Protections April 21, 2026 08:00 Failure
@mergify mergify Bot requested a review from a team April 21, 2026 08:06
@jd jd force-pushed the devs/jd/worktree-rust-port/add-mergify-py-shim-embedded-python-fallback--0c73b039 branch from b187fb4 to 8f404aa Compare April 21, 2026 09:10
@mergify mergify Bot had a problem deploying to Mergify Merge Protections April 21, 2026 09:11 Failure
@jd jd force-pushed the devs/jd/worktree-rust-port/scaffold-cargo-workspace-rust-ci--4b278cf7 branch from 576b4cc to a5202f3 Compare April 21, 2026 13:13
@jd jd force-pushed the devs/jd/worktree-rust-port/add-mergify-py-shim-embedded-python-fallback--0c73b039 branch from 8f404aa to 27194e6 Compare April 21, 2026 13:13
@mergify mergify Bot had a problem deploying to Mergify Merge Protections April 21, 2026 13:14 Failure
@jd jd force-pushed the devs/jd/worktree-rust-port/add-mergify-py-shim-embedded-python-fallback--0c73b039 branch from 27194e6 to 1823ce5 Compare April 22, 2026 13:36
@mergify mergify Bot had a problem deploying to Mergify Merge Protections April 22, 2026 13:36 Failure
Base automatically changed from devs/jd/worktree-rust-port/scaffold-cargo-workspace-rust-ci--4b278cf7 to main April 22, 2026 15:27
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 22, 2026

@jd this pull request is now in conflict 😩

@mergify mergify Bot added the conflict label Apr 22, 2026
Copilot AI review requested due to automatic review settings April 22, 2026 18:25
@jd jd force-pushed the devs/jd/worktree-rust-port/add-mergify-py-shim-embedded-python-fallback--0c73b039 branch from 1823ce5 to 7617830 Compare April 22, 2026 18:25
@mergify mergify Bot had a problem deploying to Mergify Merge Protections April 22, 2026 18:25 Failure
@mergify mergify Bot removed the conflict label Apr 22, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a new Rust “Python shim” crate that embeds the existing mergify_cli/ Python sources, extracts them to a per-user cache on first run, and forwards all CLI invocations to python3 -m mergify_cli so the Rust binary behaves like the Python CLI during the port.

Changes:

  • Added crates/mergify-py-shim to embed, extract (locked + atomic), and invoke the Python CLI, with unit tests for extraction behavior.
  • Updated crates/mergify-cli’s main.rs to delegate all args to the shim and propagate the Python exit code.
  • Wired the new shim crate into the Rust workspace dependency graph (Cargo.toml + Cargo.lock).

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
crates/mergify-py-shim/src/lib.rs Implements embedding/extraction/invocation logic and adds unit tests.
crates/mergify-py-shim/Cargo.toml Defines the new shim crate and its dependencies.
crates/mergify-cli/src/main.rs Delegates CLI execution to the shim and forwards exit status/errors.
crates/mergify-cli/Cargo.toml Adds dependency on mergify-py-shim.
Cargo.lock Records new Rust dependency resolution for the shim crate.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/mergify-py-shim/src/lib.rs Outdated
Comment thread crates/mergify-py-shim/src/lib.rs Outdated
Comment thread crates/mergify-py-shim/src/lib.rs Outdated
@jd jd marked this pull request as ready for review April 23, 2026 07:28
@jd jd force-pushed the devs/jd/worktree-rust-port/add-mergify-py-shim-embedded-python-fallback--0c73b039 branch from 7617830 to 6c4294a Compare April 23, 2026 07:32
@mergify mergify Bot had a problem deploying to Mergify Merge Protections April 23, 2026 07:32 Failure
kozlek
kozlek previously approved these changes Apr 23, 2026
Copy link
Copy Markdown
Contributor

@kozlek kozlek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really impressive ! Impatient to see how you will deal with the python runtime dependencies !
What about relying on uv to install them at runtime ? (uv needs to be installed though...)

Adds the Python shim that lets the Rust binary dispatch every
command to the existing Python implementation. Phase 1.1's
deliverable: a Rust binary that, from a user's perspective,
behaves identically to the Python CLI.

## What the shim does

1. Embeds the full ``mergify_cli/`` Python source at compile time
   via ``include_dir!``. Today the payload is ~400KB of .py files
   plus tests; Phase 1.5 will add a build-time filter to drop the
   tests tree and any ``__pycache__`` directories.
2. On first invocation, extracts to
   ``~/.cache/mergify/py/<CARGO_PKG_VERSION>/mergify_cli/`` under a
   file lock. Extraction is atomic: write to a temp dir, rename,
   then write a ``.complete`` sentinel. Concurrent first-runs
   serialize safely; the fast path (sentinel present) touches only
   one ``exists()`` syscall.
3. Invokes ``python3 -m mergify_cli <args>`` with the extracted
   directory prepended to ``PYTHONPATH`` and ``PYTHONSAFEPATH=1``
   so the cwd never shadows the extracted tree. Exit code and
   stdin/stdout/stderr are forwarded transparently.

## Cache keying

Today the cache is keyed on ``CARGO_PKG_VERSION``. Since that's
``0.0.0`` during dev, the cache is shared across builds — clear
``~/.cache/mergify/py/`` by hand when iterating on Python source
in a worktree. Phase 1.5 stamps a real version + git SHA, after
which every build invalidates cleanly.

## Binary wire-up

``crates/mergify-cli/src/main.rs`` now delegates every argv tail
to ``mergify_py_shim::run``. When Phase 1.3+ ports the first
command, native Rust dispatch intercepts it before falling back
to the shim for un-ported commands. Phase 6 deletes the shim
crate entirely.

## Known gap

``python3`` on PATH must have mergify_cli's runtime dependencies
(``httpx``, ``rich``, ``pydantic``, etc.) importable — the shim
embeds only the source, not site-packages. Locally you get this
via ``uv run``; in production the Python deps have to come from
somewhere (PyPI install, a bootstrap venv, or a ``pip``-backed
first-run step). Phase 1.5 decides the mechanism. For Phase 1.1
the shim mechanics are validated in isolation.

## Tests

Three unit tests in the shim crate cover the extraction logic:
files get written, nested directories preserved, second call is
idempotent (sentinel fast-path). End-to-end smoke: ``uv run
./target/release/mergify --help`` extracts and produces Python's
help output.

Binary size: 2.6 MB. Well under the 15 MB design target.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Change-Id: I0c73b039c98011716bc72a441e92e57c6fab9926
@jd jd force-pushed the devs/jd/worktree-rust-port/add-mergify-py-shim-embedded-python-fallback--0c73b039 branch from 91c97d0 to b10a54b Compare April 23, 2026 19:02
@mergify mergify Bot deployed to Mergify Merge Protections April 23, 2026 19:03 Active
@mergify mergify Bot requested a review from a team April 24, 2026 09:05
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 24, 2026

Merge Queue Status

This pull request spent 6 minutes 46 seconds in the queue, including 6 minutes 24 seconds running CI.

Required conditions to merge

mergify Bot added a commit that referenced this pull request Apr 24, 2026
@mergify mergify Bot added the queued label Apr 24, 2026
@mergify mergify Bot merged commit 7345ba9 into main Apr 24, 2026
23 checks passed
@mergify mergify Bot deleted the devs/jd/worktree-rust-port/add-mergify-py-shim-embedded-python-fallback--0c73b039 branch April 24, 2026 09:28
@mergify mergify Bot removed the queued label Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants