Skip to content

fix(zipapp): exec the interpreter so signals reach the Python process - #4210

Open
gfrankliu wants to merge 3 commits into
bazel-contrib:mainfrom
gfrankliu:fix/zipapp-exec-signals
Open

gfrankliu wants to merge 3 commits into
bazel-contrib:mainfrom
gfrankliu:fix/zipapp-exec-signals

Conversation

@gfrankliu

@gfrankliu gfrankliu commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

The self-executable py_zipapp_binary launcher (zip_shell_template.sh) runs
the interpreter as a child process. Signals sent to the PID the caller started,
e.g. by a process supervisor stopping a long-running service, only reach the
launcher shell, so the Python program never sees them. If the supervisor then
escalates to SIGKILL, the shell dies, the Python program keeps running as an
orphan (still holding its ports and stdio), and the EXIT trap that removes
the temporary extraction directory never runs, leaking the whole extraction on
every restart. This is the zipapp equivalent of #2043, which #2047 fixed for
the non-zip bootstrap; the launcher still had a TODO pointing at it.

To fix, exec the interpreter, as the non-zip bootstrap does. Because exec
replaces the shell, the EXIT trap can no longer clean up the temporary
directory, so just before exec the launcher starts a small detached watcher
that removes the directory once the PID (now the Python process) exits. Unlike
the trap, this also works after SIGKILL. The watcher is double-forked so it
is not a child of the Python program (which could otherwise reap it via
os.wait()), and its stdio is redirected so callers capturing the output
(e.g. $(zipapp)) don't wait on it. It compares the process state and start time (from /proc or ps), so a zombie or a reused PID can't delay cleanup, and it ignores the signals sent to a whole process group (a terminal's Ctrl-C, a hangup, or a supervisor stopping the group) so it survives to clean up. The trap stays in place until the exec,
so early failures still clean up. Nothing changes when
RULES_PYTHON_EXTRACT_ROOT or RULES_PYTHON_BOOTSTRAP_VERBOSE is set; those
directories are intentionally kept.

  • Adds //tests/py_zipapp:zipapp_signals_test, which checks that the started
    PID is the Python process, that SIGTERM reaches it, that the extraction
    directory is removed after SIGTERM, after SIGKILL (before and after the
    process is reaped), and after SIGINT/SIGHUP/SIGTERM to the whole process
    group, and that exit codes propagate. Against the previous launcher it fails (39 != 35 for the PID
    check, and the SIGKILL case times out because the orphan keeps stdout open).

The self-executable `py_zipapp_binary` launcher (`zip_shell_template.sh`) runs
the interpreter as a child process. Signals sent to the PID the caller started,
e.g. by a process supervisor stopping a long-running service, only reach the
launcher shell, so the Python program never sees them. If the supervisor then
escalates to `SIGKILL`, the shell dies, the Python program keeps running as an
orphan (still holding its ports and stdio), and the `EXIT` trap that removes
the temporary extraction directory never runs, leaking the whole extraction on
every restart. This is the zipapp equivalent of bazel-contrib#2043, which bazel-contrib#2047 fixed for
the non-zip bootstrap; the launcher still had a TODO pointing at it.

To fix, `exec` the interpreter, as the non-zip bootstrap does. Because `exec`
replaces the shell, the `EXIT` trap can no longer clean up the temporary
directory, so just before `exec` the launcher starts a small detached watcher
that removes the directory once the PID (now the Python process) exits. Unlike
the trap, this also works after `SIGKILL`. The watcher is double-forked so it
is not a child of the Python program (which could otherwise reap it via
`os.wait()`), and its stdio is redirected so callers capturing the output
(e.g. `$(zipapp)`) don't wait on it. The trap stays in place until the `exec`,
so early failures still clean up. Nothing changes when
`RULES_PYTHON_EXTRACT_ROOT` or `RULES_PYTHON_BOOTSTRAP_VERBOSE` is set; those
directories are intentionally kept.

* Adds `//tests/py_zipapp:zipapp_signals_test`, which checks that the started
  PID is the Python process, that `SIGTERM` reaches it, that the extraction
  directory is removed after both `SIGTERM` and `SIGKILL`, and that exit codes
  propagate. Against the previous launcher it fails (`39 != 35` for the PID
  check, and the `SIGKILL` case times out because the orphan keeps stdout open).
Copilot AI balanced review requested due to automatic review settings October 3, 2026 23:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

PID polling can delay cleanup indefinitely, and test cleanup may signal an unrelated reused PID.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
What changed in this PR

Updates the zipapp launcher so signals reach Python directly while preserving temporary extraction cleanup.

Changes:

  • Executes Python in place and adds a detached cleanup watcher.
  • Adds signal, cleanup, PID, and exit-code tests.
  • Documents the user-visible fix.
File Description
python/​private/​zipapp/​zip_shell_template.sh Adds exec and asynchronous cleanup.
tests/​py_zipapp/​BUILD.bazel Registers signal-test targets.
tests/​py_zipapp/​zipapp_signals_test.py Tests signals and cleanup behavior.
tests/​py_zipapp/​signals_main.py Provides the test application.
news/​zipapp-exec-signals.fixed.md Adds the release note.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +91 to +93
while kill -0 "$launcher_pid" 2>/dev/null; do
sleep 1
done

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, fixed in d9a6687. The watcher now compares a process identity instead of kill -0: the state and start time from /proc/<pid>/stat on Linux, or ps -o stat=,lstart= elsewhere (e.g. macOS), falling back to kill -0 only if neither is available. A zombie counts as exited, and a reused PID has a different start time. Added test_extracted_files_removed_before_process_is_reaped, which SIGKILLs the process without reaping it; it failed with the kill -0 version and passes now.

Comment thread tests/py_zipapp/zipapp_signals_test.py Outdated
pid = int(pid_line.split("=", 1)[1])
# If the launcher doesn't exec, the Python process outlives it and keeps
# stdout open, so make sure it's killed too.
self.addCleanup(self._kill_pid, pid)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, fixed in d9a6687: the fallback kill of the reported Python PID is now only registered when it differs from proc.pid, i.e. the non-exec launcher case it exists for.

The cleanup watcher polled `kill -0`, which still succeeds while the exited
interpreter is a zombie and can't tell a reused PID from the original
process, so cleanup could wait on the caller reaping it or on an unrelated
process. Compare a process identity instead: the state and start time from
/proc/<pid>/stat (Linux) or `ps` (e.g. macOS), treating a zombie as exited.

* Adds a test that the extraction directory is removed before the killed
  process is reaped.
* The test only registers the fallback kill of the reported Python PID when
  it differs from the started PID, so it can't signal a reused PID.
The watcher stays in the caller's process group, so a terminal's Ctrl-C
(SIGINT), a hangup, or a supervisor signaling the whole group killed it along
with the program and leaked the extraction directory. Bash's implicit SIGINT
ignore for background jobs doesn't extend to the commands the watcher runs
(`sleep`, command substitutions), and when those died the watcher exited too.

Ignore INT, QUIT, HUP and TERM in the watcher with `trap ''`, which its
commands inherit. It still exits on its own once the process is gone.

* Adds a test that signals the whole process group with SIGINT, SIGHUP and
  SIGTERM and checks the extraction directory is removed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants