feat(exec-runner): Agent and Wrap sessions - #414
Open
raphaelvigee wants to merge 4 commits into
Open
Conversation
raphaelvigee
marked this pull request as ready for review
August 23, 2026 11:19
raphaelvigee
force-pushed
the
raphaelvigee/exec-runners-phase2b
branch
from
August 23, 2026 11:29
7559beb to
132b3dc
Compare
Phase 2b of docs/EXEC_RUNNERS.md, and the last of it. `mode = "session"` holds
a `devenv shell` open and forks every target's process from inside it, and
`WrapSession` covers container/chroot runners.
Proved end to end against this repo's own devenv: a bash function defined by
the environment is NOT available under `mode = "snapshot"` and IS under
`mode = "session"`, which is the entire reason the second mode exists.
**A correction to §4.6.** The design had `ExecSession::spawn` *overridden* for
`Agent`. It cannot be: `spawn` returns a `proc_exec::Handle`, which can only be
built for a child of this process, and the whole point of an agent is that the
child is forked by something else. Returning a trait object instead would undo
the reason `prepare` is the seam at all — `proc_exec` would lose its
synchronous spawn, its "the spawn is the API" invariant and its OS-divergent
reader discipline behind a `dyn`.
So `Agent` is *also* a pure spec transformation. The spec is rewritten to run
`heph __runner-exec`, which heph spawns as its own ordinary child; the client
hands the agent its own stdio descriptors over SCM_RIGHTS and exits with the
child's status:
heph ──spawn──> client ──unix socket──> agent (inside `devenv shell`)
│ SCM_RIGHTS │
└── its own 0/1/2 ──────────┴──fork/exec──> the target
`Handle`, the bounded drain, the PTY and the supervisor keep working unchanged,
and the child's stdio are the *same* descriptors — passed, not proxied, which
is what §4.6 exists to protect. Cancellation falls out: heph kills the client's
process group and the agent sees the socket close.
The agent applies the composed environment with `env_clear` rather than letting
its own through. It lives inside the devenv shell, so inheriting would put the
developer's ambient GOFLAGS/RUSTFLAGS/PATH tail into every build — unhashed,
under a key that reports as lockfile-pinned. That is the hole the design's
first M2 analysis missed, and this is the fix.
Shell functions are `export -f`'d into the prelude. Without that they are
defined in a shell the target never runs in: `pluginexec` invokes every target
through its own `bash -c`, and a function is not inherited by a child shell
unless exported. Found by the first end-to-end run, which reported the
function as still missing.
`SNAPSHOT_FORMAT_VERSION` bumped to 2 for the prelude change — and it earned
its keep immediately: without the bump the fix was invisible behind a cached
v1 snapshot.
`WrapSession` prepends a wrapper command. Its `WrapEnv` distinguishes wrappers
that exec the inner program (chroot, bwrap, `nix develop --command` — the spec's
env carries through) from those that create it elsewhere (`docker exec` — the
environment must be rendered into the wrapper's own argv, or the container
process sees none of it). A `Wrap` runner of the second kind reports no
enumerable environment rather than handing back the host-side map as if it
described what the inner process will see.
**Teardown is now wired**, which it was not: nothing called it. `ExecSessionPool`
runs every session's teardown on drop, synchronously — an orderly-only teardown
leaks exactly when things are not orderly, which is the `docker run -d`
container and the devenv shell surviving Ctrl-C. Regression-tested.
The fd-passing code is one unsafe operation per block with its own safety
condition, rather than one block around the whole message-building sequence.
Caught by CI on linux/arm64, where the teardown test failed with a count of zero; it passed on darwin. The difference is not the platform — it is that `Drop` runs only if the last `Arc<Engine>` is actually released, and something still held one. Silence is exactly the failure mode teardown must not have. Worse, heph's second-Ctrl-C path calls `std::process::exit`, which runs no destructors at all — the file's own comment says so, which is why the terminal restore already lives there rather than in a `Drop`. So a `Drop`-only teardown was guaranteed to be skipped at the moment a leaked `docker run -d` container or `devenv shell` matters most. `ExecSessionPool::teardown_all` is now idempotent and public through `Engine::shutdown_exec_sessions`, called from `hard_abort` alongside the terminal restore and for the same reason. `Drop` still calls it, as a backstop for the ordinary path rather than as the mechanism. The test now exercises the explicit path — the one production takes and the only one guaranteed to run — and additionally asserts that a second call does nothing, so a `Drop` backstop after an explicit shutdown cannot run a container's `docker rm` twice.
raphaelvigee
force-pushed
the
raphaelvigee/exec-runners-phase2b
branch
from
August 23, 2026 17:55
132b3dc to
68bcdcf
Compare
…; add devenv `mode = "wrap"` Two things, both landing here because this is the commit that introduces `AgentSession` and `WrapSession`. ## The agent had no cancellation path at all This module's own docs claimed one: "heph kills the client's process group, the agent sees the socket close, and kills the child it forked." Neither half existed. `serve_one` blocked in `child.wait()` with nothing watching, and the target could not be reached by any kill heph issues — heph forked the *client*, while the target was forked by the agent and then `setsid` into a session of its own. The session teardown's `SIGTERM` to the `devenv shell` misses it for the same reason. So a cancelled build left every in-flight target running to completion. The new test measures exactly that: with the watch neutered it fails after the target outlives its client by 30.03 s. `serve_one` now hands `run` a descriptor to poll. The client sends nothing after its request, so anything readable means it is gone; the agent then `SIGKILL`s the process group — which works precisely because `setsid` made the target a group leader. `HangupWatch` lives in `hexec_runner::agent` rather than the binary so it can be tested without a full session. ## `mode = "wrap"` The first real constructor of `WrapSession`, and deliberately framed as a demonstration of that lane rather than a recommendation. Measured before writing it: - `devenv shell -- true` is **~4.5 s warm** on this repo. Snapshot mode pays `devenv print-dev-env --json` once, as a *cached target*; wrap pays a full shell entry per spawn, and a Go build is thousands of processes. - `enterShell` runs once per target instead of once per build. - It does **not** buy shell functions — `devenv shell -- bash -c 'declare -F'` reports none, so the one thing `session` mode exists for is still missing. Hence `Identity::Asserted`, and a doc comment that says all of this at the constructor. It is worth having: it exercises `WrapEnv::Inherit`, and it is the right shape for a runner a handful of targets name. The mode now travels in the snapshot artifact (`SNAPSHOT_FORMAT_VERSION` 2 → 3) because `open` receives artifacts and no def. That also removes an inference that could not have expressed this: the runner used to decide the mode from whether `shell_prelude` was empty, which cannot tell `snapshot` from `wrap`. The artifact carries `bin` for the same reason — and session mode now honours a configured `bin`, which it silently ignored before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RC5ykWRMPjRKGk1Tz1KTja
…ds are for The Modes section said "the plugin never spawns, holds a process, or touches a descriptor". Session mode has always spawned a `devenv shell`, so that was wrong; the table now says who forks the target in each mode, which is the only axis they actually differ on. Adds a plain-language account of `__runner-agent` and `__runner-exec`, motivated rather than described. The agent is the piece that can create a process inside a held-open shell; the client exists because heph then has a target it did not fork, and a `Handle` — all of heph's output streaming and cancellation — can only be built for its own child. The client restores that illusion for one small process per target, and hands over the real descriptors so the bytes never travel through it. Also documents `mode = "wrap"` with the numbers that make it a demonstration rather than a recommendation, and the socket watch that stops a cancelled build from leaving its targets running. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RC5ykWRMPjRKGk1Tz1KTja
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 2b of #404, on top of #413 — the last of the stack's core work.
mode = "session"holds adevenv shellopen and forks every target's process from inside it;WrapSessioncovers container/chroot runners.Proved end to end
Against this repo's own devenv, the distinction the second mode exists for:
mode = "snapshot"mode = "session"Agent mode is a spec transformation, not an overridden spawn
The first sketch had
ExecSession::spawnoverridden forAgent. It cannot be:spawnreturns aproc_exec::Handle, which can only be built for a child of this process — and the whole point of an agent is that the child is forked by something else. Returning a trait object instead would undo the reasonprepareis the seam at all.So
Agentis also a pure spec transformation. The spec is rewritten to runheph __runner-exec, which heph spawns as its own ordinary child:Handle, the bounded drain, the PTY and the supervisor keep working unchanged, and the child's stdio are the same descriptors — passed, not proxied. That is the property the fd-passing protocol exists to protect: a proxy would insert a copy loop between the target and the terminal, breaking PTY semantics and interactive output. Cancellation falls out: heph kills the client's process group, the agent sees the socket close.Two bugs the first end-to-end run found
The agent must
env_clear. It lives inside the devenv shell, so inheriting its environment would put the developer's ambientGOFLAGS/RUSTFLAGS/PATH tail into every build — unhashed, under a key reporting as lockfile-pinned. That hole was missed when session mode was first analysed.Shell functions must be
export -f'd. Otherwise they're defined in a shell the target never runs in:pluginexecinvokes every target through its ownbash -c, and functions aren't inherited by a child shell unless exported. The first run reported the function still missing.SNAPSHOT_FORMAT_VERSION→ 2 for the prelude change, and it earned its keep immediately: without the bump the fix was invisible behind a cached v1 snapshot.WrapSessionPrepends a wrapper command.
WrapEnvdistinguishes wrappers that exec the inner program (chroot,bwrap,nix develop --command— the spec's env carries through) from those that create it elsewhere (docker exec— the environment must be rendered into the wrapper's own argv, or the container process sees none of it). The second kind reports no enumerable environment rather than handing back the host-side map as if it described what the inner process will see.Teardown is now wired — it wasn't
Nothing called
teardown().ExecSessionPoolnow runs every session's teardown explicitly, and idempotently: an orderly-only teardown leaks exactly when things aren't orderly, which is thedocker run -dcontainer and the devenv shell surviving Ctrl-C. Regression-tested (runs once, and not while the session is still in use).Tests
20 in
exec-runner(SCM_RIGHTS round-trip proving descriptors arrive open on the far side, stale-vs-live socket handling,Wrapargv/env rendering,Agentspec rewrite, teardown handed over at most once), 9 inplugin-devenv, 4 inrunner_agent(the--boundary, so a target's own--socketcan't retarget the client), 12 e2e.lintclean — the fd-passing code is one unsafe operation per block, each with its own safety condition.Stack created with GitHub Stacks CLI • Give Feedback 💬