Skip to content

feat(exec-runner): Agent and Wrap sessions - #414

Open
raphaelvigee wants to merge 4 commits into
raphaelvigee/exec-runners-devenvfrom
raphaelvigee/exec-runners-phase2b
Open

feat(exec-runner): Agent and Wrap sessions#414
raphaelvigee wants to merge 4 commits into
raphaelvigee/exec-runners-devenvfrom
raphaelvigee/exec-runners-phase2b

Conversation

@raphaelvigee

@raphaelvigee raphaelvigee commented Aug 22, 2026

Copy link
Copy Markdown
Member

Phase 2b of #404, on top of #413the last of the stack's core work. mode = "session" holds a devenv shell open and forks every target's process from inside it; WrapSession covers container/chroot runners.

Proved end to end

Against this repo's own devenv, the distinction the second mode exists for:

shell function under mode = "snapshot" NOT available
shell function under mode = "session" available

Agent mode is a spec transformation, not an overridden spawn

The first sketch had ExecSession::spawn overridden for Agent. It cannot be: spawn returns a proc_exec::Handle, which can only be built for a child of this process — and the whole point of an agent is that the child is forked by something else. Returning a trait object instead would undo the reason prepare is the seam at all.

So Agent is also a pure spec transformation. The spec is rewritten to run heph __runner-exec, which heph spawns as its own ordinary child:

  heph ──spawn──> client ──unix socket──> agent (inside `devenv shell`)
                    │        SCM_RIGHTS         │
                    └── its own 0/1/2 ──────────┴──fork/exec──> the target

Handle, the bounded drain, the PTY and the supervisor keep working unchanged, and the child's stdio are the same descriptors — passed, not proxied. That is the property the fd-passing protocol exists to protect: a proxy would insert a copy loop between the target and the terminal, breaking PTY semantics and interactive output. Cancellation falls out: heph kills the client's process group, the agent sees the socket close.

Two bugs the first end-to-end run found

The agent must env_clear. It lives inside the devenv shell, so inheriting its environment would put the developer's ambient GOFLAGS/RUSTFLAGS/PATH tail into every build — unhashed, under a key reporting as lockfile-pinned. That hole was missed when session mode was first analysed.

Shell functions must be export -f'd. Otherwise they're defined in a shell the target never runs in: pluginexec invokes every target through its own bash -c, and functions aren't inherited by a child shell unless exported. The first run reported the function still missing.

SNAPSHOT_FORMAT_VERSION → 2 for the prelude change, and it earned its keep immediately: without the bump the fix was invisible behind a cached v1 snapshot.

WrapSession

Prepends a wrapper command. WrapEnv distinguishes wrappers that exec the inner program (chroot, bwrap, nix develop --command — the spec's env carries through) from those that create it elsewhere (docker exec — the environment must be rendered into the wrapper's own argv, or the container process sees none of it). The second kind reports no enumerable environment rather than handing back the host-side map as if it described what the inner process will see.

Teardown is now wired — it wasn't

Nothing called teardown(). ExecSessionPool now runs every session's teardown explicitly, and idempotently: an orderly-only teardown leaks exactly when things aren't orderly, which is the docker run -d container and the devenv shell surviving Ctrl-C. Regression-tested (runs once, and not while the session is still in use).

Tests

20 in exec-runner (SCM_RIGHTS round-trip proving descriptors arrive open on the far side, stale-vs-live socket handling, Wrap argv/env rendering, Agent spec rewrite, teardown handed over at most once), 9 in plugin-devenv, 4 in runner_agent (the -- boundary, so a target's own --socket can't retarget the client), 12 e2e. lint clean — the fd-passing code is one unsafe operation per block, each with its own safety condition.

Stack created with GitHub Stacks CLIGive Feedback 💬

Phase 2b of docs/EXEC_RUNNERS.md, and the last of it. `mode = "session"` holds
a `devenv shell` open and forks every target's process from inside it, and
`WrapSession` covers container/chroot runners.

Proved end to end against this repo's own devenv: a bash function defined by
the environment is NOT available under `mode = "snapshot"` and IS under
`mode = "session"`, which is the entire reason the second mode exists.

**A correction to §4.6.** The design had `ExecSession::spawn` *overridden* for
`Agent`. It cannot be: `spawn` returns a `proc_exec::Handle`, which can only be
built for a child of this process, and the whole point of an agent is that the
child is forked by something else. Returning a trait object instead would undo
the reason `prepare` is the seam at all — `proc_exec` would lose its
synchronous spawn, its "the spawn is the API" invariant and its OS-divergent
reader discipline behind a `dyn`.

So `Agent` is *also* a pure spec transformation. The spec is rewritten to run
`heph __runner-exec`, which heph spawns as its own ordinary child; the client
hands the agent its own stdio descriptors over SCM_RIGHTS and exits with the
child's status:

    heph ──spawn──> client ──unix socket──> agent (inside `devenv shell`)
                      │        SCM_RIGHTS         │
                      └── its own 0/1/2 ──────────┴──fork/exec──> the target

`Handle`, the bounded drain, the PTY and the supervisor keep working unchanged,
and the child's stdio are the *same* descriptors — passed, not proxied, which
is what §4.6 exists to protect. Cancellation falls out: heph kills the client's
process group and the agent sees the socket close.

The agent applies the composed environment with `env_clear` rather than letting
its own through. It lives inside the devenv shell, so inheriting would put the
developer's ambient GOFLAGS/RUSTFLAGS/PATH tail into every build — unhashed,
under a key that reports as lockfile-pinned. That is the hole the design's
first M2 analysis missed, and this is the fix.

Shell functions are `export -f`'d into the prelude. Without that they are
defined in a shell the target never runs in: `pluginexec` invokes every target
through its own `bash -c`, and a function is not inherited by a child shell
unless exported. Found by the first end-to-end run, which reported the
function as still missing.

`SNAPSHOT_FORMAT_VERSION` bumped to 2 for the prelude change — and it earned
its keep immediately: without the bump the fix was invisible behind a cached
v1 snapshot.

`WrapSession` prepends a wrapper command. Its `WrapEnv` distinguishes wrappers
that exec the inner program (chroot, bwrap, `nix develop --command` — the spec's
env carries through) from those that create it elsewhere (`docker exec` — the
environment must be rendered into the wrapper's own argv, or the container
process sees none of it). A `Wrap` runner of the second kind reports no
enumerable environment rather than handing back the host-side map as if it
described what the inner process will see.

**Teardown is now wired**, which it was not: nothing called it. `ExecSessionPool`
runs every session's teardown on drop, synchronously — an orderly-only teardown
leaks exactly when things are not orderly, which is the `docker run -d`
container and the devenv shell surviving Ctrl-C. Regression-tested.

The fd-passing code is one unsafe operation per block with its own safety
condition, rather than one block around the whole message-building sequence.
Caught by CI on linux/arm64, where the teardown test failed with a count of
zero; it passed on darwin. The difference is not the platform — it is that
`Drop` runs only if the last `Arc<Engine>` is actually released, and something
still held one. Silence is exactly the failure mode teardown must not have.

Worse, heph's second-Ctrl-C path calls `std::process::exit`, which runs no
destructors at all — the file's own comment says so, which is why the terminal
restore already lives there rather than in a `Drop`. So a `Drop`-only teardown
was guaranteed to be skipped at the moment a leaked `docker run -d` container
or `devenv shell` matters most.

`ExecSessionPool::teardown_all` is now idempotent and public through
`Engine::shutdown_exec_sessions`, called from `hard_abort` alongside the
terminal restore and for the same reason. `Drop` still calls it, as a backstop
for the ordinary path rather than as the mechanism.

The test now exercises the explicit path — the one production takes and the
only one guaranteed to run — and additionally asserts that a second call does
nothing, so a `Drop` backstop after an explicit shutdown cannot run a
container's `docker rm` twice.
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/exec-runners-phase2b branch from 132b3dc to 68bcdcf Compare August 23, 2026 17:55
raphaelvigee and others added 2 commits August 24, 2026 00:37
…; add devenv `mode = "wrap"`

Two things, both landing here because this is the commit that introduces
`AgentSession` and `WrapSession`.

## The agent had no cancellation path at all

This module's own docs claimed one: "heph kills the client's process group, the
agent sees the socket close, and kills the child it forked." Neither half
existed. `serve_one` blocked in `child.wait()` with nothing watching, and the
target could not be reached by any kill heph issues — heph forked the *client*,
while the target was forked by the agent and then `setsid` into a session of its
own. The session teardown's `SIGTERM` to the `devenv shell` misses it for the
same reason.

So a cancelled build left every in-flight target running to completion. The new
test measures exactly that: with the watch neutered it fails after the target
outlives its client by 30.03 s.

`serve_one` now hands `run` a descriptor to poll. The client sends nothing after
its request, so anything readable means it is gone; the agent then `SIGKILL`s
the process group — which works precisely because `setsid` made the target a
group leader. `HangupWatch` lives in `hexec_runner::agent` rather than the
binary so it can be tested without a full session.

## `mode = "wrap"`

The first real constructor of `WrapSession`, and deliberately framed as a
demonstration of that lane rather than a recommendation. Measured before
writing it:

- `devenv shell -- true` is **~4.5 s warm** on this repo. Snapshot mode pays
  `devenv print-dev-env --json` once, as a *cached target*; wrap pays a full
  shell entry per spawn, and a Go build is thousands of processes.
- `enterShell` runs once per target instead of once per build.
- It does **not** buy shell functions — `devenv shell -- bash -c 'declare -F'`
  reports none, so the one thing `session` mode exists for is still missing.

Hence `Identity::Asserted`, and a doc comment that says all of this at the
constructor. It is worth having: it exercises `WrapEnv::Inherit`, and it is the
right shape for a runner a handful of targets name.

The mode now travels in the snapshot artifact (`SNAPSHOT_FORMAT_VERSION` 2 → 3)
because `open` receives artifacts and no def. That also removes an inference
that could not have expressed this: the runner used to decide the mode from
whether `shell_prelude` was empty, which cannot tell `snapshot` from `wrap`.
The artifact carries `bin` for the same reason — and session mode now honours a
configured `bin`, which it silently ignored before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RC5ykWRMPjRKGk1Tz1KTja
…ds are for

The Modes section said "the plugin never spawns, holds a process, or touches a
descriptor". Session mode has always spawned a `devenv shell`, so that was
wrong; the table now says who forks the target in each mode, which is the only
axis they actually differ on.

Adds a plain-language account of `__runner-agent` and `__runner-exec`, motivated
rather than described. The agent is the piece that can create a process inside a
held-open shell; the client exists because heph then has a target it did not
fork, and a `Handle` — all of heph's output streaming and cancellation — can
only be built for its own child. The client restores that illusion for one small
process per target, and hands over the real descriptors so the bytes never
travel through it.

Also documents `mode = "wrap"` with the numbers that make it a demonstration
rather than a recommendation, and the socket watch that stops a cancelled build
from leaving its targets running.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RC5ykWRMPjRKGk1Tz1KTja
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant