docs(exec-runners): what an exec runner is, and where the cache key is drawn - #404
Open
raphaelvigee wants to merge 1 commit into
Open
docs(exec-runners): what an exec runner is, and where the cache key is drawn#404raphaelvigee wants to merge 1 commit into
raphaelvigee wants to merge 1 commit into
Conversation
This was referenced Aug 22, 2026
raphaelvigee
marked this pull request as ready for review
August 23, 2026 11:19
raphaelvigee
force-pushed
the
raphaelvigee/exec-runners-design
branch
from
August 23, 2026 11:29
96c88cf to
d33f464
Compare
raphaelvigee
added a commit
that referenced
this pull request
Aug 23, 2026
…e dangling doc refs `crates/plugin-sdk` covers the lane at the seam — a driver's `open_session` / `prepare_spec` / `close_session` crossing a real stabby vtable. What that cannot show is the half above it: that the engine resolves a runner target to its driver, opens one session for the environment, and routes every target's actual process through it. `crates/e2e/tests/exec_runner_spec.rs` runs a real BUILD graph against a `DriverExecRunner` over a `ManagedDriver`, which is exactly what a loaded cdylib becomes host-side: - a driver-served session reaches the target's real process - the runner is consulted **per target** while the environment opens **once** — the property that justifies the ABI lane over describing the environment as data - a driver that serves no sessions is refused, not degraded to the host environment under a key asserting the runner's - the runner's identity still reaches the cache key Writing these surfaced a semantic worth stating: under this lane the host does **not** merge `base_env` for anyone. The runner owns the whole transformation, because the runner is what starts the process; `ExecSession::base_env` is what the host *reports* for diagnostics, not what it applies. The first draft of the fixture assumed otherwise and failed with `FROM_RUNNER: unbound variable`. Also removes the `§N.N` references to the design doc's numbered sections, which have pointed at nothing since #404 made it a short reference. Each now names a real heading ("Environment layering", "How it reaches the cache key") or drops the section. `GHA_REPORTING.md` and `CONCURRENCY_MEASUREMENTS.md` refs are untouched — those docs still have those sections. e2e 147 across 18 files, plus plugin-sdk 36 and the crate suites. lint clean.
raphaelvigee
added a commit
that referenced
this pull request
Aug 23, 2026
…op the dangling doc refs
`crates/plugin-sdk` covers the component at the seam — a runner's `open_session`
/ `prepare_spec` / `close_session` crossing a real stabby vtable. What that
cannot show is the half above it: that the engine resolves a runner target to
its registered runner, opens one session for the environment, and routes every
target's actual process through it.
`crates/e2e/tests/exec_runner_spec.rs` runs a real BUILD graph against a
`PluginExecRunner` over an `ExecRunnerPlugin` — exactly what a loaded cdylib
becomes host-side:
- a plugin-served session reaches the target's real process
- the runner is consulted **per target** while the environment opens **once** —
the property that justifies a runner owning process creation
- a runner target with no registered runner is refused, not degraded to the host
environment under a key asserting the runner's
- the runner's identity still reaches the cache key
The fixture is worth a look for what it does *not* contain: no `parse`, no
`run`, no schema. A runner is its own component kind, so the mock implements
three methods. Under the earlier draft — the lane bolted onto `ManagedDriver` —
the same mock needed three `bail!("unused")` stubs.
Writing these surfaced a semantic worth stating: the host does **not** merge
`base_env` for anyone. The runner owns the whole transformation, because the
runner is what starts the process; `ExecSession::base_env` is what the host
*reports* for diagnostics, not what it applies. The first draft of the fixture
assumed otherwise and failed with `FROM_RUNNER: unbound variable`.
Also removes the `§N.N` references to the design doc's numbered sections, which
have pointed at nothing since #404 made it a short reference. Each now names a
real heading ("Environment layering", "How it reaches the cache key") or drops
the section. `GHA_REPORTING.md` and `CONCURRENCY_MEASUREMENTS.md` refs are
untouched — those docs still have those sections.
e2e 147 across 18 files, plus plugin-sdk 36 and the crate suites. lint clean.
…s drawn `docs/EXEC_RUNNERS.md`: a page on exec runners — the model, the authoring surface, how a runner reaches the cache key, the environment layering, the three session modes, the devenv and Go knobs, and what `heph inspect` will tell you. Reference rather than a design record: the rules and the reason each exists, without the phasing, the review history or the corrections log that only mattered while it was being built. That reasoning lives in the commits and pull requests that made each change, which is where it is useful when someone runs `git blame` on the line that surprised them.
raphaelvigee
added a commit
that referenced
this pull request
Aug 23, 2026
…op the dangling doc refs
`crates/plugin-sdk` covers the component at the seam — a runner's `open_session`
/ `prepare_spec` / `close_session` crossing a real stabby vtable. What that
cannot show is the half above it: that the engine resolves a runner target to
its registered runner, opens one session for the environment, and routes every
target's actual process through it.
`crates/e2e/tests/exec_runner_spec.rs` runs a real BUILD graph against a
`PluginExecRunner` over an `ExecRunnerPlugin` — exactly what a loaded cdylib
becomes host-side:
- a plugin-served session reaches the target's real process
- the runner is consulted **per target** while the environment opens **once** —
the property that justifies a runner owning process creation
- a runner target with no registered runner is refused, not degraded to the host
environment under a key asserting the runner's
- the runner's identity still reaches the cache key
The fixture is worth a look for what it does *not* contain: no `parse`, no
`run`, no schema. A runner is its own component kind, so the mock implements
three methods. Under the earlier draft — the lane bolted onto `ManagedDriver` —
the same mock needed three `bail!("unused")` stubs.
Writing these surfaced a semantic worth stating: the host does **not** merge
`base_env` for anyone. The runner owns the whole transformation, because the
runner is what starts the process; `ExecSession::base_env` is what the host
*reports* for diagnostics, not what it applies. The first draft of the fixture
assumed otherwise and failed with `FROM_RUNNER: unbound variable`.
Also removes the `§N.N` references to the design doc's numbered sections, which
have pointed at nothing since #404 made it a short reference. Each now names a
real heading ("Environment layering", "How it reaches the cache key") or drops
the section. `GHA_REPORTING.md` and `CONCURRENCY_MEASUREMENTS.md` refs are
untouched — those docs still have those sections.
e2e 147 across 18 files, plus plugin-sdk 36 and the crate suites. lint clean.
raphaelvigee
added a commit
that referenced
this pull request
Aug 23, 2026
…op the dangling doc refs
`crates/plugin-sdk` covers the component at the seam — a runner's `open_session`
/ `prepare_spec` / `close_session` crossing a real stabby vtable. What that
cannot show is the half above it: that the engine resolves a runner target to
its registered runner, opens one session for the environment, and routes every
target's actual process through it.
`crates/e2e/tests/exec_runner_spec.rs` runs a real BUILD graph against a
`PluginExecRunner` over an `ExecRunnerPlugin` — exactly what a loaded cdylib
becomes host-side:
- a plugin-served session reaches the target's real process
- the runner is consulted **per target** while the environment opens **once** —
the property that justifies a runner owning process creation
- a runner target with no registered runner is refused, not degraded to the host
environment under a key asserting the runner's
- the runner's identity still reaches the cache key
The fixture is worth a look for what it does *not* contain: no `parse`, no
`run`, no schema. A runner is its own component kind, so the mock implements
three methods. Under the earlier draft — the lane bolted onto `ManagedDriver` —
the same mock needed three `bail!("unused")` stubs.
Writing these surfaced a semantic worth stating: the host does **not** merge
`base_env` for anyone. The runner owns the whole transformation, because the
runner is what starts the process; `ExecSession::base_env` is what the host
*reports* for diagnostics, not what it applies. The first draft of the fixture
assumed otherwise and failed with `FROM_RUNNER: unbound variable`.
Also removes the `§N.N` references to the design doc's numbered sections, which
have pointed at nothing since #404 made it a short reference. Each now names a
real heading ("Environment layering", "How it reaches the cache key") or drops
the section. `GHA_REPORTING.md` and `CONCURRENCY_MEASUREMENTS.md` refs are
untouched — those docs still have those sections.
e2e 147 across 18 files, plus plugin-sdk 36 and the crate suites. lint clean.
raphaelvigee
added a commit
that referenced
this pull request
Aug 23, 2026
…op the dangling doc refs
`crates/plugin-sdk` covers the component at the seam — a runner's `open_session`
/ `prepare_spec` / `close_session` crossing a real stabby vtable. What that
cannot show is the half above it: that the engine resolves a runner target to
its registered runner, opens one session for the environment, and routes every
target's actual process through it.
`crates/e2e/tests/exec_runner_spec.rs` runs a real BUILD graph against a
`PluginExecRunner` over an `ExecRunnerPlugin` — exactly what a loaded cdylib
becomes host-side:
- a plugin-served session reaches the target's real process
- the runner is consulted **per target** while the environment opens **once** —
the property that justifies a runner owning process creation
- a runner target with no registered runner is refused, not degraded to the host
environment under a key asserting the runner's
- the runner's identity still reaches the cache key
The fixture is worth a look for what it does *not* contain: no `parse`, no
`run`, no schema. A runner is its own component kind, so the mock implements
three methods. Under the earlier draft — the lane bolted onto `ManagedDriver` —
the same mock needed three `bail!("unused")` stubs.
Writing these surfaced a semantic worth stating: the host does **not** merge
`base_env` for anyone. The runner owns the whole transformation, because the
runner is what starts the process; `ExecSession::base_env` is what the host
*reports* for diagnostics, not what it applies. The first draft of the fixture
assumed otherwise and failed with `FROM_RUNNER: unbound variable`.
Also removes the `§N.N` references to the design doc's numbered sections, which
have pointed at nothing since #404 made it a short reference. Each now names a
real heading ("Environment layering", "How it reaches the cache key") or drops
the section. `GHA_REPORTING.md` and `CONCURRENCY_MEASUREMENTS.md` refs are
untouched — those docs still have those sections.
e2e 147 across 18 files, plus plugin-sdk 36 and the crate suites. lint clean.
raphaelvigee
force-pushed
the
raphaelvigee/exec-runners-design
branch
from
August 23, 2026 23:25
d33f464 to
38183ea
Compare
raphaelvigee
added a commit
that referenced
this pull request
Aug 24, 2026
…op the dangling doc refs
`crates/plugin-sdk` covers the component at the seam — a runner's `open_session`
/ `prepare_spec` / `close_session` crossing a real stabby vtable. What that
cannot show is the half above it: that the engine resolves a runner target to
its registered runner, opens one session for the environment, and routes every
target's actual process through it.
`crates/e2e/tests/exec_runner_spec.rs` runs a real BUILD graph against a
`PluginExecRunner` over an `ExecRunnerPlugin` — exactly what a loaded cdylib
becomes host-side:
- a plugin-served session reaches the target's real process
- the runner is consulted **per target** while the environment opens **once** —
the property that justifies a runner owning process creation
- a runner target with no registered runner is refused, not degraded to the host
environment under a key asserting the runner's
- the runner's identity still reaches the cache key
The fixture is worth a look for what it does *not* contain: no `parse`, no
`run`, no schema. A runner is its own component kind, so the mock implements
three methods. Under the earlier draft — the lane bolted onto `ManagedDriver` —
the same mock needed three `bail!("unused")` stubs.
Writing these surfaced a semantic worth stating: the host does **not** merge
`base_env` for anyone. The runner owns the whole transformation, because the
runner is what starts the process; `ExecSession::base_env` is what the host
*reports* for diagnostics, not what it applies. The first draft of the fixture
assumed otherwise and failed with `FROM_RUNNER: unbound variable`.
Also removes the `§N.N` references to the design doc's numbered sections, which
have pointed at nothing since #404 made it a short reference. Each now names a
real heading ("Environment layering", "How it reaches the cache key") or drops
the section. `GHA_REPORTING.md` and `CONCURRENCY_MEASUREMENTS.md` refs are
untouched — those docs still have those sections.
e2e 147 across 18 files, plus plugin-sdk 36 and the crate suites. lint clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Doc only — no code. Phase 0 is #405 on top of this.
What
docs/EXEC_RUNNERS.md: a short reference forExecRunner/ExecSession, a driver-independent answer to "in what environment is this target's process created?", with devenv.sh as the first non-trivial implementation (shipped as a cdylib plugin).The ask was: let a target run inside the devenv shell and inherit its tools; make it general rather than a
pluginexecfeature; spawn the shell once and have many targets run in it.It covers the model and the surface, how a runner reaches the cache key, the environment layering rules, the three session modes, devenv, Go, and diagnosability. It documents what the stack ships — the phasing lives in the PRs.
The shape
A runner reference is a target reference. Its identity then reaches the cache key through the existing dependency mechanism instead of a new hash component — and ordering, failure attribution and cycle checking all come along for free. In particular the runner target is built during
hashin, before any worker permit is held, so "opening a session may require building a target" cannot deadlock against the execute semaphore.Selection is a workspace-level
defaultRunner:plus a per-targetrunner =.Reviewed by the standing board
product-visionSHIP WITH CHANGES ·compatibilityCOMPATIBLE 6/7 ·feature-qualityBLOCKED ·hermeticityNOT HERMETIC ·code-qualityBLOCKED.The model survived; the execution details did not. The findings that most altered it:
prepare(spec) -> Specis the seam, not a trait-objectified process factory.DirectandWrapare pure spec transformations, soproc_execkeeps its synchronous spawn, itsHandleinvariants and its PTY handling untouched.proc_exec::output, which cannot be built onspawn— its drain is unbounded because nothing consumes until the wait returns. Ago listover 512 KiB routed throughspawn+ wait wedges on darwin and passes on linux./nix/store, andDEVENV_ROOT/$HOME/DEVENV_CMDLINEwould all have entered the snapshot — under-hashing mutable host tools while over-hashing the checkout path and the literal command line used to enter the shell. (feat(plugin-devenv): capture a devenv environment and run targets in it #413 corrects the source of that measurement; the rule it produced stayed.)runtime_pass_env = ["*"]sat above the runner's PATH, so a target could build with host tools under a key asserting devenv, then push that to the shared cache. Now a hard error.hashin— two different environments would share a cache key. Now rejected at resolution.runnerintransitivewas removed:transitiveis dependency-directional, so as specified, adding a dep would silently change a target's execution environment, with an iteration-order winner when two deps disagreed.Decisions recorded as the user's
tools =deps.remote_enabled = false, matching theplugin-nixprecedent ("wrappers point at host-local /nix/store").Left open and flagged rather than assumed: manifest provenance, which is a cache-format change needing its own
compatibilityconsult. (WhetherenterShell/ shell functions / services are load-bearing in yourdevenv.nixwas settled by shipping both modes — see #414.)Stack created with GitHub Stacks CLI • Give Feedback 💬