feat(exec-runner): runner-declared env, Go knobs, and examples - #416
Open
raphaelvigee wants to merge 3 commits into
Open
feat(exec-runner): runner-declared env, Go knobs, and examples#416raphaelvigee wants to merge 3 commits into
raphaelvigee wants to merge 3 commits into
Conversation
raphaelvigee
marked this pull request as ready for review
August 23, 2026 11:19
raphaelvigee
force-pushed
the
raphaelvigee/exec-runners-env
branch
from
August 23, 2026 11:29
ad5d85d to
9fd61f7
Compare
raphaelvigee
force-pushed
the
raphaelvigee/exec-runners-env
branch
from
August 23, 2026 17:55
9fd61f7 to
cf67110
Compare
…me pair A runner can now shape its environment with the same four keys a target has, rather than only whatever it captured. The split between them is where the cache key is drawn, and it is the whole reason there are four rather than two: env literal, captured -> in the key pass_env host VALUE captured -> in the key; changing it re-keys runtime_env literal, at spawn -> the declaration is in the key runtime_pass_env host value at SPAWN -> only the NAME is ever hashed The last row is the one that matters. `SSH_AUTH_SOCK` and `DOCKER_HOST` differ per machine and per login; passing them with `pass_env` would bake one machine's value into the environment's identity and hand every other machine a cache key that lies about what produced the artifact. `runtime_pass_env` reads them at spawn, so they reach the process and never the key. `"*"` is the same escape hatch a target has. The declarations themselves are hashed either way, because they live in the runner target's artifact and that artifact *is* the environment's identity. That is the honest line: heph will not claim two differently-declared environments are the same one, and it will not put ambient host state in the key either. Everything a runner declares still sits UNDERNEATH what the target set itself — a runner that could overwrite `$OUT`, `$SRC` or a target's own `env` would silently change what the target builds. Asserted directly, including for the `"*"` wildcard. `SessionEnv` lives in the contract rather than in the devenv plugin, so every runner gets the capability; the devenv driver populates it from four new spec keys. `runtime_*` are resolved on every `prepare` rather than once at open, so a host variable that changes mid-build is seen. `SNAPSHOT_FORMAT_VERSION` -> 3. Examples (example/exec_runner/BUILD), covering the paths this stack added: selection and `runner = None`; a tool that resolves only under the runner and the same target failing without it; the four declared keys; a target overriding its runner; session mode providing a shell function that snapshot mode cannot, with the snapshot counterpart kept as an expected failure so the difference is visible rather than described. Also fixes the example workspace, which has not loaded since #397 removed the `sh` driver: `.hephconfig2` still listed `builtin: sh`, so every `heph` command run there failed at config load.
Two knobs, deliberately separate, because "how it was built" and "how it runs"
are different questions and conflating them is what makes a cache key lie.
**`goenv`** (provider option) is environment for the **Go toolchain** — `go
list` and the compiler/assembler. `GOPRIVATE`, `GOFLAGS`, a corporate `GOPROXY`
and its `GONOSUMDB` belong here.
It is injected into each target's spec, where it is **hashed**, rather than
applied behind the drivers' backs at run time. A knob that changed compiler
input without moving the cache key would hand every machine artifacts built
under settings the key does not record — the same failure this stack has been
closing everywhere else. `GO_COMPILE_FORMAT_VERSION` -> 5 and
`GO_GOLIST_FORMAT_VERSION` -> 18, with both frozen def-hash goldens updated;
they failed on the bump, which is what they are for.
Injected at one choke point (the provider's `get`) rather than in each of the
two dozen places a spec is built: a knob that reached most of them would be
worse than one that reached none, because the gap would surface as a cache hit
on an archive built under different settings.
An allowlist decides who gets it — `go_compile` and `go_golist`. `go_toolchain`
downloads an SDK and `go_testmain` writes a file, so handing them an
environment they never use would put it in their key for nothing and re-key
everything downstream of them. The lint/format drivers exec `heph-govet`, not
`go`; stretching `goenv` to cover a different binary would make the name a lie.
Within a driver, `goenv` is applied FIRST so the driver's own settings win:
`GOROOT`, `GOOS`/`GOARCH` and `GOWORK=off` are what make the build hermetic and
match the target's variant, and letting config override them would build
something other than the address asked for, under that address's key.
**`provider_state(provider="go", test={"runner": "//:devenv"})`** sets the exec
environment the *test binary* runs in. It joins the existing `test = {...}` map
next to `env`/`pass_env`/`pre_run` — the same shape, the same allowlist that
rejects typos — rather than inventing a second place to configure tests.
Only `test`/`xtest` take it. A test binary often wants a database client, a
browser or a service on PATH; the build wants the hermetic toolchain and
nothing else. One environment for both would either leak the runtime's tools
into every compile's cache key or force the build environment onto the test.
The value is parsed where the rest of the map is validated, so a bare name
fails there with the other state errors — and it is rejected rather than
guessed at, because only a target has a hashout and a hashout is what carries
the environment into the key. Relative addresses resolve against the package
that declared the state.
…vironment Caught by CI on both Linux legs, where the image sets `CC`: a runner declaring `CC=clang` alongside `runtime_pass_env = ["*"]` produced `CC=gcc`. The test was right and the ordering was wrong. I had mirrored a target's own sequence, where `runtime_pass_env` sits on top of `env`. For a runner that is backwards: a runner exists to *provide* an environment, so a passed-through host variable outranking the captured one means `"*"` silently replaces that environment with the developer's own — while the build stays keyed as though it ran in the runner's. That is the ambient leak this whole design is built to prevent, reintroduced by the one layer meant to be an escape hatch. Weakest to strongest is now: host passthrough < captured environment < declared literals < the target A runner that genuinely wants to override its own capture says so with `runtime_env`, which is explicit rather than whatever the host happened to have. The failing assertion is replaced by three that name the rule rather than stumble into it: the capture outranks a `"*"` passthrough, `runtime_env` can override the capture where the host cannot, and the target still wins over all of it. They use uniquely-named variables, so they reproduce the CI condition without depending on whether the machine sets `CC`. §4.4 records the asymmetry with a target's own ordering, since a reader who knows one would otherwise reasonably assume the other.
raphaelvigee
force-pushed
the
raphaelvigee/exec-runners-env
branch
from
August 23, 2026 22:46
cf67110 to
94da999
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On top of #414. Two commits, reviewable separately: a runner can now declare its own environment, and the Go plugin gets the two knobs that make runners usable there.
1. A runner declares
env/pass_env/runtime_env/runtime_pass_envThe same four keys a target has. The split between them is where the cache key is drawn:
envpass_envruntime_envruntime_pass_envThe last row is the one that matters.
SSH_AUTH_SOCKandDOCKER_HOSTdiffer per machine and per login; passing them withpass_envbakes one machine's value into the environment's identity and hands every other machine a cache key that lies about what produced the artifact.runtime_pass_envreads them at spawn, so they reach the process and never the key."*"is the same escape hatch a target has.The declarations themselves are hashed either way — they live in the runner target's artifact, and that artifact is the environment's identity. heph will not claim two differently-declared environments are the same one, and will not put ambient host state in the key either.
Everything a runner declares sits underneath what the target set itself, wildcard included — a runner that could overwrite
$OUT,$SRCor a target's ownenvwould silently change what the target builds. Asserted directly.SessionEnvlives in the contract, so every runner gets it; devenv populates it from four new spec keys.SNAPSHOT_FORMAT_VERSION→ 3.2. Go:
goenv, and a runner for test executionTwo knobs, deliberately separate, because how it was built and how it runs are different questions.
goenv(provider option) — environment for the Go toolchain:go list, the compiler, the assembler.GOPRIVATE,GOFLAGS, a corporateGOPROXY.Injected into each target's spec where it is hashed, not applied behind the drivers' backs. A knob that changed compiler input without moving the key would hand every machine artifacts built under settings the key does not record.
GO_COMPILE_FORMAT_VERSION→ 5,GO_GOLIST_FORMAT_VERSION→ 18; both frozen def-hash goldens failed on the bump, which is what they are for.One choke point (the provider's
get) rather than the two dozen places a spec is built — a knob that reached most of them would be worse than one that reached none, because the gap surfaces as a cache hit on an archive built under different settings.An allowlist decides who gets it:
go_compile,go_golist.go_toolchaindownloads an SDK andgo_testmainwrites a file — giving them an environment they never use would key them on it for nothing and re-key everything downstream. Lint/format execheph-govet, notgo.Within a driver
goenvis applied first, soGOROOT/GOOS/GOARCH/GOWORK=offwin — those make the build hermetic and match the variant, and letting config override them would build something other than the address asked for, under that address's key.provider_state(provider="go", test={"runner": "//:devenv"})— the exec environment the test binary runs in. Joins the existingtest = {...}map besideenv/pass_env/pre_run, same allowlist that rejects typos, rather than inventing a second place to configure tests.Only
test/xtesttake it. A test binary often wants a database client or a browser on PATH; the build wants the hermetic toolchain and nothing else. One environment for both would either leak the runtime's tools into every compile's key or force the build environment onto the test.Examples
example/exec_runner/BUILDexercises what this stack added: selection andrunner = None; a tool that resolves only under the runner and the same target failing without it; the four declared keys; a target overriding its runner; session mode providing a shell function snapshot mode cannot — with the snapshot counterpart kept as an expected failure, so the difference is visible rather than described.Also fixes the example workspace, which has not loaded since #397 removed the
shdriver:.hephconfig2still listedbuiltin: sh, so everyhephcommand run there failed at config load.Where a runner comes from, per target type
bash,exec,http_fetch,nix, …)runner = //pkg:name, orrunner = NonedefaultRunnerso it cannot become its own dependencytest/xtest(+_race)provider_state(test={"runner": …})go_compile/go_golistdefaultRunner; environment shaped bygoenvgo_toolchain/go_testmain/ lint / formatdefaultRunner; nogoenv@heph/fs,group,hostbin,oci_*)defaultRunnerrunner: Noneon aTargetSpecmeans unauthored, notlocal— which is why provider-emitted targets inheritdefaultRunner. That reachesgo_compile, andplugin-gois cdylib-only, so that path works only because of #411's positive ack.Known gap:
go_lint/go_formatinheritdefaultRunnerbut get nogoenv, since they execheph-govetrather thango. Ifheph-govetneeds a corporateGOPROXY, that wants its own knob.Tests
26 in
exec-runner(the four-key layering,"*", and the target-wins-over-everything case), 10 inplugin-devenv, 481 inplugin-go, plus e2e.lintclean.Stack created with GitHub Stacks CLI • Give Feedback 💬