test: run regenerateLibs into a fixture dir and pin what it emits - #228
Conversation
`Build`'s generator hooks were executed by nothing: a `revert()` as the first statement of either one left the whole suite green. Every check on them was output-anchored -- the committed libs compared against the emitters -- so a wrong argument or a short loop bound was invisible until somebody re-ran the generator and committed the result. `regenerateLibs` now takes its directory from an overridable `libDir()`, for the reason `BuildScript.recordRoot` is overridable, and a harness drives it into `fixture-lib/`. A regeneration of a clean checkout is a no-op, so the committed tree is the oracle: every file the hook writes must be byte-identical to the one already there, and it must write no others. `regenerateSnapshots` gets no such seam, because `LibFs` confines every snapshot write to `src/generated/<tag>/`, which is the record the suites running beside it walk. Closes #207 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V8ViHcKLVk2YoS2joH4HdN
|
Warning Review limit reachedNext included review available in 34 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V8ViHcKLVk2YoS2joH4HdN
…nd the two new tests Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V8ViHcKLVk2YoS2joH4HdN
What comes back is every sentence the code does not state: - `Build.libDir`: why the hook is overridable at all — a lib hook that can only be pointed at the committed tree can only be RUN by overwriting files the suites forge runs in parallel are compiling, and why `regenerateSnapshots` needs no equivalent. - `BuildHarnessWouldWriteTheCommittedLibs`: what the revert means. - `BuildHarness`: that the harness is also a seam for RUNNING `regenerateLibs` somewhere nothing compiles, the empty-string sentinel on `sLibDir` and its constructor `@param`, and the precondition that the directory already exists. - `BuildTest`: that `regenerateLibs()` IS run, into a fixture directory, which is the exception to the paragraph saying nothing here runs a write. - `testTheDefaultLibDirIsWhereTheCommittedLibsAre`: why the path is asserted as text rather than against `LibRainDeploySnapshot.LIB_DIR` — against the constant the assertion cannot fail. - `LIBS_FIXTURE_DIR`: why the fixture root is outside `src/` and `test/`. - `testRegenerateLibsEmitsExactlyTheCommittedLibs`: the gap it closes — every other assertion is output-anchored and both hooks took a `revert()` as their first statement with the suite still green — why the committed tree is the oracle, why the count is asserted, why `regenerateSnapshots` has no counterpart, and the read-then-remove-then-assert ordering. - `testRegenerateLibsRefusesToWriteTheCommittedLibs`: why `setUp`'s harness is the default one and what a call that went through would rewrite. Left cut: `externalLibDir`'s doc block, which restates its signature. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V8ViHcKLVk2YoS2joH4HdN
|
This branch gives `BuildHarness` a constructor; `BuildRecordRootHarness` arrived on main after that and declares its own, so the merge left the base unconstructed and the compile failed. Empty is `Build`'s own lib directory, which is what the record-root harness wants: it only regenerates snapshots, and `externalRegenerateLibs` stays refused. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V8ViHcKLVk2YoS2joH4HdN
Closes #207.
What was wrong
Build's two generator hooks were executed by nothing. Arevert()as thefirst statement of
regenerateLibsleft the suite at 512 passed / 0 failed onbaa1a9c; the same inregenerateSnapshotsleft it at 508 passed / 4 failed,the four being public-RPC rate limits in
LibRainDeployTestthat are therewithout any mutation. Zero delta both ways, re-verified on current
mainratherthan on the
e9a922athe issue was filed against.Every check on the hooks was output-anchored: it compared a committed lib
against the emitters, so it saw drift only AFTER somebody re-ran the generator
and committed what came out. A loop bound that stopped one contract short, or a
constantPrefixtaken fromcontracts[0]on every pass, was invisible untilthe next release cut it into the append-only record.
The issue's own recommended residual — extend the emitter pins to
MigrationRegistry's two committed libs — has already landed, in4eb5363(PR #176), which postdates the commit the issue was verified against. That half
is closed; this PR closes the half the issue's title names.
What changed
regenerateLibstakes its directory from a newinternal virtual libDir()instead of hard-coding
LibRainDeploySnapshot.LIB_DIR, for exactly the reasonBuildScript.recordRootis already overridable: a hook that can only be pointedat the committed tree can only be RUN by overwriting files the rest of the suite
compiles and reads, and forge runs test contracts in parallel. No parameter is
added to
run()orcutRelease(), and nothing outsideBuildcan see thehook.
BuildHarnessgains the fixture directory, anexternalRegenerateLibs()thatrefuses to run when it is left pointing at
src/lib, and anexternalLibDir()so the production default is still assertable.
regenerateSnapshotsgets no such seam and can have none:LibFs.dirForTagconfines every snapshot write to
src/generated/<tag>/, which is the recordfrozenSnapshotPathswalks, so there is no directory to drive it into that thesuites running beside it do not read.
Migration
None.
libDir()isinternalwith the previous constant as its default, so adownstream
Buildthat overrides nothing behaves exactly as before. A consumerrepo with its own
BuildHarnesscopy must pass the constructor's new argument;""reproduces the old behaviour.QA
BuildTest.testRegenerateLibsEmitsExactlyTheCommittedLibs,BuildTest.testTheDefaultLibDirIsWhereTheCommittedLibsAre,BuildTest.testRegenerateLibsRefusesToWriteTheCommittedLibs- each fails onbase because on base the hook cannot be run at all: the two mutations the
issue names were applied to
mainitself and the whole suite stayed green(
revert()inregenerateLibs: 512 passed, 0 failed;i + 1 < contracts.lengthin the same loop: 505 passed, 7 failed, every one of the 7a
vm.createForkrate limit, which is the same kind of failure the unmutatedbaseline has and never an assertion). The same two mutations on this branch
are killed, named below.
own failure message):
script/Build.sol:78regenerateLibs->revert();as its first statement->
testRegenerateLibsEmitsExactlyTheCommittedLibs(EvmError: Revert)script/Build.sol:81i < contracts.length->i + 1 < contracts.length->
testRegenerateLibsEmitsExactlyTheCommittedLibs(emitted a different number of files than the repo commits: 3 != 5)script/Build.sol:83contracts[i].constantPrefix->contracts[0].constantPrefix->testRegenerateLibsEmitsExactlyTheCommittedLibs(emitted something other than the committed LibMigrationRegistryDeploy.sol)script/Build.sol:83LibRainDeploySnapshot.CANDIDATE->"0_1_10"->testRegenerateLibsEmitsExactlyTheCommittedLibs(emitted something other than the committed LibAddressRegistryDeploy.sol)script/Build.sol:72return LibRainDeploySnapshot.LIB_DIR;->return "fixture-lib";->testTheDefaultLibDirIsWhereTheCommittedLibsAre(
fixture-lib != src/lib)test/concrete/BuildHarness.sol:48the empty-dir refusal deleted ->testRegenerateLibsRefusesToWriteTheCommittedLibs(next call did not revert as expected). That run also tooktestEveryCommittedAliasLibIsWhatTheGeneratorEmitsdown with it, readingsrc/lib/LibAddressRegistryDeploy.solas empty while the unguarded writewas in it - the race the refusal exists for, observed.
git status src/libwas clean afterwards, which is the no-op oracle above holding in the one
place it can be checked directly.
checkout is a no-op, so every file
regenerateLibs()writes must bebyte-identical to the file already in
src/lib/, and it must write no others.That is independent of the emitters in the way the existing pins are not: they
say the committed files are what the emitters produce, and this says the hook
asks the emitters for those files.
testTheDefaultLibDirIsWhereTheCommittedLibsAreasserts the literal
src/librather than the constant the default returns,so it is not the implementation compared against itself.
ordering. Arguments: every argument of all three calls in
regenerateLibsiscovered, the three mutations above being
constantPrefix,CANDIDATEand thedirectory;
contractNameandcandidate.snapshotreach the file name and theemitted entries, and
recordRoot()reaches the released lib's import block, soa wrong value in any of them changes the bytes compared. Loop bound: the file
COUNT is asserted as well as the contents, because a short loop writes nothing
wrong - it writes nothing at all - and an extra write is a generated file
nothing regenerates. Call ordering: there is no ordering property inside
regenerateLibsto assert, and asserting one would be theatre -writeAliasLiband
writeReleasedSuitesAggregateread nothing,writeReleasedSuitesLibreadsonly the record, so no permutation of the three changes a byte anywhere. The
ordering that IS observable,
regenerateSnapshotsbeforeregenerateLibs, isalready asserted by
BuildScriptTest.testRunRegeneratesAndFreezesNothingthrough
BuildScriptHarness's markers.regenerateSnapshotsitself staysundrivable for the
LibFsreason above, which is a boundary in a pinnedupstream package rather than a seam this repo declined to open.
nix develop -c forge test --threads 1onbaa1a9c: 507 passed / 5 failed, all5 being public-RPC rate limits in
LibRainDeployTestandRainDeployBroadcastTest(that suite passes 79/79 on its own when the endpointis not rate-limiting). On this branch: 515 passed / 0 failed, 512 + the three
new tests.
🤖 Generated with Claude Code
https://claude.ai/code/session_01V8ViHcKLVk2YoS2joH4HdN