docs: retract a false claim in the handoff β the runner does NOT exit 0 on failure - #312
Merged
Merged
Conversation
β¦ 0 on failure #308 asserted that scripts/run-tests.sh "can exit 0 while printing RESULT: FAIL / collected=0". That does not reproduce and is structurally impossible on this revision: `RESULT: FAIL` prints only inside `if [ "$fail" -ne 0 ]`, and the very next statement is `exit "$fail"`. Measured on 13bc8bd, a run without pytest importable prints 17 x "could not parse pytest's summary", RESULT: FAIL, and exits 1 (2 in a second probe). The original report most likely read a status through a pipeline -- the rc=$? trap RULES.md already names. The real defect, fixed in #310, is narrower: REQUIRED_TOOLS checks BINARIES via `command -v`, but pytest is a MODULE, so a missing pytest yielded a diagnostic blaming pytest's output format instead of a clean precondition failure. It also asserted `python3` while the runner calls `python`. Bad diagnostics, not a false green. This was relayed from a subagent report and repeated to the operator twice without being tested -- the same class of error as the four corrections the doc itself records, except this one reached a merged document before anyone checked it. Left visible as a strikethrough rather than deleted, because the doc's purpose is recording what was believed and then disproved. Also updates three items now closed: - #309 gated the node suites: 529 ungated tests, not the 508 first reported (scripts/collector/browser-ext/tests was ungated too). Node gate 468 -> 997. - #311 shipped the telemetry deadman, including the verified property that "cannot tell" cannot render as "all healthy" -- deadman returns count=0 for both ok and unreachable, so a consumer reading count alone would be wrong. - session-create's cause is now known: `session.created` is a bus EVENT TYPE, not a plugin hook name, on opencode 1.18.4. Consequence: 2,736 of 2,799 tool-call rows carry session=''. Not fixed -- that file's last edit killed 11 hours of telemetry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#308 asserted
run-tests.sh"can exit 0 while printingRESULT: FAIL/collected=0". It doesn't, and it can't.Measured on
13bc8bd: a run without pytest importable prints 17 Γ "could not parse pytest's summary",RESULT: FAIL, exit 1 (2 in a second probe). The original report most likely read a status through a pipeline β therc=$?trap RULES.md already names.The real defect (fixed in #310) is narrower:
REQUIRED_TOOLSchecks binaries viacommand -v, but pytest is a module, so a missing pytest produced a diagnostic blaming pytest's output format rather than a clean precondition failure. Bad diagnostics, not a false green.I relayed this from a subagent report and repeated it to the operator twice without testing it β the same class of error as the four corrections the doc itself records, except this one reached a merged document first. Kept as strikethrough rather than deleted, since the doc's whole purpose is recording what was believed and then disproved.
Also closes three items: #309 (529 ungated node tests, not 508 β
collector/browser-ext/testswas ungated too; gate 468 β 997), #311 (deadman, including the verified "cannot tell β healthy" property), andsession-create's cause (session.createdis a bus event type, not a hook name β 2,736 of 2,799tool-callrows carrysession='').π€ Generated with Claude Code