Skip to content

docs: retract a false claim in the handoff β€” the runner does NOT exit 0 on failure - #312

Merged
ZacxDev merged 1 commit into
mainfrom
docs/correct-handoff-claims
Aug 3, 2026
Merged

docs: retract a false claim in the handoff β€” the runner does NOT exit 0 on failure#312
ZacxDev merged 1 commit into
mainfrom
docs/correct-handoff-claims

Conversation

@ZacxDev

@ZacxDev ZacxDev commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

#308 asserted run-tests.sh "can exit 0 while printing RESULT: FAIL / collected=0". It doesn't, and it can't.

if [ "$fail" -ne 0 ]; then
  echo "RESULT: FAIL"
else
  echo "RESULT: PASS"
fi
exit "$fail"

Measured on 13bc8bd: a run without pytest importable prints 17 Γ— "could not parse pytest's summary", RESULT: FAIL, exit 1 (2 in a second probe). The original report most likely read a status through a pipeline β€” the rc=$? trap RULES.md already names.

The real defect (fixed in #310) is narrower: REQUIRED_TOOLS checks binaries via command -v, but pytest is a module, so a missing pytest produced a diagnostic blaming pytest's output format rather than a clean precondition failure. Bad diagnostics, not a false green.

I relayed this from a subagent report and repeated it to the operator twice without testing it β€” the same class of error as the four corrections the doc itself records, except this one reached a merged document first. Kept as strikethrough rather than deleted, since the doc's whole purpose is recording what was believed and then disproved.

Also closes three items: #309 (529 ungated node tests, not 508 β€” collector/browser-ext/tests was ungated too; gate 468 β†’ 997), #311 (deadman, including the verified "cannot tell β‰  healthy" property), and session-create's cause (session.created is a bus event type, not a hook name β€” 2,736 of 2,799 tool-call rows carry session='').

πŸ€– Generated with Claude Code

… 0 on failure

#308 asserted that scripts/run-tests.sh "can exit 0 while printing RESULT: FAIL
/ collected=0". That does not reproduce and is structurally impossible on this
revision: `RESULT: FAIL` prints only inside `if [ "$fail" -ne 0 ]`, and the very
next statement is `exit "$fail"`. Measured on 13bc8bd, a run without pytest
importable prints 17 x "could not parse pytest's summary", RESULT: FAIL, and
exits 1 (2 in a second probe). The original report most likely read a status
through a pipeline -- the rc=$? trap RULES.md already names.

The real defect, fixed in #310, is narrower: REQUIRED_TOOLS checks BINARIES via
`command -v`, but pytest is a MODULE, so a missing pytest yielded a diagnostic
blaming pytest's output format instead of a clean precondition failure. It also
asserted `python3` while the runner calls `python`. Bad diagnostics, not a false
green.

This was relayed from a subagent report and repeated to the operator twice
without being tested -- the same class of error as the four corrections the doc
itself records, except this one reached a merged document before anyone checked
it. Left visible as a strikethrough rather than deleted, because the doc's
purpose is recording what was believed and then disproved.

Also updates three items now closed:
  - #309 gated the node suites: 529 ungated tests, not the 508 first reported
    (scripts/collector/browser-ext/tests was ungated too). Node gate 468 -> 997.
  - #311 shipped the telemetry deadman, including the verified property that
    "cannot tell" cannot render as "all healthy" -- deadman returns count=0 for
    both ok and unreachable, so a consumer reading count alone would be wrong.
  - session-create's cause is now known: `session.created` is a bus EVENT TYPE,
    not a plugin hook name, on opencode 1.18.4. Consequence: 2,736 of 2,799
    tool-call rows carry session=''. Not fixed -- that file's last edit killed
    11 hours of telemetry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ZacxDev
ZacxDev merged commit 57dde3c into main Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant