Skip to content

Commit 44149b3

Browse files
authored
test: fail the core coverage step when no test ran (#2733) (#3053)
Written-by: Dev Fixes #2733 Owner: Chief Reviewer: Reviewer Tip SHA: d4b68fc Base: 6bd98a7 Docs-not-needed: this changes a step's failure condition inside `.github/actions/flybot-tests/action.yml`; no `docs/subsystems/` page owns it (the pages' owned paths start in `core/`), and no architecture moves. ## What changed The `Unit tests with coverage` step ran the core suite and then only pasted coverage into the step summary, so a glob that matched nothing stayed green: node exits 0 with `ℹ tests 0` / `ℹ skipped 0`, the coverage table reads `all files | 100.00 | 100.00 | 100.00`, and `lcov.info` holds `TN:` alone while `if-no-files-found: ignore` uploads it as an artifact. After the `tee` and before the summary block, the step now reads the count back out of the captured output: ```sh tests=$(sed -n 's/^ℹ tests \([0-9]*\)$/\1/p' test-output.txt | tail -1) echo "tests: ${tests:-?}" [ "${tests:-0}" -ge 1 ] || { echo "::error::core suite ran no tests (glob matched nothing)"; exit 1; } ``` - A missing `ℹ tests` line falls back to `0` and fails, so truncating or renaming the reporter output cannot buy a pass. - `set -o pipefail` is kept and the summary's two lines still come from the same `grep -E 'file +\| line %|all files' | sed 's/^ℹ //' | head -2`, so what the summary shows does not move. - The failing step prints `tests: 0` before the `::error::` annotation; a real run prints `tests: <n>` with `n ≥ 1`. One file changes (`+4`); `.github/workflows/test.yml` and `policy/` are untouched, so this pull request carries no `Policy-sign:`. ### Call sites, checked before landing `grep -rn "flybot-tests" .github/workflows` returns exactly one: ``` .github/workflows/test.yml:69: - uses: ./.github/actions/flybot-tests ``` That call passes `node: '24'`, `dsh: 0.2.1-rc.2`, `coverage: 'true'`, `plugins: 'false'` — a real run of the 300+ file core suite, not an intentional empty one. No call site wants an empty run, so none is exempted and no condition was added to skip the guard. ## Verification at the tip Tip SHA: `d4b68fc022a63bb454b201af04858e800fc90be7` Ran: the step's `run:` block, extracted verbatim from `action.yml` at the tip and at the base blob, executed in a scratch `core/` with `GITHUB_STEP_SUMMARY` set. **Red first — base blob (no guard), empty glob.** The step exits 0 with nothing run: ```text $ node --test … "test/*.test.js" | tee test-output.txt # test/ holds only harness-resolve.mjs ℹ tests 0 ℹ pass 0 ℹ fail 0 ℹ skipped 0 ℹ all files | 100.00 | 100.00 | 100.00 | BASE EXIT=0 ``` **Red first — tip blob, same empty glob.** Same output up to the guard, then it refuses: ```text tests: 0 ::error::core suite ran no tests (glob matched nothing) TIP EXIT=1 ``` **Green — tip blob, one real test file.** One matched test, exit 0, and the summary block is byte-identical in shape to the old one: ```text ℹ tests 1 tests: 1 TIP GREEN EXIT=0 ``` and `summary.md` afterwards (same two lines the old step wrote): ### Coverage: core (ds-bot) (Node 24, DSH x) ``` file | line % | branch % | funcs % | uncovered lines all files | 100.00 | 100.00 | 100.00 | ``` Not run (and why): `npm --prefix core test` / the full 300+ file suite was not run on this box — `core/node_modules` is absent from a tarball checkout and `dev/harness` has no `@deepseek-ai/*` install here, so a local attempt died on `Cannot find module 'jsdom'` / `ERR_MODULE_NOT_FOUND: @deepseek-ai/dsh-llm` before any test of this change could run; that is the environment, not the diff. The full suite is the `test` job's job: `tests-ok` at this tip is the evidence, and the guard itself only reads that job's own output. ## Risk T3 by shape (`.github/actions/**`), and the behavioral change is the point: an empty core run now fails instead of reporting 100% coverage. `merge-gate` recomputes the tier; no `risk:*` label is written by hand. ## Checkpoint <!-- flybot:checkpoint --> ```json { "pr": 3053, "claim": "Chief", "issue": 2733, "summary": "The core coverage step captured node's output but never read the test count, so an empty glob passed with `tests 0`, 100% coverage in the summary and a 4-byte lcov artifact. It now parses the `ℹ tests` line out of test-output.txt after the tee and fails with `::error::core suite ran no tests (glob matched nothing)` when the count is missing or below 1. The summary's two lines and the pipefail behaviour are unchanged, and the repository's only flybot-tests call site is a real run, so nothing is exempted.", "files": [".github/actions/flybot-tests/action.yml"], "checks": { "ran": "bash step-tip.sh / step-base.sh over an empty and a one-test glob in a scratch core/", "result": "green", "at": "d4b68fc022a63bb454b201af04858e800fc90be7" }, "next": "Wait for tests-ok on the tip; the step's own red/green is proven locally, the full suite is CI's." } ``` <!-- /flybot:checkpoint -->
1 parent 6bd98a7 commit 44149b3

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

‎.github/actions/flybot-tests/action.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,10 @@ runs:
132132
--test-reporter=spec --test-reporter-destination=stdout \
133133
--test-reporter=lcov --test-reporter-destination=lcov.info \
134134
"test/*.test.js" | tee test-output.txt
135+
# A glob that matches nothing makes node exit 0 with `tests 0` and 100% coverage (#2733).
136+
tests=$(sed -n 's/^ℹ tests \([0-9]*\)$/\1/p' test-output.txt | tail -1)
137+
echo "tests: ${tests:-?}"
138+
[ "${tests:-0}" -ge 1 ] || { echo "::error::core suite ran no tests (glob matched nothing)"; exit 1; }
135139
{
136140
echo "### Coverage: core (ds-bot) (Node $NODE_V, DSH $DSH)"
137141
echo '```'

0 commit comments

Comments
 (0)