A QSKIP lets the corpus and stress suites pass having compared nothing

[mberrys]

A QSKIP in a suite whose purpose is to compare something lets the suite report Passed having compared nothing: QtTest's exit code counts only failing rows, so a skipped row exits 0 and ctest reports the test as passed.

Found while fixing exactly this defect in the new GUI↔CLI preflight parity guard (UnitTests/tst_editorhosttest.cpp), where the reviewer proved it by hiding the fixture:

mv loop-preflight/testdata/fixtures/color-rgb.pdf $TEMP
UnitTestsEditorHost exportedPreflightReportMatchesPdfToolForTheSameInputs
  SKIP : EditorHostTest::exportedPreflightReportMatchesPdfToolForTheSameInputs() Corpus fixture not generated yet...
  Totals: 2 passed, 0 failed, 1 skipped          <-- exit 0
ctest -R '^UnitTestsEditorHost$'  ->  100% tests passed, 0 tests failed out of 1

The same pattern remains at six sites:

• UnitTests/tst_preflightcorpus.cpp:267, :274, :312, :319 — the CLI-side oracle that pins PdfTool preflight against the committed snapshots. A partial or sparse checkout silently disables the oracle.
• UnitTests/tst_bleedstresstest.cpp:188, :221 — the AI-artwork fixture, which is generated rather than committed, so this is the expected state on a clean checkout. Those two may legitimately need a skip; the corpus and parity cases do not, because their fixtures are tracked (git ls-files loop-preflight/testdata/fixtures).

Why it matters: these are the tests that catch divergence between the surfaces. A guard that passes when it cannot run is worse than no guard, because the green is read as evidence.

Suggested fix, matching the change already made to the parity slot: where the fixture is tracked, replace the skip with a hard failure carrying the actionable message (QVERIFY2(QFile::exists(...), "...; run <generator> ...")). Where the fixture is genuinely generated and absent by design (the bleed stress artwork), keep the skip but make its absence visible in the gate rather than indistinguishable from success — e.g. enough distinct suites that a report-level check can tell "skipped" from "passed", or a forced-failure mode that CI sets when the fixture should exist.

Not in scope of the #195 train that found it; filed so the class is fixed rather than the single instance.

[mberrys]

Status update (2026-09-12, at origin/dev = 0d412b01): four of the six sites are already gone.

11d0deaa ("fix: close 0.2.1 processing and conversion gaps") reworked exactly these files and removed all four corpus-side skips:

$ git show origin/dev:UnitTests/tst_preflightcorpus.cpp | grep -n QSKIP
(no output — the 267/274/312/319 skips are gone)

Two remain, both in the bleed stress suite:

$ git show origin/dev:UnitTests/tst_bleedstresstest.cpp | grep -n QSKIP
207:        QSKIP(qPrintable(message));
245:        QSKIP(qPrintable(message));

Those two are the case this issue's body already distinguished: the AI-artwork fixture is generated, not committed, so absence is the expected state on a clean checkout and a hard failure there would break every fresh clone. The remaining question is narrower — that absence must be visible in the gate rather than indistinguishable from a pass. Options: keep the skip and report the suite as skipped with its own result (so a fully-skipped suite cannot read as green), or gate on the fixture's existence when LOOP_*_ARTWORK_FIXTURES=require.

Closing is not warranted yet for that reason; the corpus oracle — the part that could silently disable a comparison on a sparse checkout — is fixed.