ci: fuzz for real with Jazzer.js; cflite.yml is the coverage report it always was - #89
Conversation
sprayberry-redline
left a comment
There was a problem hiding this comment.
Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the gating lane (gating review).
Verdict: request changes. One blocking finding in fuzz/run.mjs: when FUZZ_CORPUS_DIR is unset (the documented local default, "Unset: no corpus"), the only corpus directory handed to Jazzer/libFuzzer is fuzz/seeds/, and libFuzzer writes every new interesting input to the first corpus directory it is given, so a plain node fuzz/run.mjs mutates the committed seed corpus. CI is unaffected because the workflow always sets FUZZ_CORPUS_DIR. I checked the workflow's permissions (contents: read only), pinned actions, the cache restore/save key scheme, that untrusted input only reaches env rather than run:, the fleet-status self-test's name-extraction regex against the new name: Fuzz, Jazzer's extensionless target resolution, and the cflite.yml text changes. Two minor notes: the runner's comment claims npm run fuzz invokes it, but package.json's fuzz script still runs the fast-check battery; and .gitignore now lists fuzz/build/ twice.
1. Blocking: fuzz/run.mjs:43
const seeds = path.join(dir, 'seeds', name);
if (existsSync(seeds)) args.push(seeds);
With FUZZ_CORPUS_DIR unset (the default a developer running node fuzz/run.mjs gets, documented at the top of the file as "Unset: no corpus"), the corpusRoot branch is skipped and fuzz/seeds/ becomes the FIRST and only corpus directory passed to Jazzer, which forwards it to libFuzzer as a positional corpus argument. libFuzzer writes every new interesting input into the first corpus directory, so a local run fills the committed fuzz/seeds/detect_spans, fuzz/seeds/redact_leak and fuzz/seeds/reidentify_stream with generated files that show up in git status and can be committed. The file's own comment ("writes new inputs to the first (persisted) directory only") is only true when FUZZ_CORPUS_DIR is set.
Suggested fix:
Either default corpusRoot to a scratch directory (for example path.join(dir, 'corpus'), which .gitignore already covers) so the seeds dir is always second, or when corpusRoot is empty pass the seeds via a throwaway first directory (mkdtemp) so libFuzzer never writes into fuzz/seeds. Update the header comment to match.
Minor:
fuzz/run.mjs:1: package.json'sfuzzscript isnode --import tsx _test_fuzz.js(the fast-check battery), not this runner; the PR does not change package.json, so the claim is false and a reader following it runs a different tool..gitignore:37: fuzz/build/ is already ignored three lines above (line 33); the new entry is a duplicate.
rule:correctness
…uild is the native binary)
…writes into fuzz/seeds With FUZZ_CORPUS_DIR unset, fuzz/seeds/<target> was the first corpus directory handed to libFuzzer, which writes new inputs there. Default corpusRoot to the gitignored fuzz/corpus so the seeds are always the second (read-only) directory. Also drop the false "npm run fuzz" claim from the header (that script runs the fast-check battery) and the duplicate fuzz/build/ .gitignore entry.
sprayberry-redline
left a comment
There was a problem hiding this comment.
Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the gating lane (gating review).
Verdict: approve. Approve. Checked fuzz.yml for permissions (contents: read only), pinned actions, trigger scope (fork job guard, pull_request rather than pull_request_target, cache saved only on non-PR runs so a PR cannot poison the shared corpus), and the FUZZ_SECONDS expression for schedule/dispatch/PR. Checked fuzz/run.mjs against .clusterfuzzlite/build.sh (same esbuild bundling, --sync, seeds passed as the read-only second corpus dir, Jazzer's ensureFilepath appends .js to the extensionless target) and against package.json ("type": "module", pinned @jazzer.js/core and esbuild). Verified that scripts/fleet-status.test.mjs's workflow_run check is satisfied by adding "Fuzz" (fuzz.yml's on: block contains pull_request). Public text reads as human-written, no attribution or secrets. Two minor notes below, neither blocking.
Minor:
fuzz/run.mjs:34: The entry point, outfile and Jazzer target are cwd-relative (fuzz/...) while the corpus default, seeds dir andmkdirSync(path.join(dir, 'build'))are resolved from the script's own directory. Run from anywhere but the repo root (e.g.cd fuzz && node run.mjs), the mkdir succeeds in the right place but esbuild fails with "Could not resolve fuzz/detect_spans.fuzz.ts". Not a CI problem since the workflow runs from the root, but the header advertises this as the local repro loop.fuzz/run.mjs:53: A crash in the first target (sorted: detect_spans) exits the process, so redact_leak and reidentify_stream are not fuzzed on that run and a second independent bug would only surface after the first is fixed. The workflow header says every target runs for FUZZ_SECONDS.
What
.github/workflows/fuzz.yml(new): every target infuzz/runs under Jazzer.js (libFuzzer) for 300 s weekly and on demand, 60 s on pull requests that touchfuzz/or the code under it. The corpus persists between runs through the Actions cache (restored by prefix, saved under the run id on the default branch). A crash, uncaught throw or timeout fails the job and the reproducing input is uploaded as an artifact.fuzz/run.mjs: the runner the workflow and a developer both use, withFUZZ_SECONDS,FUZZ_CORPUS_DIRandFUZZ_ARTIFACT_DIR; same Jazzer invocation as before, plus the corpus and artifact arguments..github/workflows/cflite.yml: renamed "ClusterFuzzLite coverage", job "Coverage report", header and matrix comment say what it does. Nothing else changes in it;.clusterfuzzlite/stays so Scorecard's Fuzzing check keeps its source..github/workflows/fleet-status.yml:Fuzzjoins theworkflow_runlist (its self-test requires every pull_request workflow to be listed)..gitignore: the corpus and crash dirs.Why
The ClusterFuzzLite action cannot fuzz JavaScript at all, so no run of cflite.yml has ever fuzzed anything: the OSS-Fuzz builder rejects every sanitizer for JS ("JavaScript projects cannot be fuzzed with sanitizers", plumbline run 36204398437), the action's config rejects
none("Invalid SANITIZER: none", run 36204619235), andcoverageselects the coverage-report runner (every past run here: "MODE is: coverage", a 3–5 s "Run fuzzers" step, "Contains 0 elements", only acifuzz-coverage-latestartifact). The workflow's old header promised continuous fuzzing with crashes failing the job; Scorecard credits Fuzzing 10/10 from the config. Same change as askalf/plumbline#60, whose pull-request run fuzzed three targets for real (36205040701).Proof
The pull request run of fuzz.yml on this branch is the test: 60 s per target, Jazzer's own output (INITED, NEW cov, REDUCE) for every target, corpus files written.