Skip to content

ci: fuzz for real with Jazzer.js; cflite.yml is the coverage report it always was - #89

Merged
askalf merged 7 commits into
mainfrom
ci/fuzz-for-real
Sep 26, 2026
Merged

askalf merged 7 commits into
mainfrom
ci/fuzz-for-real

Conversation

@askalf

@askalf askalf commented Sep 26, 2026

Copy link
Copy Markdown
Owner

What

  • .github/workflows/fuzz.yml (new): every target in fuzz/ runs under Jazzer.js (libFuzzer) for 300 s weekly and on demand, 60 s on pull requests that touch fuzz/ or the code under it. The corpus persists between runs through the Actions cache (restored by prefix, saved under the run id on the default branch). A crash, uncaught throw or timeout fails the job and the reproducing input is uploaded as an artifact.
  • fuzz/run.mjs: the runner the workflow and a developer both use, with FUZZ_SECONDS, FUZZ_CORPUS_DIR and FUZZ_ARTIFACT_DIR; same Jazzer invocation as before, plus the corpus and artifact arguments.
  • .github/workflows/cflite.yml: renamed "ClusterFuzzLite coverage", job "Coverage report", header and matrix comment say what it does. Nothing else changes in it; .clusterfuzzlite/ stays so Scorecard's Fuzzing check keeps its source.
  • .github/workflows/fleet-status.yml: Fuzz joins the workflow_run list (its self-test requires every pull_request workflow to be listed).
  • .gitignore: the corpus and crash dirs.

Why

The ClusterFuzzLite action cannot fuzz JavaScript at all, so no run of cflite.yml has ever fuzzed anything: the OSS-Fuzz builder rejects every sanitizer for JS ("JavaScript projects cannot be fuzzed with sanitizers", plumbline run 36204398437), the action's config rejects none ("Invalid SANITIZER: none", run 36204619235), and coverage selects the coverage-report runner (every past run here: "MODE is: coverage", a 3–5 s "Run fuzzers" step, "Contains 0 elements", only a cifuzz-coverage-latest artifact). The workflow's old header promised continuous fuzzing with crashes failing the job; Scorecard credits Fuzzing 10/10 from the config. Same change as askalf/plumbline#60, whose pull-request run fuzzed three targets for real (36205040701).

Proof

The pull request run of fuzz.yml on this branch is the test: 60 s per target, Jazzer's own output (INITED, NEW cov, REDUCE) for every target, corpus files written.

@github-actions github-actions Bot added github_actions Pull requests that update GitHub Actions code fuzz Fuzzing and ClusterFuzzLite size/M 50-199 hand-written lines labels Sep 26, 2026

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer.

Reviewed by the gating lane (gating review).

Verdict: request changes. One blocking finding in fuzz/run.mjs: when FUZZ_CORPUS_DIR is unset (the documented local default, "Unset: no corpus"), the only corpus directory handed to Jazzer/libFuzzer is fuzz/seeds/, and libFuzzer writes every new interesting input to the first corpus directory it is given, so a plain node fuzz/run.mjs mutates the committed seed corpus. CI is unaffected because the workflow always sets FUZZ_CORPUS_DIR. I checked the workflow's permissions (contents: read only), pinned actions, the cache restore/save key scheme, that untrusted input only reaches env rather than run:, the fleet-status self-test's name-extraction regex against the new name: Fuzz, Jazzer's extensionless target resolution, and the cflite.yml text changes. Two minor notes: the runner's comment claims npm run fuzz invokes it, but package.json's fuzz script still runs the fast-check battery; and .gitignore now lists fuzz/build/ twice.

1. Blocking: fuzz/run.mjs:43

const seeds = path.join(dir, 'seeds', name);
if (existsSync(seeds)) args.push(seeds);

With FUZZ_CORPUS_DIR unset (the default a developer running node fuzz/run.mjs gets, documented at the top of the file as "Unset: no corpus"), the corpusRoot branch is skipped and fuzz/seeds/ becomes the FIRST and only corpus directory passed to Jazzer, which forwards it to libFuzzer as a positional corpus argument. libFuzzer writes every new interesting input into the first corpus directory, so a local run fills the committed fuzz/seeds/detect_spans, fuzz/seeds/redact_leak and fuzz/seeds/reidentify_stream with generated files that show up in git status and can be committed. The file's own comment ("writes new inputs to the first (persisted) directory only") is only true when FUZZ_CORPUS_DIR is set.

Suggested fix:

Either default corpusRoot to a scratch directory (for example path.join(dir, 'corpus'), which .gitignore already covers) so the seeds dir is always second, or when corpusRoot is empty pass the seeds via a throwaway first directory (mkdtemp) so libFuzzer never writes into fuzz/seeds. Update the header comment to match.

Minor:

  • fuzz/run.mjs:1: package.json's fuzz script is node --import tsx _test_fuzz.js (the fast-check battery), not this runner; the PR does not change package.json, so the claim is false and a reader following it runs a different tool.
  • .gitignore:37: fuzz/build/ is already ignored three lines above (line 33); the new entry is a duplicate.

rule:correctness

…writes into fuzz/seeds

With FUZZ_CORPUS_DIR unset, fuzz/seeds/<target> was the first corpus directory
handed to libFuzzer, which writes new inputs there. Default corpusRoot to the
gitignored fuzz/corpus so the seeds are always the second (read-only) directory.
Also drop the false "npm run fuzz" claim from the header (that script runs the
fast-check battery) and the duplicate fuzz/build/ .gitignore entry.

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer.

Reviewed by the gating lane (gating review).

Verdict: approve. Approve. Checked fuzz.yml for permissions (contents: read only), pinned actions, trigger scope (fork job guard, pull_request rather than pull_request_target, cache saved only on non-PR runs so a PR cannot poison the shared corpus), and the FUZZ_SECONDS expression for schedule/dispatch/PR. Checked fuzz/run.mjs against .clusterfuzzlite/build.sh (same esbuild bundling, --sync, seeds passed as the read-only second corpus dir, Jazzer's ensureFilepath appends .js to the extensionless target) and against package.json ("type": "module", pinned @jazzer.js/core and esbuild). Verified that scripts/fleet-status.test.mjs's workflow_run check is satisfied by adding "Fuzz" (fuzz.yml's on: block contains pull_request). Public text reads as human-written, no attribution or secrets. Two minor notes below, neither blocking.

Minor:

  • fuzz/run.mjs:34: The entry point, outfile and Jazzer target are cwd-relative (fuzz/...) while the corpus default, seeds dir and mkdirSync(path.join(dir, 'build')) are resolved from the script's own directory. Run from anywhere but the repo root (e.g. cd fuzz && node run.mjs), the mkdir succeeds in the right place but esbuild fails with "Could not resolve fuzz/detect_spans.fuzz.ts". Not a CI problem since the workflow runs from the root, but the header advertises this as the local repro loop.
  • fuzz/run.mjs:53: A crash in the first target (sorted: detect_spans) exits the process, so redact_leak and reidentify_stream are not fuzzed on that run and a second independent bug would only surface after the first is fixed. The workflow header says every target runs for FUZZ_SECONDS.

@askalf
askalf merged commit 4aad780 into main Sep 26, 2026
15 checks passed
@askalf
askalf deleted the ci/fuzz-for-real branch September 26, 2026 00:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fuzz Fuzzing and ClusterFuzzLite github_actions Pull requests that update GitHub Actions code size/M 50-199 hand-written lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants