vuln-scanner: fuzz targets a repo already ships (cargo-fuzz) - #863
Conversation
|
hey @aaronjmars — two PRs based on the fuzzing idea from our earlier chat, both tested end to end, not just proposed: this one adds cargo-fuzz to vuln-scanner. proof's in the PR: ran it on firecrawl/anydoc, found a real overflow panic in a dependency (calamine), fixed it upstream (tafia/calamine#705), verified clean. second one: #864 — hunter-22, a bounty-discovery skill with a one-tap button to dispatch vuln-scanner at whatever repo it finds. point-at-a-bounty-run-an-audit, the thing you mentioned wanting to see work first. no rush, just flagging both are up. |
|
Thanks @Svector-anu - the cargo-fuzz-on-shipped-targets idea is a good addition and the crash-routing model (own code -> PVR / dependency -> public PR / harness -> drop) fits the existing disclosure flow well. Two things to fix before this can land: 1. Regenerate eyebrowlock.json (CI blocker). The ~85 lines added for A3.5 shift the third pre-approved 2. Strip secrets from the cargo step (security). A3.5 compiles and runs the target repo's own code, and every dependency's (or run fuzz in a network-denied sub-sandbox / before the disclosure secrets are wired). Nits (not blocking):
Once the lockfile is regenerated and the secrets are scrubbed, happy to run CI and merge. |
…seline lockfile addresses review on aeonfun#863: - the A3.5 fuzz run compiles and executes the target's own code (and every dependency's build.rs) with network open. GH_GLOBAL/GH_TOKEN and the RESEND_* disclosure secrets were live in that env, so a malicious target could exfiltrate them at compile time. scrub them with env -u immediately before both the fuzz run and the crash-reproduction command - nowhere else in this skill executes target code, so nowhere else needs it. - re-baselined eyebrowlock.json for the line-number shift the new section caused (704 -> 796). left the unrelated .claude/skills/aeon/SKILL.md entry untouched - that file is byte-identical to main, its drift is pre-existing and not something this PR should carry. on the two nits: the cargo-fuzz install stays unconditional in stage-vuln-scanner.sh on purpose, matching slither's existing pattern in the same file - the target isn't known until arm A picks it, staging runs before that. and the PR description's calamine claim is corrected below.
|
fixed both blockers:
on the nits: left cargo-fuzz install unconditional in stage-vuln-scanner.sh - matches slither's existing pattern in the same file, and the target genuinely isn't known yet at staging time (arm A picks it after). happy to revisit if you want a different tradeoff there. corrected the calamine line in the description - #705 is closed as a duplicate of #696, not a landed fix, updated to say that plainly. ready for another look. |
|
Triage: NEEDS-CHANGES — this PR bundles skill-scope changes (
|
Run the target's own cargo-fuzz harness (when it ships one) as part of arm A: seed the corpus from tests/fixtures where they exist, run each target for ~90s (capped at 8 targets), and route a crash through the same triage as any scanner hit before it counts as a finding. Crash routing depends on where it lands: target's own code -> PVR, a dependency -> public PR against that dependency, harness lying -> drop. Verified end to end against firecrawl/anydoc: its xlsx fuzz target crashed on an integer-overflow panic inside calamine (a dependency), root-caused, fixed, and filed upstream (tafia/calamine#705). Secrets are scrubbed from the fuzz env (env -u GH_TOKEN -u GH_GLOBAL -u RESEND_*) immediately before compiling/running the target's own code, since a malicious target's build.rs could otherwise exfiltrate them. Split out of aeonfun#863 per triage: this PR is skill-scope only (skills/vuln-scanner/SKILL.md + eyebrowlock.json). The paired runtime-infrastructure changes (.github/workflows/aeon.yml, scripts/stage-vuln-scanner.sh, scripts/skill_mode.sh) are in a separate follow-up PR for a maintainer to review, since they touch the fork's trust boundary (Bash(cargo:*) grant, toolchain staging).
0329bdc to
0cb736b
Compare
|
split per triage:
#868 has no effect until this one merges — it just stages a toolchain and widens a grant that nothing uses yet. |
…#868) Runtime-infrastructure half of the cargo-fuzz addition, split out of #863 per triage since it touches the fork's trust boundary and needs maintainer review separately from the skill-scope change. - stage-vuln-scanner.sh: install a nightly Rust toolchain + cargo-fuzz unconditionally in the pre-run staging step, mirroring the existing slither/foundry pattern (the sandbox denies toolchain installs in-run, and the scan target isn't known until Arm A picks it — the skill's own command -v + fuzz/fuzz_targets guard makes this a no-op on repos that don't ship a fuzz harness). - skill_mode.sh: grant Bash(cargo:*) to the write-tier tool allowlist. Wider than the single-purpose scanner grants beside it since `cargo fuzz run` dispatches through cargo itself — called out inline since it's a real widening of what vuln-scanner can execute. - aeon.yml: comment update describing the new staging step. Depends on the skill-scope PR (vuln-scanner: fuzz targets a repo already ships) landing first — this has no effect without the A3.5 step in SKILL.md that reaches for cargo-fuzz.
what this adds
vuln-scanner's scan arm only ever ran static tools (semgrep, trufflehog, osv-scanner) — they read the target's files, they never execute anything. some repos already ship their own
cargo fuzzharnesses for exactly the class of bug static tools can't see: a panic that only shows up on one specific malformed input. if the scanned repo has one, arm A now runs it.step A3.5 in
skills/vuln-scanner/SKILL.md: seed the corpus from the repo's owntests/fixtureswhere they exist, run each fuzz target for ~90s (capped at 8 targets so this stays bounded), and treat a crash as a candidate that gets the same triage rigor as any scanner hit before it counts as a finding.staging is unconditional, same pattern as slither today — the scan target isn't known until arm A picks it, so
stage-vuln-scanner.shinstalls a nightly toolchain + cargo-fuzz on every run, beforeclaude -pstarts (mirrors howstage-deploy-uni-hook.shalready stages foundry — the sandbox denies toolchain installs in-run). the skill's owncommand -v cargo-fuzz+fuzz/fuzz_targetscheck makes it a clean no-op for the repos that don't have one, so it never costs anything on a run that can't use it.the trade-off, stated plainly
the existing scanners are read-only. this compiles and runs the target's own code inside the sandboxed run. that's a real step up in what the skill trusts, not a free extra scanner, and the SKILL.md addition says so explicitly rather than glossing over it. it's the same trust boundary any CI system already accepts when it builds a repo's test suite — ephemeral runner, only this skill's own scoped secrets — but worth being upfront about since it's a different risk shape than the rest of arm A.
scripts/skill_mode.shneedsBash(cargo:*)for this, which is broader than the single-purpose scanner grants (cargo fuzz rundispatches throughcargoitself). called that out in the comment next to the grant.routing addition
a fuzz crash can land in a dependency instead of the target's own code. that routes differently: fix or report it in the dependency's repo, not the original target, since the target's only real next step is bumping a version once one exists. added a row to the A5 table and a short section under A3.5 walking through the three shapes a crash can take (target's own code / a dependency / the harness itself lying) and how each one routes or gets dropped.
proof, not a guess
ran this against
firecrawl/anydoc(a real, unrelated target, not a demo repo). its ownxlsxfuzz target crashed within about a minute on an integer-overflow panic — insidecalamine, a dependency, not anydoc's own code. root-caused it, fixed it matching the file's existing conventions, added a regression test, verified clean against the exact crash input, filed tafia/calamine#705. turned out someone else (#696) had already reported and mostly fixed the same root cause two weeks earlier — closed #705 as a duplicate once that surfaced. genuine bug, real fix, just not our fix that lands: exactly the case a prior-art check before filing is meant to catch.also filed against a real, already-public CVE the osv-scanner leg of the same run turned up: firecrawl/pdf-inspector#310, firecrawl/anydoc#67.
scope, on purpose
rust + cargo-fuzz only, for this pass. other ecosystems (libfuzzer/afl for C/C++, go-fuzz, atheris for python, trident for solana/anchor) fit the same shape later — each behind its own
command -vguard, staged the same unconditional way. didn't want to land a bigger surface than what's actually been run and checked.