Repository navigation
CI: real-tool smoke for govulncheck and grype - #2370
Merged
Merged
Conversation
Adds real semgrep 1.157.0 (repo rules, no registry pack) and njsscan 0.4.2 canaries to the real-hub job. They install from their own hash-locked scripts/real-hub-requirements-semgrep.txt venv because semgrep pins click 8.1.8, jsonschema 4.25.1 and wcmatch 8.5.2, older than the versions the other tools resolve to. The trufflehog canary matches the exact detector name Github. provision_tools.sh pins checkov 3.4.5. review_checks.sh runs njsscan with the sandboxed HOME and no semgrep metrics, asserted by the hub stub test. Tests: 21/21 (hub secrets-node stub test; k8s-js and js-iac-license hub tests pass; real-hub canaries run in CI) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The real-hub semgrep and njsscan venv now resolves semgrep 1.182.1, the version provision_tools.sh installs for users (njsscan 0.4.2 is compatible). Its Linux wheels are manylinux_2_34, so the lock is re-derived for that platform. test-provision-tools.sh now expects the checkov==3.4.5 pin. Tests: 61/61 (test-provision-tools.sh) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The real-hub job now installs tflint 0.66.0 and kics 2.1.20 (sha256 from each release checksums.txt; kics queries from the release commit fetched by full sha and verified), cppcheck 2.13.0-2ubuntu3 (exact version from the signed noble archive; job pinned to ubuntu-24.04) and DevSkim CLI 1.0.100 (nupkg sha512 = nuget.org catalog packageHash). Each gets a fictional canary asserted through review_checks.sh: tool ran, expected rule on the canary file, no not-run or NOT CHECKED line. syft is skipped: the hub never counts it as ran. Tests: kics canary passes locally; tflint, cppcheck and devskim canaries run only in CI (not runnable in the local sandbox). Gates: bash -n, actionlint, privacy, size, size-ratchet, test-provision-tools.sh all green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…x comments cppcheck is now the noble .deb fetched from its stable Launchpad URL and checked with sha256 (an exact apt version breaks once superseded); runtime deps still come from apt. The DevSkim nupkg is also checked with `dotnet nuget verify` when the runner's dotnet has that verb. The real-hub job header no longer calls cppcheck stub-only, and the raw kics rerun prints results.json even when kics exits non-zero. Tests: 5/5 gates (bash -n, actionlint, privacy, size, size-ratchet); test-provision-tools.sh passes against the committed provision_tools.sh (the worktree copy is stale and unrestorable in this sandbox). Canaries unchanged: kics ran locally earlier, tflint/cppcheck/devskim run in CI only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extend the real-hub job to run govulncheck 1.9.0 and grype 0.121.3 through review_checks.sh on fictional canaries, asserting each RAN with its expected rule and no not-run or NOT CHECKED line. grype: linux_amd64 archive checked against the release checksums file (and a literal hash). The canary is a one-component CycloneDX SBOM (lodash 4.17.15), because the hub points grype at SBOM files only, never at a lockfile. govulncheck: no release binaries, so built with go install on the pinned Go 1.26.11 (GOTOOLCHAIN=local), module hash asserted from go version -m, source verified by the Go checksum DB. The canary calls yaml.Unmarshal of the real module gopkg.in/yaml.v2 2.2.2; go mod tidy fills go.sum and the module cache, then the hub runs with GOPROXY=off. Both vulnerability DBs are ONLINE and NOT pinned: govulncheck has no env override for -db, so it reads vuln.go.dev at run time; the grype DB comes from grype db update (checksum-verified against its listing; a dated archive pin would rot when upstream rotates it out). Locally both tools passed the hub canary against mirrored DBs; the real CI run is the first with the live endpoints. Tests: 7/7 (bash -n, actionlint, privacy grep, size gate, size-ratchet, fragments, scripts/test-provision-tools.sh 61 ok 0 fail) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d grype smoke Raise the real-hub timeout to 25 minutes. Retry up to 3 times with backoff around the govulncheck go install, grype db update, go mod tidy for the canary, and the hub run while govulncheck or grype reads as not run (vuln.go.dev is fetched at run time). The grype canary now also accepts the CVE aliases of its GHSA advisories. The canary go.mod stays at the repo root: the hub runs govulncheck only from a root go.mod, so moving it would make the tool NOT CHECKED. Instead the test now fails if gosec or staticcheck report any row on the govulncheck canary package; gosec's own canary rows are unchanged in a local run. Tests: 7/7 (bash -n, actionlint, privacy grep, size gate, size-ratchet, fragments, scripts/test-provision-tools.sh 61 ok 0 fail) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts in ci.yml and test-review-checks-real-hub.sh resolved to the smoke-7 side (main carried only the squashed smoke-6 content). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the real-hub CI job to govulncheck 1.9.0 (built with go install on a pinned Go 1.26.11 toolchain, module hash asserted) and grype 0.121.3 (sha256 checked against the release checksums file). Each tool runs on a fictional canary (a go.mod requiring a known-vulnerable module version, and a CycloneDX SBOM listing a known-vulnerable package), and the job fails unless the tool ran and reported the expected finding there.
Both vulnerability databases are fetched online and are not pinned: govulncheck has no override for its database, and the grype database cannot be hash-pinned yet. A database outage therefore fails this job loudly; it never passes silently.
Tests: actionlint, bash -n, the privacy gate, the size gate and ratchet, the fragments check and test-provision-tools pass locally. The canaries run in CI.
🤖 Generated with Claude Code