Skip to content

feat: add zizmor audit job to CI quality workflow - #1530

Open
planetf1 wants to merge 5 commits into
generative-computing:mainfrom
planetf1:issue-331
Open

feat: add zizmor audit job to CI quality workflow#1530
planetf1 wants to merge 5 commits into
generative-computing:mainfrom
planetf1:issue-331

Conversation

@planetf1

Copy link
Copy Markdown
Contributor

Summary

Closes #331

Adds a zizmor static analysis job to the reusable quality.yml workflow, which runs on every PR via ci.yml. The job fails on findings, blocking PR merges natively — as requested in the issue.

Changes

Single job added to .github/workflows/quality.yml, modelled on the existing actionlint job:

zizmor:
  runs-on: ubuntu-latest
  permissions:
    contents: read
  steps:
    - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
      with:
        persist-credentials: false
    - name: Audit GitHub Actions workflows with zizmor
      uses: zizmorcore/zizmor-action@3dc1ecc9bcb9e94e9b2c709687979e1298497054 # v0.6.2
      with:
        advanced-security: false
      env:
        GH_TOKEN: ${{ github.token }}

Design decisions

  • advanced-security: false — job fails on findings, which natively blocks the PR. The Advanced Security mode (SARIF upload to Security tab) does not fail the job by default and would require a separate repo ruleset to gate merges.
  • GH_TOKEN from github.token — enables online audits (impostor-commit, ref-confusion, typosquat-uses, known-vulnerable-actions). Uses the ephemeral per-job token, not a stored secret. Scoped to contents: read by the permissions: block.
  • SHA-pinned at v0.6.2 (3dc1ecc9...), matching the supply-chain discipline of the rest of the repo.
  • No new violations — the existing # zizmor: ignore[] annotations on hold.yml, pr-label.yml, pr-update.yml, and the HF_TOKEN steps cover all known intentional exceptions. zizmor reports 5 suppressed, 0 findings.

Adds a zizmor static analysis job to the reusable quality workflow,
running on every PR via ci.yml. The job fails on findings, which
blocks PR merges natively.

- advanced-security: false so the job exit code gates the PR check
- GH_TOKEN set from github.token to enable online audits (impostor
  commits, ref-confusion, typosquat-uses, known-vulnerable-actions)
- contents: read only; no additional secrets required
- SHA-pinned at zizmorcore/zizmor-action v0.6.2
- Existing zizmor: ignore[] annotations suppress known intentional
  exceptions (dangerous-triggers on hold/pr-label/pr-update,
  secrets-outside-env on HF_TOKEN steps)

Closes generative-computing#331

Assisted-by: IBM Bob
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@planetf1
planetf1 requested a review from a team as a code owner August 11, 2026 10:24
@github-actions github-actions Bot added the enhancement New feature or request label Aug 11, 2026
Adds .github/zizmor.yml with per-line ignore rules for all pre-existing
findings discovered when the zizmor CI job ran for the first time. Each
suppressed finding has a corresponding open issue (to be filed); the
definition of done for each issue is: fix the root cause AND remove the
suppression entry.

Also corrects stale version comments on SHA-pinned actions (# v6 ->
# v6.0.2, # v5 -> # v5.0.5, etc.) where those were mechanical and safe.

Findings suppressed (all pre-existing, none introduced by this PR):
- ref-version-mismatch: stale version comments across 7 files
- unpinned-uses: actions/create-github-app-token@v1 in release workflows
- github-app: app token permission scoping in release workflows
- template-injection: workflow_dispatch inputs in dispatch-to-contribs.yml
- cache-poisoning: setup-uv/setup-node caching in docs-publish.yml
- excessive-permissions: missing permissions block in dispatch-to-contribs.yml
- artipacked: implicit credential persistence in publish-release.yml snapshot job

New violations in any of these rules on any OTHER line will still fail CI.

Assisted-by: IBM Bob
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

@jakelorocco jakelorocco left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is added as a part of the quality.yml, does this mean it will run on release CI/CD as well? I'm not sure it matters but that might be unnecessary and we should make sure that we can still push through a release if this check fails.

@markstur markstur left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the "issue to be filed" placeholders should be filled out before merging otherwise why comment.

The other concern is whether this should be merged before some of those are fixed. Ideally a few of those existing issues would be fixed before this is merged (basically as part of testing this), but exactly how reasonable that effort is I don't know. It makes sense that at some point we merge this with issues created to fix the existing problems.

Replaces the "<issue to be filed>" placeholders in .github/zizmor.yml
with the follow-up issues (generative-computing#1531-generative-computing#1534) filed for each finding group,
addressing review feedback that placeholders shouldn't ship unfilled.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@planetf1

planetf1 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

@markstur Filled in the zizmor.yml placeholders in 2464f43: ref-version-mismatch#1533, unpinned-uses/github-app#1531, template-injection#1532, cache-poisoning/excessive-permissions/artipacked#1534 (mapping from #331). Each issue's DoD is fix + remove the suppression.

@jakelorocco fair point re release CI — in 950a82d, moved the zizmor job out of quality.yml into its own reusable workflow (actions-audit.yml), wired into ci.yml (PR gate) only, not into publish-release.yml's code-checks. zizmor audits the workflow files, not the code being released, so a new finding there shouldn't be able to block a release.

quality.yml is called by both ci.yml (PR merge gate) and
publish-release.yml (code-checks, required for release). zizmor audits
the CI/CD workflow files themselves rather than the code being
released, so a new finding there shouldn't be able to block a release
the way a ruff/mypy/test failure can.

Move the zizmor job into its own reusable workflow
(actions-audit.yml) and wire it into ci.yml only. Verified locally
with actionlint and zizmor (offline mode): no new findings, existing
suppressions still apply.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

@ajbozarth ajbozarth left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some feedback from Claude:

Nice — design is sound (separate actions-audit.yml off the release path, SHA-pinned, correctly scoped perms), and all four tracking issues (#1531–1534) exist and map to the right suppression groups. Verified locally with zizmor v1.29.0: config passes clean at 20 ignored, 36 suppressed. One cleanup below.

Comment thread .github/zizmor.yml Outdated
The version comments the ignores referenced (e.g. "# v6" -> "# v6.0.2")
were already corrected in this PR, so the rule no longer fires on any of
the 14 previously-ignored lines. Verified with zizmor 1.29.0 at both
default and pedantic personas: removing the group leaves the result
identical (20 ignored, 36 suppressed, 0 findings). Dropping the
suppression keeps the rule enforced everywhere going forward instead of
leaving blind spots on those specific lines.

generative-computing#1533 can be closed as already fixed by this PR.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@planetf1
planetf1 enabled auto-merge August 17, 2026 09:08
@planetf1

Copy link
Copy Markdown
Contributor Author

I’ve removed the unnecessary suppression - should be good to go with a review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: add a test that enforces zizmor checks on our github actions

4 participants