Skip to content

Deploy the documentation site from its own release job - #594

Merged
MaxGhenis merged 3 commits into
masterfrom
fix-docs-deploy
Oct 8, 2026
Merged

MaxGhenis merged 3 commits into
masterfrom
fix-docs-deploy

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

What was broken

The release workflow's "Deploy documentation" step has not run since September 2024.

  • Bug fixes #282 (2024-09-26) dropped the Test job's strategy.matrix (the Windows leg). The deploy step kept if: matrix.os == 'ubuntu-latest', and with no matrix that condition never holds.
  • In release run 37468917800 (2026-10-06) and every recent "Update PolicyEngine Core" run, "Generate documentation" succeeds and "Deploy documentation" is skipped.
  • actionlint on master flags it: property "os" is not defined in object type {}. It also flags GITHUB_TOKEN, which is not an input of JamesIves/github-pages-deploy-action@v4.

What serves the docs today

  • gh api repos/PolicyEngine/policyengine-core/pages: GitHub Pages, build_type: legacy, source gh-pages /, public, at https://policyengine.github.io/policyengine-core/. The README's "documentation" link points there.
  • The gh-pages branch's last commit is f5cd24b8, dated 2024-09-26T00:14:59Z ("Deploying to gh-pages from @ 2e177cc"). Before that, the release workflow redeployed it often: five deploys between 2024-09-03 and 2024-09-26. The latest Pages build is from that same commit. The live site is therefore about two years stale; the package is at 3.32.20.
  • The docs toolchain has not moved. make documentation still runs jb build docs (jupyter-book <1, Sphinx 5.0.0, furo), and both the PR workflow and the release workflow build the docs.
  • Nothing else deploys these docs: the repo has no Read the Docs, Vercel or Netlify config. The default token permission on the repo and the org is write, and no ruleset or protection covers gh-pages.

The deploy is still wanted, so this PR fixes it rather than removing the step.

Change

  • New Docs job in push.yaml: needs: [Test, Publish], the same release-commit if as Test, permissions: contents: write, no matrix. It installs the package, runs make documentation, and deploys docs/_build/html to gh-pages. The deploy step has no condition.
  • Docs deploy only a release that reached PyPI. Because Docs needs Publish, a release whose PyPI publish or tag step fails never reaches the site, and the next good release deploys instead.
  • The build and deploy steps leave the Test job. Once Publish a release only after its tests pass #591 makes Publish needs: Test, a failed docs push inside Test would fail Test and skip the PyPI publish. Publish does not depend on Docs, so a docs failure shows up as its own red job and never holds back a release. One consequence: on release commits, the docs build no longer gates PyPI. That is intended, and pr.yaml still runs make documentation on every PR across the full OS × Python matrix.
  • One deploy at a time: concurrency: {group: gh-pages-deploy, cancel-in-progress: false}. GitHub allows one running job per group, and a newly queued job cancels and replaces an older pending one (github/docs data/reusables/actions/actions-group-concurrency.md). So deploys never race: a running deploy always finishes, and a newly queued one replaces one still waiting, which shows as cancelled in its own release run. GitHub does not guarantee release order. If an older release's Docs job starts waiting after a newer one's, the older docs stay live until the next release.
  • v4's documented inputs: branch: gh-pages and folder: docs/_build/html. The GITHUB_TOKEN input is dropped because v4's token input defaults to ${{ github.token }} (from the action's action.yml). The v4 configuration has never executed: Update GitHub Actions for Node 24 runtime #481 (2026-04) moved the action from v3 to v4 while the step was already being skipped.
  • sphinx.ext.githubpages is added to docs/_config.yml. A branch-sourced Pages site is built with Jekyll unless its root has .nojekyll (GitHub Pages docs, "Creating a GitHub Pages site"). The current jb build output has no .nojekyll. Today's deploy would still keep the existing gh-pages .nojekyll, because v4 passes --exclude .nojekyll to its rsync --delete when the build lacks one (src/git.ts). With the extension, the build writes the file itself. Sphinx 5.0.0's extension writes a CNAME only when html_baseurl is set, and it is not set here.

Invariants (asserted in tests/core/test_docs_deployment.py)

  1. In every workflow file, a job that references matrix.<key> defines that key in strategy.matrix. This is the general form of the bug, and it is checked over push.yaml and pr.yaml.
  2. Exactly one job deploys to Pages: Docs. It needs Test and Publish and has Test's if. Its deploy step is unconditional and runs after make documentation. It deploys docs/_build/html to gh-pages, has contents: write, and runs in a non-cancelling concurrency group. These checks are subset checks, so new action inputs or extra permissions don't break them.
  3. The deployed folder is what make documentation builds (jb build docs), and the docs config enables sphinx.ext.githubpages. The test checks the config; that the build then writes .nojekyll was confirmed by a local build (below).
  4. No job that publishes to PyPI is the Docs job or depends on it, directly or through needs.

On master, invariants 1, 2 and 3 fail with clear messages, so they are regression tests. Invariant 4 already holds on master and guards against future coupling.

Coordination with #591

#591 adds needs: Test to Publish and a test to tests/core/test_release_tagging.py. This PR leaves the Publish job and that test file alone, so the two merge cleanly in either order (both orders checked locally at 0870347f against #591 at 9948a8dd; the combined tree passes all 11 tests in both files and adds no actionlint findings). After both merge, the chain is Test → Publish → Docs. A Docs failure skips nothing, and a Publish failure skips only Docs. #591's assertion jobs["Publish"]["if"] == jobs["Test"]["if"] still holds.

Checks run

  • pytest tests/core/test_docs_deployment.py tests/core/test_release_tagging.py at the latest head: 10 passed (Python 3.14). The new file against master's workflow: 3 failed, 2 passed, as expected.

  • make documentation locally (jupyter-book 0.15.1, Sphinx 5.0.0): exit 0, output in docs/_build/html/, .nojekyll present and no CNAME. Warnings are identical to the build without the extension (30 lines).

  • ruff format --check . and ruff check on the new test: clean.

  • actionlint on both workflows: no findings in the Docs job. Two warnings remain, both on master already: SC2046 quoting in the versioning job's changelog step, and the deprecated skip_existing input in Publish. This PR does not touch them.

  • Pages trigger: the latest Pages build (gh api repos/PolicyEngine/policyengine-core/pages/builds/latest) was triggered by github-actions[bot]'s push of f5cd24b8. Branch pushes made with the workflow token do start legacy Pages builds here.

Not run: the deploy itself. It runs only on "Update PolicyEngine Core" commits on master, so the first real run will be the first release after merge.

Merge needs a go/no-go

Merging this resumes publishing a public site that has not been updated since 2024-09-26. The first release after merge will replace the whole of https://policyengine.github.io/policyengine-core/ with the current docs. That go/no-go is queued for Max. Merging this PR is the go; closing it is the no-go, and the alternative is to remove the step and the README link.

Review

An independent Opus review (subfleet 20261006-161055-core594-review) of a54b770d returned APPROVE WITH NITS. 0870347f addresses its findings: deploy ordering and concurrency, the comment that was only true after #591, over-exact and brittle test parsing, and this body's wording on .nojekyll. The fragment stays changed, as the task specified. A delta review of 0870347f (in-session Opus, because Subfleet had no dispatchable lanes) returned APPROVE WITH NITS. The latest commit addresses those nits: the comment's ordering promise, a constant-group check, whole-line Makefile matching, and strategy: null.

Documentation review

  • Documentation changes: docs/_config.yml (adds the .nojekyll extension). No durable docs describe the deploy, and the README link is already correct.
  • Impact: medium. It restarts a public deploy. It does not change the package.
  • Confidence: high on the workflow structure, since tests, actionlint and a local build all checked it. The first live deploy has not been observed yet.

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits October 6, 2026 16:04
The Test job lost its OS matrix in #282 (2024-09-26) but kept the
deploy step's `if: matrix.os == 'ubuntu-latest'` guard, so the step has
been skipped on every release since and gh-pages last moved on
2024-09-26.

Move the docs build and deploy into a Docs job that needs Test, has
`contents: write`, and uses the v4 action's documented inputs. Keeping
it out of Test means a failed deploy cannot hold back a PyPI release
once Publish needs Test (#591). Add sphinx.ext.githubpages so the build
writes the .nojekyll that a branch-sourced Pages site needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-ups on the Docs job:

- Need Publish as well as Test, so the site never documents a version
  that failed to reach PyPI. Publish still does not depend on Docs.
- Serialize deploys with a gh-pages-deploy concurrency group; a newer
  waiting deploy replaces an older one instead of racing it.
- Loosen the structural tests to the properties that matter (branch,
  folder, contents: write, an unconditional deploy step), parse the
  Makefile recipe by its tab-indented lines, and skip jobs whose matrix
  is built by an expression.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Oct 6, 2026
Add a "Lint workflows" job to the pull request workflow that runs
actionlint 1.7.12 from the official rhysd/actionlint image, pinned by
digest. The image bundles shellcheck and pyflakes, so `run:` scripts are
checked too.

Fix the two warnings it reports on master that #594 does not touch:
quote the version substitution passed to `towncrier build --version`
(SC2046) in push.yaml and in the Makefile `changelog` target, and rename
the deprecated `skip_existing` input of pypa/gh-action-pypi-publish to
`skip-existing`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Say what the concurrency group guarantees: one deploy at a time, a
  newly queued deploy replacing one still waiting. GitHub does not
  promise release order.
- Require a constant concurrency group (a per-run group would not
  serialize anything) and accept the string shorthand.
- Match `jb build docs` as a whole recipe line, so a changed output
  path or a commented-out line no longer passes.
- Treat `strategy: null` like no strategy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Hand-off to the US + core hub.

  • Head: d0b438765d812c89fb35d8d9012bcbc71db85a1e
  • Review: independent Opus reviews.
    • Subfleet 20261006-161055-core594-review reviewed a54b770d and returned APPROVE WITH NITS.
    • An in-session Opus delta review of 0870347f also returned APPROVE WITH NITS. Subfleet had no dispatchable lanes, which is why it ran in-session.
    • A re-check at d0b43876 returned APPROVE, with every nit addressed.
    • The reports are kept locally in ~/reviews/policyengine-core-594/ (review-r1.md, review-r2.md).
  • Passed: at a54b770d, Lint, the changelog check and all 8 Test legs passed, Windows included. Locally at the head: tests/core/test_docs_deployment.py and tests/core/test_release_tagging.py give 10 passed. They merge with Publish a release only after its tests pass #591 (9948a8dd) cleanly in both orders, giving 11 passed. ruff format --check is clean. actionlint shows only the two warnings already on master.
  • Pending: CI at d0b43876 (10 checks queued when this was posted).
  • Hold: merging needs Max's go on d1028, because merging resumes publishing https://policyengine.github.io/policyengine-core/, which has been frozen since 2024-09-26. Merging this PR is the go, and closing it is the no-go. Do not merge on green CI alone.
  • Depends on: nothing. Publish a release only after its tests pass #591 is coordinated but independent: either merge order is fine. Once both are in, the chain is Test → Publish → Docs.
  • Follow-ups owed: a chip to add actionlint to PR CI, fix the two existing actionlint warnings, and make test_publish_workflow_tags_only_after_pypi_succeeds parse YAML instead of slicing text (reviewer finding: it can pass falsely). After the first release that follows the merge, check that the Docs job pushed to gh-pages and that a Pages build followed: gh api repos/PolicyEngine/policyengine-core/pages/builds/latest.

MaxGhenis added a commit that referenced this pull request Oct 6, 2026
test_publish_workflow_tags_only_after_pypi_succeeds sliced push.yaml's
text from "  Publish:" to the end of the file. With a later job holding
contents: write and Publish's own permissions deleted, the slice read
the later job's block and the test passed. It now asserts on
yaml.safe_load(...)["jobs"]["Publish"]: contents: write, a fetch-depth 0
checkout, the tag step after the PyPI step with no condition, no
continue-on-error on either step, and no `|| true` on the tag script.
Mutation cases check that each assertion rejects the mistake it names.

Add .github/actionlint.yaml so Lint workflows passes before #594 lands:
it ignores, in push.yaml only, the two errors in the Test job's docs
deploy step that #594 fixes. Delete both entries in whichever PR fixes
or removes that step.

Document running actionlint locally in the README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Coordination with #597, which adds actionlint to PR CI.

#597 adds .github/actionlint.yaml. Its only entries ignore, in push.yaml only, the two actionlint errors that this PR fixes: property "os" is not defined in object type {} from the matrix.os condition, and the undefined GITHUB_TOKEN input of JamesIves/github-pages-deploy-action@v4. Without them, the new "Lint workflows" job would fail on every PR until this one lands.

The two PRs merge cleanly in either order. Once both have landed, the ignores are stale, and actionlint does not warn about stale ignores. Delete the whole file in whichever PR lands second:

With both merged and the file deleted, actionlint 1.7.12 finds nothing in either workflow. I checked locally on master + #594 (d0b43876) + #597; this PR's test_docs_deployment.py and test_release_tagging.py pass (17 tests).

MaxGhenis added a commit that referenced this pull request Oct 7, 2026
…ublish job (#597)

* Lint GitHub Actions workflows on pull requests

Add a "Lint workflows" job to the pull request workflow that runs
actionlint 1.7.12 from the official rhysd/actionlint image, pinned by
digest. The image bundles shellcheck and pyflakes, so `run:` scripts are
checked too.

Fix the two warnings it reports on master that #594 does not touch:
quote the version substitution passed to `towncrier build --version`
(SC2046) in push.yaml and in the Makefile `changelog` target, and rename
the deprecated `skip_existing` input of pypa/gh-action-pypi-publish to
`skip-existing`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Parse the Publish job in the release-ordering test

test_publish_workflow_tags_only_after_pypi_succeeds sliced push.yaml's
text from "  Publish:" to the end of the file. With a later job holding
contents: write and Publish's own permissions deleted, the slice read
the later job's block and the test passed. It now asserts on
yaml.safe_load(...)["jobs"]["Publish"]: contents: write, a fetch-depth 0
checkout, the tag step after the PyPI step with no condition, no
continue-on-error on either step, and no `|| true` on the tag script.
Mutation cases check that each assertion rejects the mistake it names.

Add .github/actionlint.yaml so Lint workflows passes before #594 lands:
it ignores, in push.yaml only, the two errors in the Test job's docs
deploy step that #594 fixes. Delete both entries in whichever PR fixes
or removes that step.

Document running actionlint locally in the README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Tighten the Publish-job checks from review

- Accept `permissions: write-all`, which also grants contents: write,
  and fail with the named message (not AttributeError) on any other
  string.
- Compare fetch-depth as a string: action inputs are strings, so 0 and
  "0" are the same input, and `false` no longer passes as 0.
- Require the tag step to run exactly `bash .github/publish-git-tag.sh`,
  which also rejects `|| exit 0`, `set +e` and the like, not only
  `|| true`.
- Say that fetch-depth 0 fetches every tag, not that only it does:
  checkout's fetch-tags input also fetches tags.
- Pin the README's local actionlint command to the version CI runs, and
  mention pyflakes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Reject a condition on the PyPI step, and harden the lint job

From the delta review of cbb88b5:

- A condition on the PyPI step (for example `if: github.ref ==
  'refs/heads/main'` on a master-branch repo) skips the upload without
  failing the job, so the tag step still ran. The check now rejects it,
  with its own mutation case.
- Test that the equivalent settings the check accepts, `write-all` and
  `fetch-depth: "0"`, pass.
- Lint workflows runs with `contents: read`, and its checkout does not
  persist the token into the workspace the container mounts.
- The README notes that actionlint must run from the repo root: the
  path-scoped ignores match paths relative to the working directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Say a condition could, not would, let the tag step run

An `if: success()` on either step changes nothing; the check still
rejects any `if:`, since most conditions would skip a step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit b2192d7 into master Oct 8, 2026
26 of 36 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

US + core hub merge audit, core#594 at d0b438765d812c89fb35d8d9012bcbc71db85a1e (squash)

  • Max's ruling d1028 (2026-10-08): resume publishing the policyengine-core docs site.
  • Reviews: Opus APPROVE r1 (Subfleet), then an in-session delta r2, then a re-check APPROVE at d0b4387, this head, with every nit addressed.
  • Gates checked live at merge:
    • gh pr checks exits 0;
    • MERGEABLE;
    • not a draft;
    • no CHANGES_REQUESTED review;
    • head pinned.
  • Impact: release workflow only; no model code changes.
  • Follow-up after core#597: .github/actionlint.yaml held only Deploy the documentation site from its own release job #594's two ignores. Delete it once both have landed.

@MaxGhenis
MaxGhenis deleted the fix-docs-deploy branch October 8, 2026 14:22
MaxGhenis added a commit that referenced this pull request Oct 8, 2026
Keep the #594 release version and changelog alongside the unchanged disk storage repair.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant