Repository navigation
ci: install recorded dependency versions; watch the newest releases daily - #101
Merged
Merged
Conversation
…aily Required CI installed the newest release of every dependency, so an upstream release could turn an unchanged main red: jax 0.12.0 changed a trace count the FDTD tests pin, and a new transitive dependency (tzdata) failed the license check, each first seen as the failure of an unrelated pull request. Every job of tests.yml except lint, and the docs deploy, now installs through tools/ci_constraints.py at the exact versions in .github/constraints/: one file per environment (tests, docs, distribution), because they disagree -- the PyTorch CPU index mirrors only some versions of torch's dependencies, and the docs extra and twine hold different docutils. PIP_CONSTRAINT and PIP_BUILD_CONSTRAINT carry the distribution file into the smoke's own venv and into isolated builds, setuptools included. The files are seeded from the install logs of main run 39984885892 (051bcae, all jobs green). pyproject.toml is unchanged: floors, extras and what users install stay as they are. latest-deps.yml, outside tests.yml so release-gate never counts it, installs the same environments at the newest releases daily and runs the license check, the quick profile, the backend-sensitive files, the -W docs build and the Linux distribution smoke. A failed scheduled run opens or comments on one issue, closed by the next passing run. On Mondays it opens a draft pull request replacing the files when the newest versions differ (or adds a commit to the open one); that pull request's required CI is the full validation of the new versions. Documented in docs/development/verification.md (English and Korean); the new contract test holds the files' format and the workflows' use of them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsCWziLSzoAVuBy4pCV7J6
Review of the first commit found paths where the recorded versions could still drift or the alarm could mislead: - A commit the refresh job pushed onto an open pull request would start no workflow (pushes made with the workflow token never do), so a ready pull request could sit on unchecked versions, diffed against an old branch rather than main. The job now always opens a new draft pull request from main and closes the one it supersedes; one that already records the same versions is left alone. - macOS and Windows install distributions Linux does not (colorama for build, keyring's backends), which only a run there can record. The distribution canary runs on all three OSes, and `ci_constraints.py merge` combines their freezes, refusing a disagreement. - A refresh could record one of torch's dependencies at a version only PyPI has, which the CPU-index step cannot install. With the recorded versions that step now also looks on PyPI, and torch keeps its `+cpu` label so PyPI's CUDA build of the same version can never match. - A cancelled run (replaced by a newer one, or stopped) no longer opens or closes the canary issue. - The release build (python-publish.yml) used whatever build and setuptools were newest; it now builds on Python 3.12 with the versions in distribution.txt, as the distribution smoke does, and the contract test covers that workflow too. - freeze skips metadata without a name and keeps the first copy of a distribution on the path, the one Python imports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsCWziLSzoAVuBy4pCV7J6
yonghakim
marked this pull request as ready for review
October 27, 2026 04:27
yonghakim
enabled auto-merge (rebase)
October 27, 2026 04:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Required CI installed the newest release of every dependency, so an upstream release could turn an unchanged
mainred. This happened twice: jax 0.12.0 changed a trace count the FDTD tests pin (fixed in #73), and a new transitive dependency (tzdata 2026.5) failed the license check. Both times the break first showed up as a failure on an unrelated pull request.What changes
tests.ymljob exceptlint, plus the docs deploy and the release build, installs through the newtools/ci_constraints.pyat the exact versions in.github/constraints/.tests.txt,docs.txt,distribution.txt) because the environments disagree on versions: the PyTorch CPU index mirrors only some versions of torch's dependencies (MarkupSafe 3.0.3 vs 3.0.4), and the docs extra and twine hold different docutils (0.22.4 vs 0.23).PIP_CONSTRAINTandPIP_BUILD_CONSTRAINTcarry the files into the smoke's own venv and into isolated builds, so setuptools is pinned there too.2.14.1+cpu, so PyPI's CUDA build of the same version can never match it.051bcae, all jobs green). setuptools for the isolated builds is the newest release on that date, which the log does not show directly.pyproject.tomlis unchanged. Floors, extras and what users install stay exactly as they are.latest-deps.yml, a new workflow kept outsidetests.ymlsorelease-gate.ymlnever counts it, and not a required check:-Wdocs build in both languages, and the distribution smoke on Linux, macOS and Windows.refresh): if the newest versions differ from main, it opens a new draft pull request from main replacing.github/constraints/, with the three OS freezes of the distribution set merged, and closes the older refresh PR it supersedes. It never pushes onto an open PR, because a push made with the workflow token starts no CI.docs/development/verification.mdgains "Dependency versions in CI" (English and Korean), with links fromthird-party.mdandtools/README.md.tests/api/test_ci_constraints.pychecks the files' format, that the three workflows install only through the tool, and the merge and freeze rules.Needs a repository setting
For the Monday job to open its pull request: Settings → Actions → General → Allow GitHub Actions to create and approve pull requests. Without it, that step fails with a message naming the setting.
Local checks (Linux, Python 3.12.3)
tools/ci_constraints.py install docs→freeze docsreproducesdocs.txtexactly;check_licenses.pypasses on that env.install distribution-tools+env distribution+tools/run_distribution_smoke.pypasses; the isolated build used the pinned setuptools 84.2.1.test_ci_constraints.py,test_docs_contract.py,test_repository_layout.py,test_installation_metadata.py,test_third_party_licenses.py,test_verification_runner.py,test_verification_environment.py: 271 passed, 1 skipped; quick profile 17 passed;ruff check .clean.ghstubbed, in four cases: nothing new; new versions and no open PR; an open PR that already matches; an open PR that is older (it is superseded and closed).Not verified locally
testsenvironment with torch: this sandbox cannot reach the PyTorch CPU index, sotests.txtis verified by this PR's CI.python-publish.ymlruns only on a release, so its change is checked by the contract test and actionlint, not by a run.🤖 Generated with Claude Code
https://claude.ai/code/session_01EsCWziLSzoAVuBy4pCV7J6