Repository navigation
Regenerate Dependabot pip locks with the repository lock script - #326
Merged
Merged
Conversation
Dependabot's pip updater rewrites requirements*.txt without scripts/lock_requirements.ps1. It drops the --require-hashes header and the universal markers, so every Python update PR needed a manual rerun of the lock script. The Dependabot workflow gains a refresh-python-locks job next to the npm vendor refresh. It reuses the same gate in scripts/dependabot_vendor.py, now with a per-ecosystem profile: Dependabot actor and PR author, a dependabot/pip/ branch in this repository, a failed "Check dependency locks" step, and a change set limited to the three requirement inputs and locks. The lock inputs must be plain version specifiers, the lock script, gate and uv.toml must match the default branch, uv comes from uv.toml without a cache, and UV_NO_BUILD keeps resolution free of package code. Only the three lock files are staged and pushed. Write permissions move from the workflow to each job. The lock script now seeds uv with the committed versions only. uv kept seeded hash lines without checking them, so Dependabot's duplicated numpy hashes survived regeneration and -Check accepted an added hash.
Codex Review Summary: ❌ FailedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Dependabot's pip updater rewrites
requirements.txt,requirements-test.txtandrequirements-graph.txtwithoutscripts/lock_requirements.ps1. It drops the--require-hashesheader and the universal markers (async-timeoutbelow Python 3.11.3,colorama/sspilibon win32) and duplicates the numpy hashes. That breaksdependency-locks,tests/test_dependency_policy.pyand the Redis jobs. In #322 and #325 the lock script had to be rerun by hand.Change
refresh-python-locksjob in.github/workflows/dependabot-vendor.yml, next to the existing npm vendor refresh. It runs only when a Dependabotpull_requestrun ofTestsfails on adependabot/pip/branch of this repository. It reruns the lock script and pushes the three lock files as a[dependabot skip]follow-up commit. Then it dispatchestests.ymlfor the exact new SHA, the same way the npm path does.scripts/dependabot_vendor.pygets one profile per ecosystem (--ecosystem npm|pip, npm is the default). The gate logic is shared. The newcheck-python-inputssubcommand checks the requirement inputs.scripts/lock_requirements.ps1seeds uv with the committed versions only (see "Lock check finding" below).permissions: {}at the top). Both jobs keep the same three scopes.Why not a dependabot.yml change or the
uvecosystem--no-headerplus a custom--require-hashesfirst line, so both updaters fall back to editing pins in place. That is the failure we see.--require-hashesline, and Dependabot would resolve with its own uv rather than the version pinned inuv.toml. The pip-compile behavior described here is based on Dependabot's documented behavior and was not tested against Dependabot itself.dependabot.ymloption (versioning strategy, groups) changes how lock files are written.-Checkexpects. The npm path already follows this pattern.Threat model
Asset. A
contents: write+actions: writetoken in aworkflow_runcontext, and the integrity of the hash-locked dependency set that the release images install.Trust boundary. The workflow definition and every script it executes come from the default branch. Content from the Dependabot branch is data: requirement inputs and locks, resolved by the pinned uv.
Attacker-controlled input. Content on a branch named
dependabot/*, PR metadata, and event fields.Controls, in order:
Job
if:requires all of these: apull_requestrun that failed, actordependabot[bot], head repository equal to this repository (no forks), and adependabot/pip/branch prefix.dependabot_vendor.py validate --ecosystem pipruns from the trusted checkout and checks both the event and the live PR from the API:Tests, event and conclusiondependabot[bot]..requirements{,-test,-graph}.{in,txt}, statusmodifiedonly, and at least one lock changeddependency-locksfailed in the stepCheck dependency locksAny change to
uv.toml, the lock script, the gate, workflows or other files is rejected.Checkout of the head by fetch, then SHA comparison, then detached checkout. The SHA-256 sums of
scripts/lock_requirements.ps1,scripts/dependabot_vendor.pyanduv.tomlfrom the default branch must still match after the checkout. Only the verified copy of the gate in$RUNNER_TEMPruns afterwards.check-python-inputsallows only plainname<specifiers>lines and-rincludes of the other two inputs. That rules out index or find-links options, URLs and VCS references, editable installs, extras, markers, inline hashes and includes outside the set. A Dependabot branch therefore cannot redirect resolution away from PyPI.Resolution:
uv.tomlof the trusted checkout, viasetup-uvbefore the head checkout, withenable-cache: false. No cache is restored, and none is saved into the default-branch scope.UV_NO_BUILD=1keeps resolution metadata-only, so no sdist build backend and no package code runs. Verified locally: the locks come out the same with and without it.persist-credentials: falseleaves no git credentials on disk.Push.
stage --ecosystem pipstages only the three lock files and refuses anything else staged. The remote branch must still be at the validated SHA right before the push. The push is a fast-forward without force. Credentials are set up only in that step.No loop. The follow-up commit is made with
GITHUB_TOKEN, so it does not startpull_requestworkflows. The dispatchedTestsrun is aworkflow_dispatchevent, which the jobif:rejects. Because of[dependabot skip], Dependabot can keep rebasing the PR. After a rebase the job runs again.Residual risk.
dependabot/pip/*branches and never tomain.Lock check finding (fixed here)
lock_requirements.ps1copied the committed lock into the uv output file to keep pins stable. uv carries the hash lines of that seed over without checking them. Consequences:-Checkpassed on a lock with an extra, made-up--hash=sha256:abab…line (reproduced locally).The seed is now written without hash lines and line continuations. Version preferences stay the same, and every hash comes from the index. The committed locks on
mainare unchanged under the new script (-Checkpasses), and the made-up hash is now rejected. Locks from older Dependabot branches stop passing-Checkuntil Dependabot rebases them. The npm path behaves the same way through its checksum check.Verification
pytest tests/test_dependabot_vendor.py tests/test_supply_chain_policy.py tests/test_dependency_policy.py tests/test_toolchain_maintenance.py tests/test_documentation_surface.py tests/test_license_consistency.py tests/test_ci_change_scope.py: 250 passed, 1 skipped (newline file names are impossible on Windows).pwshran under Windows PowerShell 5.1 through a shim.pwshis present.lock_requirements.ps1 -Checkwith uv 0.13.5 (uv.toml): passes.023db50:-Checkfails,check-python-inputspasses.UV_NO_BUILD=1and an empty cache gives byte-identical locks to the manual fix0fdae61.166c599): the regenerated locks are identical to the locks merged in Update npm and Python dependency groups #324.actionlinton the workflow (without shellcheck): clean.Not verified: the workflow has not run on GitHub. The first real Dependabot pip PR after merge is the end-to-end test.