Skip to content

Regenerate Dependabot pip locks with the repository lock script - #326

Merged
bifrost0x merged 2 commits into
mainfrom
ci/dependabot-python-locks
Nov 9, 2026
Merged

bifrost0x merged 2 commits into
mainfrom
ci/dependabot-python-locks

Conversation

@bifrost0x

Copy link
Copy Markdown
Owner

Problem

Dependabot's pip updater rewrites requirements.txt, requirements-test.txt and requirements-graph.txt without scripts/lock_requirements.ps1. It drops the --require-hashes header and the universal markers (async-timeout below Python 3.11.3, colorama/sspilib on win32) and duplicates the numpy hashes. That breaks dependency-locks, tests/test_dependency_policy.py and the Redis jobs. In #322 and #325 the lock script had to be rerun by hand.

Change

  • refresh-python-locks job in .github/workflows/dependabot-vendor.yml, next to the existing npm vendor refresh. It runs only when a Dependabot pull_request run of Tests fails on a dependabot/pip/ branch of this repository. It reruns the lock script and pushes the three lock files as a [dependabot skip] follow-up commit. Then it dispatches tests.yml for the exact new SHA, the same way the npm path does.
  • scripts/dependabot_vendor.py gets one profile per ecosystem (--ecosystem npm|pip, npm is the default). The gate logic is shared. The new check-python-inputs subcommand checks the requirement inputs.
  • scripts/lock_requirements.ps1 seeds uv with the committed versions only (see "Lock check finding" below).
  • Permissions move from workflow level to each job (permissions: {} at the top). Both jobs keep the same three scopes.

Why not a dependabot.yml change or the uv ecosystem

  • Dependabot's pip and uv updaters regenerate compiled requirements only when the file carries the compile header. They read the original command from that header. Our locks are written with --no-header plus a custom --require-hashes first line, so both updaters fall back to editing pins in place. That is the failure we see.
  • Restoring a header would change the lock format. It would still not reproduce the --require-hashes line, and Dependabot would resolve with its own uv rather than the version pinned in uv.toml. The pip-compile behavior described here is based on Dependabot's documented behavior and was not tested against Dependabot itself.
  • No dependabot.yml option (versioning strategy, groups) changes how lock files are written.
  • Generating the locks in the repository with the pinned script is the only way to get the same bytes that -Check expects. The npm path already follows this pattern.

Threat model

Asset. A contents: write + actions: write token in a workflow_run context, and the integrity of the hash-locked dependency set that the release images install.

Trust boundary. The workflow definition and every script it executes come from the default branch. Content from the Dependabot branch is data: requirement inputs and locks, resolved by the pinned uv.

Attacker-controlled input. Content on a branch named dependabot/*, PR metadata, and event fields.

Controls, in order:

  1. Job if: requires all of these: a pull_request run that failed, actor dependabot[bot], head repository equal to this repository (no forks), and a dependabot/pip/ branch prefix.

  2. dependabot_vendor.py validate --ecosystem pip runs from the trusted checkout and checks both the event and the live PR from the API:

    • workflow name Tests, event and conclusion
    • workflow actor and PR author are both dependabot[bot]
    • full branch regex, which also rejects ..
    • head SHA of the run equals the PR head SHA
    • base is the default branch
    • the run has exactly one PR, and the changed-file count equals the file list (≤ 20)
    • every changed file is one of the six names requirements{,-test,-graph}.{in,txt}, status modified only, and at least one lock changed
    • the job dependency-locks failed in the step Check dependency locks

    Any change to uv.toml, the lock script, the gate, workflows or other files is rejected.

  3. Checkout of the head by fetch, then SHA comparison, then detached checkout. The SHA-256 sums of scripts/lock_requirements.ps1, scripts/dependabot_vendor.py and uv.toml from the default branch must still match after the checkout. Only the verified copy of the gate in $RUNNER_TEMP runs afterwards.

  4. check-python-inputs allows only plain name<specifiers> lines and -r includes of the other two inputs. That rules out index or find-links options, URLs and VCS references, editable installs, extras, markers, inline hashes and includes outside the set. A Dependabot branch therefore cannot redirect resolution away from PyPI.

  5. Resolution:

    • uv comes from uv.toml of the trusted checkout, via setup-uv before the head checkout, with enable-cache: false. No cache is restored, and none is saved into the default-branch scope.
    • UV_NO_BUILD=1 keeps resolution metadata-only, so no sdist build backend and no package code runs. Verified locally: the locks come out the same with and without it.
    • The generation step has no token in its environment, and persist-credentials: false leaves no git credentials on disk.
  6. Push. stage --ecosystem pip stages only the three lock files and refuses anything else staged. The remote branch must still be at the validated SHA right before the push. The push is a fast-forward without force. Credentials are set up only in that step.

  7. No loop. The follow-up commit is made with GITHUB_TOKEN, so it does not start pull_request workflows. The dispatched Tests run is a workflow_dispatch event, which the job if: rejects. Because of [dependabot skip], Dependabot can keep rebasing the PR. After a rebase the job runs again.

Residual risk.

  • A compromised Dependabot itself is out of scope. It could already open PRs with arbitrary version bumps.
  • The resolved versions still come from PyPI within the specifiers in the inputs. The follow-up commit, like every Dependabot PR, needs review before merge.
  • The job writes only to dependabot/pip/* branches and never to main.

Lock check finding (fixed here)

lock_requirements.ps1 copied the committed lock into the uv output file to keep pins stable. uv carries the hash lines of that seed over without checking them. Consequences:

  • Dependabot's duplicated numpy hashes survived a regeneration.
  • -Check passed on a lock with an extra, made-up --hash=sha256:abab… line (reproduced locally).

The seed is now written without hash lines and line continuations. Version preferences stay the same, and every hash comes from the index. The committed locks on main are unchanged under the new script (-Check passes), and the made-up hash is now rejected. Locks from older Dependabot branches stop passing -Check until Dependabot rebases them. The npm path behaves the same way through its checksum check.

Verification

  • pytest tests/test_dependabot_vendor.py tests/test_supply_chain_policy.py tests/test_dependency_policy.py tests/test_toolchain_maintenance.py tests/test_documentation_surface.py tests/test_license_consistency.py tests/test_ci_change_scope.py: 250 passed, 1 skipped (newline file names are impossible on Windows).
    • In that run, the lock-generator tests that need pwsh ran under Windows PowerShell 5.1 through a shim.
    • Without the shim they skip locally. On ubuntu CI pwsh is present.
  • The new seed test fails against the old lock script.
  • lock_requirements.ps1 -Check with uv 0.13.5 (uv.toml): passes.
  • Replay of deps(python): bump flask-socketio from 5.7.0 to 5.7.2 in the python-minor-and-patch group across 1 directory #325:
    • Raw Dependabot commit 023db50: -Check fails, check-python-inputs passes.
    • Regenerating with UV_NO_BUILD=1 and an empty cache gives byte-identical locks to the manual fix 0fdae61.
  • Replay of deps(python): bump the python-minor-and-patch group with 3 updates #322 (166c599): the regenerated locks are identical to the locks merged in Update npm and Python dependency groups #324.
  • actionlint on the workflow (without shellcheck): clean.

Not verified: the workflow has not run on GitHub. The first real Dependabot pip PR after merge is the end-to-end test.

Dependabot's pip updater rewrites requirements*.txt without
scripts/lock_requirements.ps1. It drops the --require-hashes header and
the universal markers, so every Python update PR needed a manual rerun of
the lock script.

The Dependabot workflow gains a refresh-python-locks job next to the npm
vendor refresh. It reuses the same gate in scripts/dependabot_vendor.py,
now with a per-ecosystem profile: Dependabot actor and PR author, a
dependabot/pip/ branch in this repository, a failed "Check dependency
locks" step, and a change set limited to the three requirement inputs and
locks. The lock inputs must be plain version specifiers, the lock script,
gate and uv.toml must match the default branch, uv comes from uv.toml
without a cache, and UV_NO_BUILD keeps resolution free of package code.
Only the three lock files are staged and pushed. Write permissions move
from the workflow to each job.

The lock script now seeds uv with the committed versions only. uv kept
seeded hash lines without checking them, so Dependabot's duplicated
numpy hashes survived regeneration and -Check accepted an added hash.
@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-09T08:24:16.279474Z
We couldn’t complete this review. Comment @codex review to try again.
1160bcc PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@bifrost0x
bifrost0x merged commit e00b84f into main Nov 9, 2026
34 checks passed
@bifrost0x
bifrost0x deleted the ci/dependabot-python-locks branch November 9, 2026 08:53
@github-project-automation github-project-automation Bot moved this from Backlog to Done in WebSSH Roadmap Nov 9, 2026
@bifrost0x bifrost0x added this to the v3.1.0 milestone Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant