Skip to content

ci: Pin CPU-only torch through the uv torch-backend setting - #6825

Open
patelchaitany wants to merge 2 commits into
feast-dev:masterfrom
patelchaitany:ci/pytorch-cpu-index
Open

ci: Pin CPU-only torch through the uv torch-backend setting#6825
patelchaitany wants to merge 2 commits into
feast-dev:masterfrom
patelchaitany:ci/pytorch-cpu-index

Conversation

@patelchaitany

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Every CI job currently fails in the install step:

× Failed to download `torchvision==0.28.0+cpu`
╰─▶ Hash mismatch for `torchvision==0.28.0+cpu`

The CI requirement files are compiled without a torch backend, so their hashes describe PyPI wheels, but install-python-dependencies-ci synced them on Linux with --torch-backend cpu, which serves torch==2.13.0+cpu and torchvision==0.28.0+cpu from download.pytorch.org. Those are different builds with different hashes.

The divergence has existed since #6588 added the flag; what changed is verification. uv 0.12.11 began applying hashes from public-version pins to matching local versions when no exact local-version hash is provided, so the PyPI hashes recorded for torchvision==0.28.0 are now checked against the +cpu wheel. Earlier releases found no hash for the local version and skipped the check. uv is unpinned in the workflows, so CI moved from 0.12.7 to 0.12.11 and every job began failing.

Approach

Set torch-backend in [tool.uv] rather than passing it on the command line. The setting is read by every uv pip command, so compile and sync can no longer disagree about where torch comes from, and the install-time override is then redundant and comes out.

An explicit index with [tool.uv.sources] does not serve here. The pip interface applies sources only at compile time; uv pip sync ignores them, so the +cpu builds would not be found at all:

× No solution found ... Because there is no version of
  torch{sys_platform == 'linux'}==2.14.0+cpu ...

The CI locks are compiled with --universal so one file can serve both the Linux and macOS runners, pinning each variant with its own hashes:

torch==2.14.0           ; sys_platform == 'darwin'
torch==2.14.0+cpu       ; sys_platform != 'darwin'
torchvision==0.29.0     ; sys_platform == 'darwin'
torchvision==0.29.0+cpu ; sys_platform != 'darwin'

Without --universal the lock pins an unmarked torch==2.14.0 carrying PyPI hashes. Linux still resolves that to 2.14.0+cpu, because ==2.14.0 matches the local version under PEP 440, and the download then fails verification — the original breakage in a different guise.

Finally, pixi raises its uv floor. The environment was locked to uv 0.6.14, which does not know the field: it emits unknown field torch-backend as a warning during settings discovery, exits 0, and resolves the CUDA build anyway. Compiling under it produced plain torch==2.14.0 and fifteen nvidia-* pins, so the setting was silently inert. The field is rejected through 0.9.13 and accepted from 0.9.18; pixi now provides uv 0.12.12, matching the version CI installs.

No nvidia-* package remains in any CI requirements file, which is what #6588 set out to achieve.

Which issue(s) this PR fixes:

None filed; this repairs CI on master.

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests

Verified locally, on a macOS host, so the full Linux install was not exercised end to end:

  • uv pip sync --dry-run --python-platform x86_64-unknown-linux-gnu against py3.10-ci-requirements.txt resolves all 440 packages and selects torch==2.14.0+cpu and torchvision==0.29.0+cpu.
  • A real hash-checked install of the CPU torchvision wheel for the Linux platform downloads from download.pytorch.org and verifies clean — the specific artifact that was failing.
  • The same install against a non-universal lock reproduces the failure, confirming --universal is load-bearing rather than cosmetic.

CI on this PR is the actual proof.

The CI requirement files are compiled without a torch backend, so their
hashes describe PyPI wheels, but Makefile:110 syncs them on Linux with
--torch-backend cpu, which serves torch==2.13.0+cpu and
torchvision==0.28.0+cpu from download.pytorch.org. Those are different
builds with different hashes:

    Failed to download `torchvision==0.28.0+cpu`
    Hash mismatch for `torchvision==0.28.0+cpu`

The divergence has existed since feast-dev#6588 added the flag. Both the old and
the new uv redirect to the CPU index; what changed is verification. uv
0.12.11 began to "apply hashes from public-version pins to matching
local versions when no exact local-version hash is provided", so the
PyPI hashes recorded for torchvision==0.28.0 are now checked against the
+cpu wheel. Earlier releases found no hash for the local version and
skipped the check. uv is unpinned in the workflows, so CI moved from
0.12.7 to 0.12.11 and every job began failing in the install step.

Set torch-backend in [tool.uv] rather than passing it on the command
line. The setting is read by every uv pip command, so compile and sync
can no longer disagree about where torch comes from. An explicit index
with [tool.uv.sources] would not serve, because the pip interface
applies sources only at compile time and sync would then fail to find
the +cpu builds at all.

Compile the CI locks universally so that one file can serve both the
Linux and macOS runners, pinning each variant with its own hashes:

    torch==2.14.0+cpu ; sys_platform != 'darwin'
    torch==2.14.0 ; sys_platform == 'darwin'

The install-time override is then redundant and comes out, and pixi
raises its uv floor to a release that understands the setting.

Signed-off-by: Chaitany Patel <patelchaitany93@gmail.com>
@patelchaitany
patelchaitany requested a review from a team as a code owner September 10, 2026 10:38
@codecov-commenter

codecov-commenter commented Sep 10, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 47.08%. Comparing base (81e1546) to head (36c6951).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #6825   +/-   ##
=======================================
  Coverage   47.08%   47.08%           
=======================================
  Files         419      419           
  Lines       51877    51877           
  Branches     7525     7525           
=======================================
  Hits        24428    24428           
  Misses      25700    25700           
  Partials     1749     1749           
Flag Coverage Δ *Carryforward flag
go-feature-server 30.58% <ø> (ø)
python-unit 48.39% <ø> (ø) Carriedforward from dd7b805

*This pull request uses carry forward flags. Click here to find out more.


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 81e1546...36c6951. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@patelchaitany
patelchaitany force-pushed the ci/pytorch-cpu-index branch 2 times, most recently from 36c6951 to 2ec44b1 Compare September 10, 2026 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants