skip build scripts during uv sync - #3897
EmilyRagan wants to merge 8 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3897 +/- ##
==========================================
- Coverage 80.05% 80.04% -0.02%
==========================================
Files 901 901
Lines 68355 68355
Branches 2645 2698 +53
==========================================
- Hits 54722 54714 -8
- Misses 12961 12973 +12
+ Partials 672 668 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
805ba9e to
babf74b
Compare
babf74b to
2ab8e53
Compare
jmthomas
left a comment
There was a problem hiding this comment.
Mostly looks good. --locked plus --no-build at build and CI time is a solid tightening. One backwards-compatibility concern is inline on openc3/bin/uvinstall: runtime plugin installs with a non-default PYPI_URL.
Minor: the PR description says --no-build --no-binary-package openc3 was added to the uv scripts. uvinstall intentionally doesn't get it, since user plugins may depend on sdist-only packages (see the comment in docker-package-build.sh). Worth saying so in the description so it doesn't read as missed.
| # uploaded into this plugin venv via PythonPackageModel.install (which uses | ||
| # the additive `pipinstall`). | ||
| uv_offline_then_online sync --frozen --inexact --no-dev --no-install-project ${UV_ARGS} | ||
| uv_offline_then_online sync --locked --inexact --no-dev --no-install-project ${UV_ARGS} |
There was a problem hiding this comment.
Backwards-compat concern: --frozen → --locked here changes runtime behavior for already-built third-party plugins.
UV_ARGS includes -i <pypi_url> from PluginModel.build_pypi_args. uv treats a lockfile whose registry source doesn't match the configured index as stale under --locked, but not under --frozen:
# lock generated against pypi.org
uv sync --frozen --no-install-project -i https://test.pypi.org/simple # ok, installs locked six==1.17.0
uv sync --locked --no-install-project -i https://pypi.org/simple # ok
uv sync --locked --no-install-project -i https://test.pypi.org/simple
# The lockfile at `uv.lock` needs to be updated, but `--locked` was provided.
Impact:
- Default
PYPI_URL=https://pypi.org:-i https://pypi.org/simplematches the lock. No change. - Mirror or air-gapped
PYPI_URL(common in Enterprise): every plugin that ships auv.lockfails path 1. This includes the demo plugin and generator-template plugins. It then falls to path 2 (uv pip installfrompyproject.toml), which re-resolves from scratch and ignores the lock. The install still succeeds, so the only signal is theWarning:line. Versions can drift, and the baked offline cache (warmed for the locked versions) is more likely to miss. - Existing plugins whose
uv.lockis slightly out of date withpyproject.toml: they used to install exactly what was locked. Now they silently get an unpinned install.migrate_to_uv!goes through the same path.
This also works against the goal of the PR: path 2 has no --no-build and no pins, so a failing path 1 sends plugins down the less safe route.
Suggestion: keep --frozen here. These are gems the user already built, and drift detection is for our own repo and build, where --locked stays. Alternatively, try --locked, then retry with --frozen before falling back to path 2, and log loudly when the retry is needed.
There was a problem hiding this comment.
Good point, --frozen here. It's worse than drift, too: with a mirror -i, --locked re-resolves against the mirror, so the offline attempt fails on a warm cache (six was not found in the cache ... network was disabled). --frozen passes the same case. The baked plugin cache is never used, and every locked plugin falls to path 2, which has no lock and no --no-build.
There was a problem hiding this comment.
Yeah this is the big regression. It's so hard to test against a non-default PYPI_URL. Wonder if we can create better tests around this use case.
There was a problem hiding this comment.
I'm testing against a local Nexus repository in a separate PR, it's not as simple as pulling the image down thought
|
To Jason's point it would be worth making sure a plugin can be installed and uninstalled still. I verified that images build and the demo plugin is installed and runs, trying to install the TSDB plugin would be worth a run as well. I only verified the build-time piece and dropped the ball on the runtime pieces |
|
Working with Claude confirmed @jmthomas concern about Stock images are unaffected. The base image bakes in Images built against a mirror break. With docker exec -e UV_DEFAULT_INDEX=http://127.0.0.1:9/simple cosmos-openc3-cosmos-cmd-tlm-api-1 sh -c '
GEM=$(ls -d /gems/gems/openc3-cosmos-demo-* | head -1)
/openc3/bin/uvinstall zz_locked_test "$GEM" -i http://127.0.0.1:9/simple
sed "s/sync --locked/sync --frozen/g" /openc3/bin/uvinstall > /tmp/uvinstall_frozen
sh /tmp/uvinstall_frozen zz_frozen_test "$GEM" -i http://127.0.0.1:9/simple
rm -rf /gems/plugin_venvs/zz_* /tmp/uvinstall_frozen'
With the network up, the same mismatch fails the build-time syncs with Suggested changes:
|
|
I have a bunch of changes in #3891 to support building cosmos against a different pypi mirror |
uv rejects a lockfile whose recorded registry does not match the configured index under --locked, so plugin installs and image builds against a PyPI mirror re-resolve instead of honoring the lock. In uvinstall that drops every locked plugin into the unpinned pip fallback, which has neither the lock nor --no-build. Keep --locked in CI and the justfile, where resolution is always against pypi.org and lock drift is still caught before merge. --no-build and --no-binary-package openc3 are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
openc3_set_versions.rb rewrites the version in pyproject.toml, and uv.lock records that version too, so every release left the lockfile stale and failed `uv sync --locked` on main. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
@mcosgriff @jmthomas hmmm even after reverting some of the
|
jmthomas
left a comment
There was a problem hiding this comment.
I think this is correct now but not sure how it will interact with @mcosgriff changes in the other PR. Not sure about the warnings installing the tsdb-migration plugin. Are they present in the current release? Do they prevent the plugin from working? I assume we're expecting no warnings so we should understand that first before merging.
|
@jmthomas installing tsdb-migration in current release does not have the same error, so I think I have some more work to do. This is a good test case to go through |



What changed
Add
--no-build --no-binary-package openc3to theuvscripts, and split--lockedvs--frozenby where the sync runs:--lockedin CI and the justfile--frozeninopenc3/Dockerfile,docker-package-build.sh,verify-uv-cache.sh,openc3/bin/uvinstall, and the plugin docsWhy it changed
Valid SonarQube findings about installation scripts being potentially dangerous to allow to run.
The
--no-buildpart is the fix for that finding. The--frozen->--lockedpart started out global, but review found it regresses any stack whosePYPI_URLis not pypi.org, so it is now scoped to CI.Testing strategy
CI, plus a manual check of
uvinstallin a running stack (cmd-tlm-api, uv 0.12.5, demo plugin) withUV_DEFAULT_INDEXand-ipointed at an unreachable mirror.Review notes
uv sync --lockederrors ifuv.lockdoes not matchpyproject.toml, where--frozenalways installs what is pinned inuv.lockregardless.--lockedis what keeps us from changing a package and forgetting to commit the lock update, so it stays on every sync that resolves against pypi.org.--locked. That makes--lockedwrong wherever the index can be a mirror:uvinstallpasses-i ${PYPI_URL}/simplefromPluginModel.build_pypi_args. Under--locked, every plugin shipping auv.lockfalls through to path 2 (uv pip install), which has neither the lock nor--no-build- the opposite of this PR's goal. The failure is also not clean: rejecting the lock means uv re-resolves, which needs the index's package listings, and the baked offline cache holds wheels only, so the install fails on a warm cache. The only signal is the existingWarning:line.openc3/Dockerfilesyncs, thedocker-package-build.shwarm-up andverify-uv-cache.shhit the same mismatch throughUV_DEFAULT_INDEX, so a mirror build breaks at image build time. These can move to--lockedonce the relock in fix(build): relock uv against PYPI_URL so air-gapped builds resolve #3891 lands, since each sync will then follow a relock against the same index.uvinstalldeliberately does not get--no-build. User plugins are allowed to depend on sdist-only packages; see the comment indocker-package-build.sh.[tool.uv] build-constraint-dependenciespins the build backend, but those pins are not hash-verified the wayuv.lockentries are, so the comment says it narrows the gap rather than closes it.latestas default image tag to pull in Dockerfile.Follow-ups (not in this PR)
PYPI_URL, which is why this regression got as far as it did. Worth a job that stands up a local index and runs a plugin install against it.uvinstall's fallback to path 2 is silent apart from a warning. Arguably it should hard-fail when the plugin ships auv.lock.