Skip to content

fix(build): relock uv against PYPI_URL so air-gapped builds resolve - #3891

Draft
mcosgriff wants to merge 13 commits into
mainfrom
2867-pypi_url-not-passed-to-uv-during-docker-image-builds-breaking-air-gappedproxy-environments
Draft

mcosgriff wants to merge 13 commits into
mainfrom
2867-pypi_url-not-passed-to-uv-during-docker-image-builds-breaking-air-gappedproxy-environments

Conversation

@mcosgriff

@mcosgriff mcosgriff commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • uv sync --frozen fetches the pythonhosted.org URLs pinned in uv.lock, so a non-default PYPI_URL never reached the mirror.
  • New uv-mirror-relock (from both base images) re-resolves the lock against ${PYPI_URL}/simple with --no-config --no-sources, so the mirror is the only index. It does nothing at the default pypi.org.
  • Runs before both syncs in openc3/Dockerfile, and before rake build in docker-package-build.sh so plugin gems ship mirror locks.
  • A PYPI_URL_OVERRIDE build arg keeps the mirror when an image is rebuilt on its own against an older base image.
  • Docs: the mirror must serve every dependency, including dev groups, and a plugin that needs a private index requires the mirror to proxy it. Versions can drift from the committed lock.
  • Overlaps with skip build scripts during uv sync #3897 on --no-build in openc3/Dockerfile. After skip build scripts during uv sync #3897 merges, the post-relock syncs can move to --locked.
  • Build time only. Runtime plugin installs are refactor(uv): replace deprecated uv index options and seed a plugin pyproject template #3896, and the Enterprise Dockerfiles need a companion PR.
  • Installs uv from the mirror too: pip install uv in both base images ignored PYPI_URL. UV_INSECURE_HOST trusts a plain-HTTP mirror.

Closes #2867

Test plan

openc3-ruby/uv_mirror_relock.sh, run in openc3/python and the demo plugin against a local PEP 503 mirror:

  • PYPI_URL unset or https://pypi.org - prints the skip, lock untouched
  • PYPI_URL at the mirror - no pythonhosted.org URLs left, every download URL points at the mirror
  • A plugin whose [tool.uv].index or [tool.uv.sources] names another index resolves against the mirror only

scripts/linux/test_pypi_mirror.sh (runs ./openc3.sh build against the mirror with pypi.org and files.pythonhosted.org blackholed):

  • main fails; this branch builds, installing uv from the mirror and relocking both locks
  • Base image and demo plugin gem ship mirror-only locks, and verify-uv-cache.sh passes offline
  • build-ubi mode

Other:

  • docker buildx build --call=check and shellcheck clean
  • Full ./openc3.sh build against Nexus, including init plugins
  • Enterprise build against the same proxy

- Add openc3-ruby/uv_mirror_relock.sh, shipped as uv-mirror-relock,
  which re-resolves uv.lock against PYPI_URL and is a no-op at the
  default pypi.org
- Run it before both uv sync --frozen steps in openc3/Dockerfile,
  since UV_DEFAULT_INDEX only steers resolution while a frozen sync
  fetches the absolute pythonhosted.org URLs the lockfile pins
- Relock before rake build in docker-package-build.sh so the lock
  packaged into each plugin gem matches the warmed wheel cache
- Document the re-resolve and the host-side uv lock --default-index
  workflow in .env, INSTALL.md, developing.md and environment.md

Refs #2867

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
@codecov

codecov Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.86%. Comparing base (09dca0d) to head (ed03fd3).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3891      +/-   ##
==========================================
+ Coverage   79.85%   79.86%   +0.01%     
==========================================
  Files         901      901              
  Lines       68154    68251      +97     
  Branches     2698     2698              
==========================================
+ Hits        54426    54512      +86     
- Misses      13057    13065       +8     
- Partials      671      674       +3     
Flag Coverage Δ
frontend 66.68% <ø> (-0.02%) ⬇️
python 79.81% <ø> (-0.01%) ⬇️
ruby-api 82.56% <ø> (-0.03%) ⬇️
ruby-backend 85.42% <ø> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

  Pass the mirror explicitly to base and init images, report skipped relocks, and prevent source builds during dependency resolution.
…-image-builds-breaking-air-gappedproxy-environments
…sed-to-uv-during-docker-image-builds-breaking-air-gappedproxy-environments
@mcosgriff

Copy link
Copy Markdown
Contributor Author

uv_mirror_relock.sh relocks with uv lock --default-index "${PYPI_URL}/simple", which does not make the mirror authoritative. uv searches an index declared in the project's own [tool.uv].index table before the default index, and a [tool.uv].sources pin is project metadata that survives --no-config. docker-package-build.sh runs the relock per plugin after cd ${FOLDER_NAME}, so a plugin that declares its author's index resolves against that index during the relock and the air-gapped build fails there.

Against uv 0.12.16, with both indexes on unroutable .invalid hosts so the error names whichever index uv chose:

plugin declares --default-index + --no-config + --no-sources both
named [tool.uv].index plugin's mirror plugin's mirror
[tool.uv].sources pin plugin's plugin's mirror mirror

Neither flag is sufficient alone:

-uv lock --default-index "${PYPI_URL}/simple" "$@"
+uv lock --no-config --no-sources --default-index "${PYPI_URL}/simple" "$@"

This does not change the results in your test plan. Neither openc3/python/pyproject.toml nor the demo plugin has a [tool.uv] table, so both flags are no-ops for the projects this repo relocks.

One consequence for the description: once the mirror is authoritative, a plugin depending on a package published only to its author's private index fails to build rather than silently fetching from that index, so the mirror has to proxy any such index.

#3896 makes the same change on the runtime plugin install path, in PypiUrl.build_args and openc3/bin/{uvinstall,pipinstall}.

Add --no-config --no-sources to the uv lock invocation. --default-index
does not win on its own: uv searches a named index declared in the
project's own [tool.uv].index table first, and a [tool.uv].sources pin is
project metadata that survives --no-config. Without both flags a plugin
configuring its author's index relocks against that index, and the
air-gapped build fails at the point this script exists to prevent.

Refs #2867

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
…sed-to-uv-during-docker-image-builds-breaking-air-gappedproxy-environments
- Pass --index-url "${PYPI_URL}/simple" to the pip install of uv in
  openc3-ruby Dockerfile and Dockerfile-ubi so air-gapped builds stop
  reaching for pypi.org
- Add UV_INSECURE_HOST build arg that maps to pip --trusted-host,
  letting a plain-HTTP mirror be trusted
- Thread UV_INSECURE_HOST through .env, compose-build.yaml and
  openc3_build_ubi.sh

Refs #2867

Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com
Refs #2867

Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com
- Pass UV_INSECURE_HOST to pip through PIP_TRUSTED_HOST instead of an
  unquoted ${UV_INSECURE_HOST:+--trusted-host ...} expansion
- Add --only-binary :all: so a mirror-served sdist never runs a setup
  script
- Apply both to Dockerfile and Dockerfile-ubi in step

Refs #2867

Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com
@sonarqubecloud

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PYPI_URL not passed to uv during Docker image builds, breaking air-gapped/proxy environments

1 participant