You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 1e976bd
Browse filesBrowse the repository at this point in the historyBrowse files
Pull once more from the next registry when a pull limit runs out
Docker Hub does not count the HEAD of pick-base-image. So its pull limit
can run out between the pick and the pull, and the leg failed with 429.
- scripts/build-port makes the builder and runs the push and the export
of each leg. When a pull fails with "toomanyrequests" or "429 Too Many
Requests", the step runs once more with the same digest from the next
registry. In a build, the 429 line must name the repository of the base
image. Any other failure stops the step at once.
- pick-base-image --after VALUE asks only the registries after the one of
VALUE.
- build-deploy.yml calls the script. The two builds start from
actions/github-script, because a run: step gets no ACTIONS_RUNTIME_TOKEN
for the gha cache. The export and the digest record take the base image
that the push used. The buildkit pin keeps its "# tag:" marker.
- tests/static/36-build-port.sh runs the script against a stand-in docker.
Tests 35, 64 and 68 follow the new steps. docs/tests.md records each
fault that each new assertion was seen to fail on.
Copy file name to clipboardExpand all lines: docs/design.md
+13-9Lines changed: 13 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,7 +23,7 @@ Each rule below keeps one of these true for one way it could fail.
23
23
| rule | failure it prevents | where |
24
24
| --- | --- | --- |
25
25
| The bootstrap stage starts from the official Arch Linux image, pinned by digest. No `FROM` names an image that this repository publishes. | A build from its own last output keeps each defect of that output, and nobody can rebuild it without trust in this project first. |`Dockerfile`, `tests/static/10-bootstrap-not-circular.sh`|
26
-
| The build asks for the digest of the base image and of the buildkit image from `docker.io`, then `mirror.gcr.io`. For the base image, a Docker official image, `public.ecr.aws` is third. The same digest is the same bytes. A registry that answers with `ratelimit-remaining: 0` is passed over: its pull limit is used up. ⚠ The limit can run out between the probe and the pull. | One registry outage or one used-up pull limit stops every build. |`scripts/pick-base-image`|
26
+
| The build asks for the digest of the base image and of the buildkit image from `docker.io`, then `mirror.gcr.io`. For the base image, a Docker official image, `public.ecr.aws` is third. The same digest is the same bytes. A registry that answers with `ratelimit-remaining: 0` is passed over: its pull limit is used up. A pull that fails with HTTP 429 or `toomanyrequests` runs once more, from the next registry at the same digest. ⚠ A second failure stops the leg, and the next run picks again. | One registry outage or one used-up pull limit stops every build. Docker Hub does not count the HEAD of the pick, so its limit can run out before the pull. |`scripts/pick-base-image`, `scripts/build-port`, `tests/static/36-build-port.sh`|
27
27
| Each shipped `pacman.conf` sets `SigLevel = Required DatabaseOptional`. No SigLevel is `Never`. | A forged or unsigned package installs. Most ports do not sign their databases, so `DatabaseOptional` is necessary. |`rootfs/*/etc/pacman.conf`, `tests/static/30-signature-checking-on.sh`|
28
28
| A port whose keys are not in `archlinux-keyring` has a keyring pin: the package, its sha256, each trusted fingerprint with its expiry. The package is in the tree. | The port deletes a superseded keyring package from its mirrors, and the build cannot get the trust root. |`bootstrap/keyrings/*.pin`, `bootstrap/any/usr/local/bin/install-port-keyring`, `tests/static/88-keyring-pins-vendored.sh`|
29
29
| After populate, each pinned key that is not expired must be fully valid. | A key at marginal validity signs nothing that pacman accepts. The keyring installs, and the transaction commits nothing. |`install-port-keyring`|
@@ -135,18 +135,22 @@ The steps of a build leg, in order:
135
135
136
136
1.`scripts/binfmt-register ARCH`: the emulator, with the `F` flag.
137
137
2.`scripts/pick-base-image`: the registry for the base image digest.
138
-
3.`scripts/pick-base-image --image "$BUILDKIT_IMAGE"`: the registry for the buildkit image of the builder.
139
-
4.`scripts/gen-mirrorlist ARCH`: exit 0 writes the lists in the checkout of the runner. Any other exit keeps the committed lists.
140
-
5.`scripts/pick-transport ARCH`: `worker` or `oci`.
141
-
6.`scripts/check-mirror-lag ARCH`: each `drop LIST SERVER` line becomes a `LIST=SERVER` pair of `SKIP_SERVERS`.
142
-
7.`scripts/resolve-anchor ARCH`. ⛔ A port that no server answers for fails here: a red leg, never a green skip.
143
-
8.`scripts/check-anchor-floor ARCH ANCHOR`. An anchor that no tag can carry fails here too.
144
-
9.`docker buildx build`, pushed by digest to the target, with provenance (`mode=max`) and an SBOM. The cache is `type=gha` with `ignore-error=true`.
145
-
10.The`dbsnapshot` target, exported to a directory.
138
+
3.`scripts/gen-mirrorlist ARCH`: exit 0 writes the lists in the checkout of the runner. Any other exit keeps the committed lists.
139
+
4.`scripts/pick-transport ARCH`: `worker` or `oci`.
140
+
5.`scripts/check-mirror-lag ARCH`: each `drop LIST SERVER` line becomes a `LIST=SERVER` pair of `SKIP_SERVERS`.
141
+
6.`scripts/resolve-anchor ARCH`. ⛔ A port that no server answers for fails here: a red leg, never a green skip.
142
+
7.`scripts/check-anchor-floor ARCH ANCHOR`. An anchor that no tag can carry fails here too.
143
+
8.`scripts/build-port builder`: the registry for the buildkit image (`pick-base-image --image "$BUILDKIT_IMAGE"`), its pull, and the builder that runs it.
144
+
9.`scripts/build-port push`: `docker buildx build`, pushed by digest to the target, with provenance (`mode=max`) and an SBOM. The cache is `type=gha` with `ignore-error=true`.
145
+
10.`scripts/build-port snapshot`: the`dbsnapshot` target, exported to a directory.
146
146
11.`scripts/gen-evidence`, with `DB_SNAPSHOT` and `EXPECT_ANCHOR`.
147
147
12.`tests/run.sh image` against the pushed digest.
148
148
13. The artifacts `evidence-<arch>` (90 days) and `digest-<arch>` (7 days).
149
149
150
+
⭐ Steps 8 to 10 run once more when a pull fails on a used-up pull limit. The second try takes the same digest from the next registry: `pick-base-image --after`. A failure counts when the output has a line with `toomanyrequests` or `429 Too Many Requests`. In a build, that line must also name the repository of the base image, because a server of a mirror list can answer 429 too. Any other failure stops the step at once. The push names the base image it used, and the export and the digest record take that value.
151
+
152
+
Steps 9 and 10 start the script from `actions/github-script`. A `run:` step gets no `ACTIONS_RUNTIME_TOKEN`, and the `type=gha` cache needs it.
153
+
150
154
The build args of the Dockerfile: `BASE_IMAGE`, `TRANSPORT` (default `worker`), `SKIP_SERVERS` (default empty), `IMAGE_VERSION`, `SOURCE_COMMIT` and `BUILD_DATE`. Its stages: `bootstrap`, `dbsnapshot` and the final unnamed stage.
151
155
152
156
`digest-<arch>.json` has the keys `arch`, `platform`, `digest`, `anchor`, `transport`, `skip_servers`, `base_image`, `mirrorlist` (`generated` or `committed`) and `built_at`.
Copy file name to clipboardExpand all lines: docs/measurements.md
+5Lines changed: 5 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -169,6 +169,11 @@ Measured facts only. Each row gives its date and its source: the command, the ru
169
169
| image pins: 5, all current; tag push ages `archlinux:latest` 4 days, `registry:3` 17 days, `busybox:latest` 23 days | 2026-10-16 |`scripts/check-image-pins --max-age 14`|
170
170
|`pick-base-image`: docker.io serves the digest of the base image, and of the buildkit image | 2026-10-16 |`scripts/pick-base-image`, and `scripts/pick-base-image --image` with the buildkit pin of `build-deploy.yml`|
171
171
|`moby/buildkit:buildx-stable-1` at `sha256:cec9f139f45e93c5c69c60f8b07cfad9f43f4ef6b6a6cd917527fea5ff2e3dea`: docker.io and mirror.gcr.io answer 200 with that digest; public.ecr.aws (`docker/moby/buildkit`) and ghcr.io answer 404 | 2026-10-16 |`PROG=m; . scripts/lib.sh; reg_head REGISTRY REPO DIGEST`|
172
+
| Docker Hub counts a GET of a manifest as a pull, and not a HEAD. Over the limit it answers HTTP 429: "You have reached your pull rate limit" | documented | docs.docker.com, Docker Hub usage and limits |
173
+
| a registry over its request limit answers HTTP 429 with the error code `TOOMANYREQUESTS`; the docker client prints it as `toomanyrequests:`| documented | distribution/distribution `registry/api/errcode/register.go`|
174
+
| the leg Build ppc64le: `setup-buildx-action` v4.4.1 ran `docker pull` of the buildkit pin, then `docker buildx create --driver docker-container --driver-opt image=<pin> --use`, then `docker buildx inspect --bootstrap`| 2026-10-16 | run 38930740850, the log of the job |
175
+
| the same leg: `build-push-action` v7.4.0 ran `docker buildx build` with the 6 build args, the gha cache, `--attest type=provenance,mode=max,builder-id=<run URL>/attempts/1` and `--metadata-file`. The push added `--attest type=sbom,disabled=false`; buildx v0.38.0, BuildKit v0.33.1 | 2026-10-16 | run 38930740850, the log of the job |
176
+
| the runner gives `ACTIONS_RUNTIME_TOKEN`, `ACTIONS_RESULTS_URL` and `ACTIONS_CACHE_SERVICE_V2` to a JavaScript action, and not to a `run:` step; buildx reads them for a `type=gha` cache | documented | actions/runner `NodeScriptActionHandler.cs`; docker/buildx `build/opt.go`|
172
177
| pacman 7.1.0, a database that no server gives, through curl: `failed retrieving file` lines, then `failed to synchronize all databases (download library error)`| 2026-10-16 |`pacman -Sy` in the test container with `--network none`, against `file://` and `http://127.0.0.1:9` servers |
173
178
| the same through an XferCommand: only `failed to synchronize all databases (error invoking external downloader)`| 2026-10-16 | the same, with an `XferCommand`|
174
179
| a package that no server gives: through curl `warning: failed to retrieve some files`, then `failed to commit transaction (download library error)`; through an XferCommand `failed to commit transaction (error invoking external downloader)`| 2026-10-16 |`pacman -S`, the same way |
Copy file name to clipboardExpand all lines: docs/operations.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -261,6 +261,7 @@ The workflows, the other scripts and the tests read the port table, and need no
261
261
| the GHCR mirror is behind | the build drops it for that run (`drop LIST SERVER` in the log). The image still ships it. Nothing to do |
262
262
| the Worker does not answer |`pick-transport` gives `oci`, and the build stage reads GHCR through pacman-oci. Nothing to do |
263
263
| the base image registry does not answer |`pick-base-image` takes the same digest from `mirror.gcr.io`, or for the base image from `public.ecr.aws`. The buildkit image falls back to `mirror.gcr.io` too. Nothing to do |
264
+
| a pull meets a used-up pull limit (HTTP 429, `toomanyrequests`) |`build-port` runs the step once more with the same digest from the next registry. The log says `met a used-up pull limit`. When the second try fails too, the port waits for the next run. Nothing to do |
264
265
|`publish-hub` is open | the next run copies each missing tag. A failure there never touches GHCR. Check the Docker Hub secrets |
265
266
|`registry-drift` is open | the job `hub` of the next build copies the missing and moved tags. A build with nothing to build runs `hub` too |
266
267
|`status-down` or `pages-stale` is open | read the run of `status.yml` that the issue names, and start it again: `gh workflow run status.yml --ref main`|
0 commit comments