From e42406e04e8cc8d0e20aab5a3ff5a9921d3e644b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 22 Sep 2026 11:15:13 +0000 Subject: [PATCH 1/4] Cross-build from bullseye main only: the -security pool is gone The staging deploy for #194's merge (Deploy main run 35719911932) failed in build.sh's Docker stage at apt-get install, before any compilation: thirteen packages apt chose from bullseye-security -- perl, libcurl4, libexpat1, libicu67, libxml2, dpkg-dev and friends -- answered 404 from the pool. The suite's index still advertises those versions (perl 5.32.1-4+deb11u5), but bullseye left LTS on 2026-08-31 and its security pool has since been pruned underneath the index, so every apt-get that consults it fails the same way. It is not this change's doing: the same stage passed on 2026-09-03 and the merge touched no Dockerfile, Cargo or apt input. Staying on bullseye is deliberate and documented (glibc 2.31 so the binary runs on Amazon Linux 2023's 2.34; bookworm's 2.36 would not), so the fix is to stop consulting the dead suite: the stage now writes a sources.list with the bullseye main suite alone before apt-get update. main is intact and carries every package this stage installs (verified against its current arm64 index and pool: perl 5.32.1-4+deb11u3, libcurl4 7.74.0-1.3+deb11u13, cmake 3.18.4-2+deb11u1, clang 1:11.0-51+nmu5, ...). The comment records the next step for when deb.debian.org drops bullseye entirely: archive.debian.org already mirrors the main suite. The Docker daemon is not available where this was prepared, so the stage was not rebuilt locally; the next push to main exercises it end to end in the staging deploy, which is exactly the run that failed. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01TGhwpdTbUqQbR5AiiN8Ljw --- deploy/native/build.sh | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/deploy/native/build.sh b/deploy/native/build.sh index fcf96d14..9f3264e4 100755 --- a/deploy/native/build.sh +++ b/deploy/native/build.sh @@ -50,7 +50,15 @@ ARG GIT_SHA=unknown ARG BUILD_TIME ENV GIT_SHA=${GIT_SHA} ENV BUILD_TIME=${BUILD_TIME} -RUN apt-get update && apt-get install -y --no-install-recommends \ +# bullseye left LTS on 2026-08-31 and its -security pool has since been pruned: +# the suite's index still lists package versions whose .deb files answer 404, so +# any apt-get that consults bullseye-security fails. The main suite is intact +# and carries everything this stage needs, so resolve from it alone. (When +# deb.debian.org drops bullseye entirely, point this at archive.debian.org, +# which already mirrors the main suite.) +RUN printf 'deb http://deb.debian.org/debian bullseye main\n' > /etc/apt/sources.list \ + && rm -rf /etc/apt/sources.list.d/* \ + && apt-get update && apt-get install -y --no-install-recommends \ build-essential cmake clang libclang-dev perl pkg-config ca-certificates \ && rm -rf /var/lib/apt/lists/* COPY Cargo.toml Cargo.lock ./ From 829622b5396df2594f813071b0a77b6861ffb52f Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 22 Sep 2026 12:23:40 +0000 Subject: [PATCH 2/4] Install the cross-build's packages from a bullseye snapshot, not main alone Copilot's review of the previous commit was right: restricting apt to bullseye main is unsatisfiable against this image. rust:1-slim-bullseye (built 2026-08-25) has its base packages at the final security versions -- perl-base 5.32.1-4+deb11u5, libc6 2.31-13+deb11u14, dpkg 1.20.14 and 25 more -- while main carries perl 5.32.1-4+deb11u3, which depends on perl-base (= 5.32.1-4+deb11u3), and libc6-dev 2.31-13+deb11u11, which depends on libc6 (= 2.31-13+deb11u11). apt will not downgrade the installed perl-base or libc6 to satisfy those, so the install fails. Verified by reading /var/lib/dpkg/status out of the image's base layer from the registry and comparing it with main's arm64 index. Install from snapshot.debian.org at 2026-08-30 instead, for both suites: the last state in which bullseye and bullseye-security were complete, after the image was built. Its security index carries exactly the versions the image has (perl-base +deb11u5, libc6 and libc6-dev +deb11u14, dpkg and dpkg-dev 1.20.14) and all thirteen packages that 404 on deb.debian.org (perl, libcurl4 7.74.0-1.3+deb11u16, libexpat1, libicu67, libxml2, libglib2.0-0, ...), and its pool serves them. Both suites are frozen, so the pin is the permanent final state, reproducible, with nothing to downgrade. check-valid-until=no because the snapshot's Release files expired on 2026-09-05; Acquire::Retries=3 for snapshot's throttling. Still not rebuilt locally (no Docker daemon here); the staging deploy on merge remains the end-to-end proof. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01TGhwpdTbUqQbR5AiiN8Ljw --- deploy/native/build.sh | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/deploy/native/build.sh b/deploy/native/build.sh index 9f3264e4..ce522d29 100755 --- a/deploy/native/build.sh +++ b/deploy/native/build.sh @@ -50,15 +50,25 @@ ARG GIT_SHA=unknown ARG BUILD_TIME ENV GIT_SHA=${GIT_SHA} ENV BUILD_TIME=${BUILD_TIME} -# bullseye left LTS on 2026-08-31 and its -security pool has since been pruned: -# the suite's index still lists package versions whose .deb files answer 404, so -# any apt-get that consults bullseye-security fails. The main suite is intact -# and carries everything this stage needs, so resolve from it alone. (When -# deb.debian.org drops bullseye entirely, point this at archive.debian.org, -# which already mirrors the main suite.) -RUN printf 'deb http://deb.debian.org/debian bullseye main\n' > /etc/apt/sources.list \ +# bullseye left LTS on 2026-08-31 and deb.debian.org has since pruned its +# -security pool: the suite's index still lists versions whose .deb files answer +# 404, so any apt-get that consults the live bullseye-security fails. Resolving +# from main alone does not work either: this image's preinstalled base packages +# are at the FINAL security versions (perl-base 5.32.1-4+deb11u5, libc6 +# 2.31-13+deb11u14, ...), and main's perl / libc6-dev depend on exactly the +# older main versions of perl-base / libc6, which apt will not downgrade to. +# So install from snapshot.debian.org at the last moment both suites were +# complete (2026-08-30; the image was built 2026-08-25). Both suites are frozen, +# so that snapshot is the permanent final state of bullseye and the versions +# match what the image already has: nothing is downgraded and the build is +# reproducible. The snapshot's Release files have passed their Valid-Until, +# hence check-valid-until=no; Acquire::Retries absorbs snapshot's occasional +# throttling. (The durable fix is a base image whose archive is alive and whose +# glibc still fits the host -- amazonlinux:2023 -- tracked separately.) +RUN printf 'deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/20260830T000000Z bullseye main\ndeb [check-valid-until=no] http://snapshot.debian.org/archive/debian-security/20260830T000000Z bullseye-security main\n' > /etc/apt/sources.list \ && rm -rf /etc/apt/sources.list.d/* \ - && apt-get update && apt-get install -y --no-install-recommends \ + && apt-get -o Acquire::Retries=3 update \ + && apt-get -o Acquire::Retries=3 install -y --no-install-recommends \ build-essential cmake clang libclang-dev perl pkg-config ca-certificates \ && rm -rf /var/lib/apt/lists/* COPY Cargo.toml Cargo.lock ./ From 69506cb8ca3d869761cbf6974c58dec0f6dcc25b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 22 Sep 2026 12:26:49 +0000 Subject: [PATCH 3/4] Narrow the reproducibility claim in build.sh's apt comment The previous commit's comment said the snapshot made "the build reproducible". It does not: rust:1-slim-bullseye is a mutable tag and BUILD_TIME is stamped at build time, so the binary and its metadata still vary from build to build. What the dated snapshot fixes is apt's inputs, so the comment now says that package resolution is reproducible and names the two things that are not. Review nit from Copilot on 829622b. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01TGhwpdTbUqQbR5AiiN8Ljw --- deploy/native/build.sh | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/deploy/native/build.sh b/deploy/native/build.sh index ce522d29..070ac507 100755 --- a/deploy/native/build.sh +++ b/deploy/native/build.sh @@ -60,8 +60,10 @@ ENV BUILD_TIME=${BUILD_TIME} # So install from snapshot.debian.org at the last moment both suites were # complete (2026-08-30; the image was built 2026-08-25). Both suites are frozen, # so that snapshot is the permanent final state of bullseye and the versions -# match what the image already has: nothing is downgraded and the build is -# reproducible. The snapshot's Release files have passed their Valid-Until, +# match what the image already has: nothing is downgraded and apt's inputs are +# fixed. (Only package resolution is reproducible: the rust:1-slim-bullseye tag +# is mutable and BUILD_TIME is stamped per build.) The snapshot's Release files +# have passed their Valid-Until, # hence check-valid-until=no; Acquire::Retries absorbs snapshot's occasional # throttling. (The durable fix is a base image whose archive is alive and whose # glibc still fits the host -- amazonlinux:2023 -- tracked separately.) From 26ff998bb94990b2ac2bb6cfed18f45483f40de5 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 22 Sep 2026 12:30:43 +0000 Subject: [PATCH 4/4] Fetch the bullseye snapshots over https With check-valid-until=no, apt no longer rejects an index past its Valid-Until, so its remaining defence against a network attacker replaying an older, still validly signed snapshot index -- and thereby an older package set than the timestamp names -- is the transport. Fetch both snapshot lines over https so the dated path is authenticated in transit. The rust:1-slim-bullseye image installs ca-certificates in its own layer (alongside gcc, libc6-dev and wget), so https works before this stage installs anything; snapshot.debian.org serves https end to end, redirecting to same-origin content-hash paths. Review finding from Copilot on 69506cb. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01TGhwpdTbUqQbR5AiiN8Ljw --- deploy/native/build.sh | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/deploy/native/build.sh b/deploy/native/build.sh index 070ac507..39c3468e 100755 --- a/deploy/native/build.sh +++ b/deploy/native/build.sh @@ -64,10 +64,13 @@ ENV BUILD_TIME=${BUILD_TIME} # fixed. (Only package resolution is reproducible: the rust:1-slim-bullseye tag # is mutable and BUILD_TIME is stamped per build.) The snapshot's Release files # have passed their Valid-Until, -# hence check-valid-until=no; Acquire::Retries absorbs snapshot's occasional +# hence check-valid-until=no. With that check off, apt's only defence against a +# replayed older (still validly signed) index is the transport, so both lines +# use https; the rust image preinstalls ca-certificates, so that works before +# anything is installed. Acquire::Retries absorbs snapshot's occasional # throttling. (The durable fix is a base image whose archive is alive and whose # glibc still fits the host -- amazonlinux:2023 -- tracked separately.) -RUN printf 'deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/20260830T000000Z bullseye main\ndeb [check-valid-until=no] http://snapshot.debian.org/archive/debian-security/20260830T000000Z bullseye-security main\n' > /etc/apt/sources.list \ +RUN printf 'deb [check-valid-until=no] https://snapshot.debian.org/archive/debian/20260830T000000Z bullseye main\ndeb [check-valid-until=no] https://snapshot.debian.org/archive/debian-security/20260830T000000Z bullseye-security main\n' > /etc/apt/sources.list \ && rm -rf /etc/apt/sources.list.d/* \ && apt-get -o Acquire::Retries=3 update \ && apt-get -o Acquire::Retries=3 install -y --no-install-recommends \