Skip to content

test(sbom): build e2e fixtures on the container-factory builder image - #323

Draft
reyreavman wants to merge 7 commits into
mainfrom
test/sbom/migrate-builders-to-base-images
Draft

reyreavman wants to merge 7 commits into
mainfrom
test/sbom/migrate-builders-to-base-images

Conversation

@reyreavman

@reyreavman reyreavman commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

Every SBOM e2e fixture now builds its trusted builder from the base-images container-factory distroless image instead of registry.werf.io/base/*, python:3.12-slim, debian:bookworm-slim or rust:1.78-slim. Language runtimes are installed by an os-pm packages entry and yarn/pnpm/poetry/uv are bootstrapped by a preceding npm/pip entry plus packages[].manager, so the fixtures exercise the workflow #318 was built for.

Kaiten: 68939510.

What

Fixtures

  • All 33 Dockerfile.builder-base start FROM registry.deckhouse.io/container-factory@sha256:b9eb… with PACKAGES_VERSION=v3.0.2; grep registry.werf.io test/e2e/sbom is empty.
  • npm, pnpm, yarn_manager: os-pm node==24.18.0; pnpm and yarn come from a tools/ javascript-npm entry and are referenced via manager: /opt/tools/node_modules/.bin/<tool>.
  • yarn_simple keeps yarn on PATH (npm install -g yarn@1.22.22 in the builder) so the default-manager path stays covered next to yarn_manager.
  • poetry, uv: os-pm python==3.12.12; the tool comes from a tools/requirements.txt python-pip entry with PIP_TARGET=/opt/tools and manager: /opt/tools/bin/<tool>. poetry runs with PYTHONPATH=/opt/tools and POETRY_VIRTUALENVS_CREATE=false.
  • uv is pinned to 0.12.8: 0.4.x and 0.9.x fail on the distroless base with Could not read ELF interpreter from any of the following paths: /bin/sh, … (busybox is static).
  • pip_simple, poetry, uv builders add ln -s pip3 /usr/bin/pip: the base-images python package ships pip3 only, and the python-pip default manager is pip.
  • cargo: os-pm rust==1.96.0 cargo==1.96.0 curl==8.12.1; without curl cargo dies with libcurl.so.4: cannot open shared object file.
  • gomod_replace, stage_deps_gomod: os-pm golang==1.25.14. type_change/state1 installs go in the builder instead, because the test asserts the SBOM loses the os-pm component after the type switch.
  • lua_simple, lua_missing_rockspec: os-pm lua==5.3.6 luarocks==3.12.2. lua 5.4.8 from the same index is compiled with AVX-512 and dies with Illegal instruction on the CI runners; 5.3.6 is a plain x86-64 build.
  • 9 werf.yaml (multiplatform, signing_multiplatform, gost*, sbom_disabled, ospm_scratch_secrets) use from: scratch instead of registry.werf.io/werf/scratch:latest.

Tests

  • broken_pm now fails in the packages stage with pm's invalid character 'h' in literal true on the corrupted /var/lib/pm/index.json; before, the ubuntu builder had no pm at all and the test duplicated no_pm_binary. The table is renamed accordingly and the assertion pins invalid character.
  • no_pm_binary removes /usr/local/bin/pm from the container-factory builder and still fails with pm: command not found (exit 127).
  • SBOMs of the migrated fixtures additionally list the os-pm runtime (pkg:generic/node@24.18.0?containerfactoryversion=v3.0.2 etc.); no assertion checks for their absence.
  • Test assertions are otherwise unchanged; werf source is untouched.

Why

The card asks to stop relabelling third-party werf.io images as trusted builders. The container-factory builder image pins PACKAGES_VERSION=v1.3.6, whose node package links libsqlite3.so unversioned and does not start, and whose index lacks node 24; index v3.0.2 provides working node/python/rust/golang/lua, so the fixtures override the version. Shipping yarn, pnpm, poetry and uv as base-images packages (the alternative from the card comment) would make the fixtures depend on a base-images release; the manager bootstrap needs nothing outside this repository and is the pattern #318 documents.

…ilder

Replace the werf.io node image with the container-factory distroless
builder; os-pm installs node==24.18.0 in the packages stage and pnpm is
bootstrapped through a preceding javascript-npm entry plus manager,
mirroring yarn_manager. yarn_simple keeps yarn on PATH via npm -g in the
builder so the default-manager path stays covered.

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
os-pm installs python==3.12.12; poetry and uv are bootstrapped by a
python-pip entry with PIP_TARGET=/opt/tools and referenced via manager.
uv is pinned to 0.12.8: earlier releases refuse the static busybox
/bin/sh of the distroless base ("Could not read ELF interpreter").

The base-images python package ships pip3 without a bare pip, so the
builders add the symlink the python-pip default manager expects.

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
…tory builder

os-pm installs rust/cargo 1.96.0, golang 1.25.14 and lua 5.4.8 with
luarocks 3.12.2. cargo additionally needs curl: the base-images cargo
package links libcurl.so.4 without declaring it. type_change/state1
installs go in the builder instead, since the test asserts the SBOM has
no os-pm component after switching from os-pm to go-mod.

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
…tures

The container-factory builder carries pm, so broken_pm now exercises the
corrupted-index path it is named after instead of a missing binary, and
the assertion pins pm's parse error. no_pm_binary removes pm explicitly.

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
… in e2e fixtures

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
…binary

The packages stage requires PACKAGES_VERSION before running pm; broken_pm relied on the base image default. no_pm_binary removes pm before the version is read. Update the type_change By text: golang is now an os-pm component of the builder, the assertion only covers jq.

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
@reyreavman

reyreavman commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator Author

Verification

  • werf build --dev + werf sbom get (linux/amd64 under Docker Desktop on arm64, local kind-registry:5001, WERF_E2E_ALLOW_LOCAL_BUILDER_IMAGES=true, python mock for WERF_EXTERNAL_REFS_SERVER_URL) for npm, yarn_simple, yarn_manager, pnpm, pip, poetry, uv, cargo, gomod_replace, type_change/state1, gost/defaults, multiplatform: expected components present (lodash@4.17.21, requests@2.32.3, anyhow@1.0.86, example.com/mylib, poetry@2.4.3, uv@0.12.8, pnpm@9.15.9, yarn@1.22.22).
  • sbom_caching state0→state0→state1→state2→state2 sequence: 4 stages built, then cache hit, cache hit, from+setup rebuilt, cache hit.
  • broken_pm: build fails in app/packages with Error executing command: invalid character 'h' in literal true (expecting 'r'). no_pm_binary: pm: command not found, exit 127.
  • lua_simple, lua_missing_rockspec: not run locally (arm64 host). First CI run failed both lua_simple entries with Illegal instruction (core dumped) luarocks install --only-deps: objdump of /usr/bin/lua 5.4.8 shows %zmm (AVX-512) instructions. Repinned to lua 5.3.6, which has none. Ginkgo suite not run locally, CI e2e jobs cover it.
  • Mutation: none run; the only assertion change is broken_pm (packages_test.go), whose failure output was observed by hand.

CI status (run 34825204476)

  • e2e_simple (carries every sbom spec): green on attempts 2 and 3 after the lua repin; attempt 1 failed only the two lua_simple entries.
  • unit: failed on attempt 1 in ./pkg/build while the suite itself logged 154/154 specs … SUCCESS!; green on rerun; no Go outside test/e2e/sbom is touched.
  • e2e_complex, e2e_extra: red on attempts 2 and 3 in test/e2e/build only. Attempt 2: Buildah specs hung past the 85m suite timeout. Attempt 3: rerun-failed skipped kind_setup, so every spec died with connection refused to the runner-local registry/kube API. Both jobs are also red on main (run 34590330771, 2026-09-11). Not caused by this diff; a fresh full run needs a new push.

Review focus

  • Whether pnpm 9.15.9 / lockfile 9.0 and uv 0.12.8 / uv.lock version = 1 stay compatible across upgrades — both pins are in tools/ and separate from the app lockfiles.

Follow-up

  • base-images: python package ships pip3 only, no pip — add the symlink (then drop the RUN ln -s pip3 /usr/bin/pip from the pip/poetry/uv builders here).
  • base-images: lua 5.4.8 is built with AVX-512 (vmovdqu8 (%rax),%zmm0) — MYCFLAGS in images/packages/lua/package.yml bypasses the -march=x86-64 -mtune=generic baseline; the binary SIGILLs on the CI runners.
  • base-images: cargo links libcurl.so.4 but does not declare curl in depends.runtime.
  • base-images: node@22.23.0 in index v1.3.6 links libsqlite3.so (unversioned); sqlite@3.49.1 provides only libsqlite3.so.0.
  • base-images: index v3.0.2node==24.18.0 together with node==22.23.0 resolves a manifest that is not in the registry (MANIFEST_UNKNOWN sha256:93a4de…); alone it installs fine.
  • base-images: rebuild builder/distroless with a PACKAGES_VERSION default whose index has working node, then drop the ENV PACKAGES_VERSION=v3.0.2 overrides from the fixtures.
  • base-images: ship yarn, pnpm, poetry, uv as packages (card 68939510 comment) — optional, the manager bootstrap does not need them.

The base-images lua 5.4.8 package is compiled with AVX-512 instructions
(zmm registers in /usr/bin/lua) and dies with "Illegal instruction" on
the CI runners and on any CPU without AVX-512. lua 5.3.6 from the same
index is a plain x86-64 build; the tests only assert that the rockspec
is cataloged, so the interpreter version is incidental.

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant