From bdb99d77500285db9e2fba0597105dbeaaf4e2b4 Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Sat, 15 Aug 2026 16:13:13 +0200 Subject: [PATCH] fix(release): stop re-scanning bytes VirusTotal has already scanned The verify pass submitted every extracted object, selected executables included, on the stated grounds that "VirusTotal is content-addressed, so identical bytes return the analysis it already holds instead of re-running 70+ engines". That is measurably false. On v0.10.5 all EIGHT re-submissions produced a NEW analysis - same VirusTotal file-id, timestamp 47 minutes later: candidate: file-id=2c00f485... ts=1786795957 (12:12:37Z) verify : file-id=2c00f485... ts=1786798758 (12:59:18Z) Re-analysing identical bytes re-rolls a probabilistic classifier, and Microsoft's ML engine answered differently within that hour, in BOTH directions: 82750cd1 (linux-amd64) microsoft-ml -> clean 6d3c5be6 (darwin-arm64) clean -> microsoft-ml The published notes are generated from the candidate scan, so v0.10.5 shipped a table calling linux-amd64 flagged when VirusTotal had it clean, and darwin-arm64 clean when VirusTotal was reporting Trojan:Script/Wacatac.B!ml. Every hash in that table links to the page that contradicted it. Corrected in place after publication; this removes the cause. The second scan proved nothing the first did not. Identity is settled by hash before this step runs: verify-release-selection.py reconciles every published container to the selected bytes, and checksums.txt binds the same digests publicly. A re-scan adds no assurance - only another roll. What still gets scanned is exactly what the candidate pass never saw: install.sh, install.ps1, LICENSE, THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the unpacked UI assets. install.sh and install.ps1 are the highest-consequence non-executable bytes we publish - users pipe them straight into a shell - and that coverage is untouched. Measured on the v0.10.5 object set: 16 objects in, 8 withheld, 8 still scanned. The withheld set is recorded as evidence (cbm-virustotal-withheld-v1) naming each sha256 and pointing at virustotal-candidate-results.tsv, so the published evidence still accounts for every shipped object. Fails closed three ways, each with an actionable message: no object matches a selected sha (the containers do not carry the recorded bytes), everything is withheld (the surface scan would be a no-op), or the selection names no shas at all. The zero-match grep is wrapped rather than left to pipefail, because a guard that aborts silently is not a guard - found by testing the guards rather than assuming them. Also drops 8 VirusTotal submissions per release. Signed-off-by: Martin Vogel --- .github/workflows/release.yml | 21 +++- .../ci/exclude-rescanned-selected-objects.sh | 101 ++++++++++++++++++ 2 files changed, 118 insertions(+), 4 deletions(-) create mode 100755 scripts/ci/exclude-rescanned-selected-objects.sh diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 1dc392505..2694bbacd 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -316,10 +316,23 @@ jobs: # install.sh and install.ps1 are the highest-consequence non-executable # bytes we publish: users pipe them straight into a shell. # - # The selected executables are re-submitted too. That is deliberate and - # nearly free: VirusTotal is content-addressed, so identical bytes return - # the analysis it already holds instead of re-running 70+ engines — the - # same property that made the analysis-id equality check untenable. + # The selected executables are NOT re-submitted. They were scanned as + # candidates, and verify-release-selection.py has just proven every + # published container carries exactly those bytes — identity is settled by + # hash, so a second scan adds no assurance. + # + # It also is not free, contrary to what this comment used to claim. + # Measured on v0.10.5: all eight re-submissions produced a NEW analysis + # (same file-id, timestamp 47 minutes later), and Microsoft's ML engine + # answered differently for two of them within that hour — in opposite + # directions. The release notes then cited one scan while the links showed + # the other. See scripts/ci/exclude-rescanned-selected-objects.sh. + - name: Withhold already-scanned selected executables + run: | + bash scripts/ci/exclude-rescanned-selected-objects.sh \ + binaries/objects "$RUNNER_TEMP/release-selection.tsv" \ + binaries/virustotal-withheld.tsv + - name: VirusTotal scan of every extracted release object uses: crazy-max/ghaction-virustotal@936d8c5c00afe97d3d9a1af26d017cfdf26800a2 # v5.0.0 id: virustotal diff --git a/scripts/ci/exclude-rescanned-selected-objects.sh b/scripts/ci/exclude-rescanned-selected-objects.sh new file mode 100755 index 000000000..22aabd2cf --- /dev/null +++ b/scripts/ci/exclude-rescanned-selected-objects.sh @@ -0,0 +1,101 @@ +#!/usr/bin/env bash +# Withhold the selected executables from the full-surface VirusTotal pass. +# +# WHY (measured, not assumed). The verify pass used to submit every extracted +# object, selected executables included, on the stated grounds that "VirusTotal +# is content-addressed, so identical bytes return the analysis it already holds +# instead of re-running 70+ engines". That is not what happens. On the v0.10.5 +# release all EIGHT re-submissions produced a NEW analysis: same VirusTotal +# file-id, a timestamp 47 minutes later. +# +# candidate: file-id=2c00f485... ts=1786795957 (12:12:37Z) +# verify : file-id=2c00f485... ts=1786798758 (12:59:18Z) +# +# Re-analysing identical bytes re-rolls a probabilistic classifier, and it +# answered differently in the same hour, in both directions: +# +# 82750cd1 (linux-amd64) microsoft-ml -> clean +# 6d3c5be6 (darwin-arm64) clean -> microsoft-ml +# +# The published release notes then cited the candidate verdict while the linked +# page showed the verify verdict, so the table said "clean" for a binary +# VirusTotal was flagging and vice versa. Nothing about the binaries differed: +# same sha256 in both scans. +# +# The second scan also proved nothing the first did not. Identity is already +# established by hash: verify-release-selection.py reconciles every published +# container to the selected bytes before this step runs, and checksums.txt binds +# the same digests publicly. A re-scan adds no assurance and one more roll. +# +# What still gets scanned is everything the candidate pass never saw: install.sh, +# install.ps1, LICENSE, THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the +# unpacked UI assets. install.sh and install.ps1 are the highest-consequence +# non-executable bytes we publish - users pipe them straight into a shell - and +# that coverage is untouched. +# +# Fails closed: if nothing matches, the selection binding is broken; if nothing +# is left, the surface scan would silently become a no-op. +# +# Usage: exclude-rescanned-selected-objects.sh [manifest-out] +set -euo pipefail + +OBJECTS_DIR="${1:?usage: exclude-rescanned-selected-objects.sh [manifest-out]}" +SELECTION="${2:?usage: exclude-rescanned-selected-objects.sh [manifest-out]}" +MANIFEST="${3:-}" + +test -d "$OBJECTS_DIR" || { echo "error: no objects directory: $OBJECTS_DIR" >&2; exit 1; } +test -s "$SELECTION" || { echo "error: no selection evidence: $SELECTION" >&2; exit 1; } + +head -n 1 "$SELECTION" | grep -qx '# cbm-release-selection-v1' || { + echo "error: wrong evidence marker in $SELECTION" >&2; exit 1; } + +# selected_sha256 is the 4th column; skip the '#' metadata block and the header. +selected="$(mktemp)" +withheld="$(mktemp)" +trap 'rm -f "$selected" "$withheld"' EXIT +# `|| true` on the grep: with `set -o pipefail` a zero-match grep would abort the +# script here, before the explicit check below could say why. A guard that exits +# silently is not a guard. +awk -F '\t' '/^#/ {next} $1=="target" {next} {print $4}' "$SELECTION" \ + | { grep -E '^[0-9a-f]{64}$' || true; } | sort -u > "$selected" +if [ ! -s "$selected" ]; then + echo "error: no selected sha256 values in $SELECTION — the selection" >&2 + echo " evidence names no shipped bytes, so nothing can be matched." >&2 + exit 1 +fi + +kept=0 +for f in "$OBJECTS_DIR"/*; do + [ -f "$f" ] || continue + sha="$(sha256sum "$f" | awk '{print $1}')" + if grep -qx "$sha" "$selected"; then + printf '%s\t%s\n' "$sha" "$(basename "$f")" >> "$withheld" + rm -f -- "$f" + else + kept=$((kept + 1)) + fi +done + +count="$(wc -l < "$withheld" | tr -d ' ')" +if [ "$count" -eq 0 ]; then + echo "error: no extracted object matched a selected sha256 — the release" >&2 + echo " containers do not carry the bytes the selection recorded." >&2 + exit 1 +fi +if [ "$kept" -eq 0 ]; then + echo "error: every extracted object was withheld; the full-surface scan" >&2 + echo " would cover nothing." >&2 + exit 1 +fi + +if [ -n "$MANIFEST" ]; then + { + echo "# cbm-virustotal-withheld-v1" + echo "# reason=already-scanned-as-candidate" + echo "# evidence=virustotal-candidate-results.tsv" + printf 'sha256\tobject\n' + cat "$withheld" + } > "$MANIFEST" +fi + +echo "withheld $count already-scanned selected executable(s); $kept object(s) remain for the surface scan"