Skip to content

Commit c751724

Browse files
Stop fetching patch views during hosted scans (#2021)
* Stop downloading vlt artifacts during hosted scans A hosted scan of a vlt project downloaded every granted patch archive the way vlt fetches it, and withheld the dependency (`redirect_vlt_artifact_unverifiable`) when the server re-gzipped it. That guarded against patch.socket.dev's CDN re-encoding archives, which vlt rejects with EINTEGRITY. The patch server has sent archives with `Cache-Control: no-transform` since depscan #26856 and #26867, and production serves them unencoded today. On any mirror that still re-encodes, `vlt ci` fails closed with EINTEGRITY, so nothing installs unpatched bytes. So the scan no longer fetches the archives. It pins vlt dependencies directly, without a full download of every patched package per run. The in-memory hosted engine (the Node addon and `hosted-bundle`) has no network, so it used to withhold every vlt dependency as "offline"; it now pins them the way the disk flow does. The warm-tree heal now checks the Socket-owned lock instances this run confirmed in `vlt-lock.json`. With no patch record and no hidden lock it reports the copy as undeterminable (asking for `vlt ci`) instead of comparing it against downloaded archive bytes. The vlt backtest still probes the live artifact itself: a must-patch hosted cell records `blocked-by-server-encoding` when its probe sees a content-encoded response, without running the CLI. Assisted-by: Claude Code:claude-opus-5-5 * Refuse vendored patches the same way everywhere A vendored run decides before any download which patches the engine would refuse, so it never fetches a view, asks for a grant or strips a hosted pin for a package it cannot vendor. That decision was assembled in the CLI from five sources (the Bun and vlt preflights, the package-lock project refusal, the lock-text gates and the pnpm takeover), and each entry point merged its own subset by hand. They had drifted: the download phase never asked the package-lock project refusal or the pnpm takeover gates, `get <uuid>` raised only the Bun and vlt refusals before the engine, and the vendor loop judged only Bun and vlt before its takeover. All of it now lives in one core function, `socket_patch_core::vendor::refusals::vendor_refusals`, which runs the checks in the engine's order and scopes the lock-text gates (hosted pins, the ledger's idempotency skip, packages that never reach their backend). The scan preview and rollout plan, the download phase, both `get` paths and the vendor loop call it. The CLI's bun_preflight.rs and vlt_preflight.rs are gone, and their tests moved to core. What users see: a v1 package-lock project, or a hosted pnpm pin whose takeover the pnpm gates refuse, is now refused before its patch view is fetched; `get <uuid> --mode vendored` raises every such refusal with its pre-record error envelope and writes nothing; and `vendor` refuses them before restoring a hosted pin instead of restoring and rolling back. Codes and details are unchanged. Assisted-by: Claude Code:claude-opus-5-5 * Stop fetching patch views during hosted scans A hosted scan fetched the full patch view (every patched file's blob content) for each confirmed redirect, only to judge installed copies and feed the same run's VEX. It now makes no view call at all: - In-run `--vex` fetches the records it attests from itself, through the same record path the standalone `vex` uses, so the scan pipeline never asks for them. - The gem stale-install probe judges an installed gem by the archive RubyGems installed it from (`<home>/cache/<leaf>.gem`) against the patched artifact's sha256. - The Python stale-install probe downloads the patched wheel the lock now pins, through its grant URL and verified against the pinned sha256, and compares the installed copy with the wheel's own RECORD when the installed WHEEL tags match. It downloads only when a copy is installed. - Both probes still treat an agent-mode manifest record of the same patch as proof of an in-place patch. - The vlt warm-tree heal judges a store entry by vlt's hidden lock alone: the pinned integrity there is healthy, another is stale, and without a readable hidden lock the copy is kept and reported as unchecked. - The in-memory hosted engine no longer calls the provider's `fetchPatch` or warns `record_fetch_failed`. The Node binding still accepts a `fetchPatch` provider method and ignores it, so existing providers keep working. Assisted-by: Claude Code:claude-opus-5-5 * Read wheel probe files through the regular-file guard; fix vlt rule (c) e2e judge_installed_wheel read the installed WHEEL and every RECORD path with bare std::fs reads. Site-packages is user-writable, so a FIFO there would block the hosted scan's blocking pool. Read through read_regular_*_sync so a FIFO or device counts as unreadable, and add a regression test for both the WHEEL and a RECORD path. The real-vlt e2e for heal rule (c) without a hidden lock still expected the old record-based heal (copies invalidated). Hosted scans no longer fetch patch records, so that case is now undeterminable: the copies are kept and the advisory says to run `vlt ci`. Match the in-process test and check that `vlt ci` then installs the patched package. Co-Authored-By: Claude <noreply@anthropic.com> * Drop wiremock imports the vlt rule (c) tests no longer use The record-fetch-failure test that mounted a 404 mock is gone, so Mock, method and path were left unused after merging main. Co-Authored-By: Claude <noreply@anthropic.com> * Judge unevidenced stale installs by the patch view, else withhold Owner decision on the hosted.rs:405 review thread. The gem and Python stale-install probes no longer turn "can't judge" into "not stale": - Gem: when there is no readable cache/<leaf>.gem (pruned cache, gem cleanup) or no patched sha to compare it with, and no agent-mode record, the probe fetches that patch's /patches/view record and judges the installed dir by its afterHash map. - Python: when the patched wheel cannot judge a copy (sdist pin, failed download or sha256 check, installed copy of another build), the probe falls back to the view record the same way, unless another variant already proved the copy patched. - A copy the view cannot judge either (fetch failure, or --offline) is reported as unjudged: it leaves assume_applied and goes to a new VexBuildParams::unjudged list, which in-run --vex withholds as stale_install_unverified, even under --vex-no-verify. The fallback fetches at most once per uuid and only for copies nothing else settled, so the normal path still makes no view request. vlt is unchanged: an undeterminable heal target already lands in stale_purls. The in-memory engine runs no stale probe, so the Node binding's fetchPatch stays ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Mark every variant of a stale gem dir stale /code-review high finding: a stale gem dir put only the variant named in its warning in stale_purls. The other platform variants stayed in assume_applied, and in-run VEX matches exemptions qualifier-insensitively, so they re-exempted the stale dir. Also drops a leftover blank line in the Node smoke test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent d19ce41 commit c751724

31 files changed

Lines changed: 2077 additions & 680 deletions

‎crates/socket-patch-cli/CLI_CONTRACT.md‎

Lines changed: 6 additions & 5 deletions
Large diffs are not rendered by default.

‎crates/socket-patch-cli/src/commands/scan/hosted.rs‎

Lines changed: 535 additions & 119 deletions
Large diffs are not rendered by default.

‎crates/socket-patch-cli/src/commands/scan/hosted/python.rs‎

Lines changed: 476 additions & 42 deletions
Large diffs are not rendered by default.

‎crates/socket-patch-cli/src/commands/vex.rs‎

Lines changed: 153 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ use std::path::{Path, PathBuf};
2121

2222
use clap::Args;
2323
use socket_patch_core::api::client::ApiClient;
24-
use socket_patch_core::manifest::schema::{PatchManifest, PatchRecord};
24+
use socket_patch_core::manifest::schema::PatchManifest;
2525
use socket_patch_core::telemetry::{track_vex_failed, track_vex_generated, TelemetryAuth};
2626
use socket_patch_core::vex::verify::{npm_manifest_after_hash, DEPENDENCY_NOT_INSTALLED};
2727
use socket_patch_core::vex::{
@@ -172,13 +172,14 @@ impl VexEmbedArgs {
172172
compact: self.vex_compact,
173173
assume_applied: Vec::new(),
174174
known_stale: Vec::new(),
175+
unjudged: Vec::new(),
175176
hosted_gem_mirror_refused: false,
176177
hosted_gem_mirror_check: false,
177178
// Embedded callers skip VEX entirely under `--dry-run`.
178179
dry_run: false,
179180
product_flag: "--vex-product",
180181
npm_prior: None,
181-
hosted_records: Default::default(),
182+
hosted_uuids: Vec::new(),
182183
}
183184
}
184185
}
@@ -205,6 +206,12 @@ pub(crate) struct VexBuildParams {
205206
/// Hosted probes positively identified unpatched installed bytes. These
206207
/// PURLs cannot be attested by another interpreter or --no-verify.
207208
pub known_stale: Vec<String>,
209+
/// Hosted probes found an installed copy but nothing to judge it by: no
210+
/// archive or patched wheel to compare, and the patch's view record
211+
/// could not be fetched (or `--offline`). These PURLs are withheld
212+
/// (`stale_install_unverified`) whatever `assume_applied`, another
213+
/// interpreter's copy or --no-verify would say.
214+
pub unjudged: Vec<String>,
208215
/// Embedded hosted scan refused gem routing because a Bundler mirror
209216
/// captures a patch source. Rediscovered hosted gem pins (including older
210217
/// pins outside this scan's candidates) then require verified installed
@@ -232,12 +239,12 @@ pub(crate) struct VexBuildParams {
232239
/// choice and order are unchanged. Ignored when taken with other crawler
233240
/// options. The standalone `vex` passes `None` and walks the tree.
234241
pub npm_prior: Option<crate::ecosystem_dispatch::NpmCrawlSnapshot>,
235-
/// Embedded hosted `scan --vex` only: the patch records THIS RUN
236-
/// fetched for the pins it confirmed, keyed by purl. v5 hosted mode
237-
/// keeps no ledger, so these are the in-run attestation's hosted
238-
/// records (the post-install standalone `vex` fetches them from the
239-
/// API instead). Empty everywhere else.
240-
pub hosted_records: std::collections::BTreeMap<String, PatchRecord>,
242+
/// Embedded hosted `scan --vex` only: the uuids of the pins THIS RUN
243+
/// confirmed. v5 hosted mode keeps no ledger, so the plan fetches their
244+
/// records as the in-run attestation's hosted records (the post-install
245+
/// standalone `vex` finds them through the lockfiles instead). Empty
246+
/// everywhere else.
247+
pub hosted_uuids: Vec<String>,
241248
/// The run's API client (see [`RunApiClient`]): seeded by a host
242249
/// command, else built at most once when the plan must fetch records.
243250
/// Telemetry reads its route when it exists.
@@ -368,12 +375,13 @@ pub async fn run(args: VexArgs) -> i32 {
368375
compact: args.compact,
369376
assume_applied: Vec::new(),
370377
known_stale: Vec::new(),
378+
unjudged: Vec::new(),
371379
hosted_gem_mirror_refused: false,
372380
hosted_gem_mirror_check: false,
373381
dry_run: args.common.dry_run,
374382
product_flag: "--product",
375383
npm_prior: None,
376-
hosted_records: Default::default(),
384+
hosted_uuids: Vec::new(),
377385
api_client: RunApiClient::new(),
378386
};
379387

@@ -597,6 +605,10 @@ async fn yarn_classic_lock_dependency_gaps(
597605
/// the dependencies the lock installs beside it (#1538).
598606
const DEPENDENCY_MANIFEST_UNVERIFIABLE: &str = "dependency_manifest_unverifiable";
599607

608+
/// Omission tag: an in-run hosted probe found an installed copy it could
609+
/// not judge (see [`VexBuildParams::unjudged`]).
610+
const STALE_INSTALL_UNVERIFIED: &str = "stale_install_unverified";
611+
600612
/// Largest patched `package.json` blob [`patched_blob`] reads.
601613
const MAX_PATCHED_MANIFEST_BYTES: usize = 4 * 1024 * 1024;
602614

@@ -946,6 +958,34 @@ async fn generate_vex(
946958
}),
947959
);
948960
}
961+
// A copy no hosted probe could judge is never attested. A failure
962+
// already recorded (positive stale evidence above, or this run's own
963+
// verification) keeps its more specific reason.
964+
if !params.unjudged.is_empty() {
965+
use socket_patch_core::utils::purl::strip_purl_qualifiers;
966+
let unjudged: std::collections::HashSet<&str> = params
967+
.unjudged
968+
.iter()
969+
.map(|purl| strip_purl_qualifiers(purl))
970+
.collect();
971+
let is_unjudged = |purl: &str| unjudged.contains(strip_purl_qualifiers(purl));
972+
outcome.applied.retain(|purl| !is_unjudged(purl));
973+
let failed: std::collections::HashSet<String> = outcome
974+
.failed
975+
.iter()
976+
.map(|failure| failure.purl.clone())
977+
.collect();
978+
outcome.failed.extend(
979+
manifest
980+
.patches
981+
.keys()
982+
.filter(|purl| is_unjudged(purl) && !failed.contains(*purl))
983+
.map(|purl| FailedPatch {
984+
purl: purl.clone(),
985+
reason: STALE_INSTALL_UNVERIFIED.to_string(),
986+
}),
987+
);
988+
}
949989

950990
// A Go replace that the go.work in effect overrides, ignores, or makes
951991
// conflict with another member's (#393, #458, #531) is not what the
@@ -1818,21 +1858,13 @@ async fn generate_vex_from_manifest_path_inner(
18181858
for diag in &discovery.diagnostics {
18191859
note_warning(warnings, common, diag.code, diag.detail.clone());
18201860
}
1821-
// This run's hosted records (embedded hosted `scan --vex`) join a
1822-
// pre-v5 ledger's as the hosted record source, newest wins.
1823-
let redirect = if params.hosted_records.is_empty() {
1824-
redirect
1825-
} else {
1826-
let mut state =
1827-
redirect.unwrap_or_else(socket_patch_core::patch::redirect::RedirectState::new);
1828-
state.records.extend(params.hosted_records.clone());
1829-
Some(state)
1830-
};
18311861
let sources = Sources {
18321862
manifest: manifest_file.unwrap_or_else(PatchManifest::new),
18331863
vendor,
18341864
redirect,
18351865
discovery,
1866+
// Their records join a pre-v5 ledger's as the hosted record source.
1867+
hosted_uuids: params.hosted_uuids.clone(),
18361868
};
18371869
if sources.is_empty() {
18381870
// A discovery diagnostic (an unparseable lock, a rejected reference)
@@ -2092,6 +2124,11 @@ fn omission_phrase(reason: &str) -> &'static str {
20922124
against"
20932125
}
20942126
"stale_install" => "the installed copy is not patched",
2127+
STALE_INSTALL_UNVERIFIED => {
2128+
"the hosted scan found an installed copy but could not check it: there was no \
2129+
installed archive or patched wheel to compare, and the patch record could not be \
2130+
fetched"
2131+
}
20952132
"native_extension_not_rebuilt" => {
20962133
"the patched native-extension sources were not compiled into the extension Ruby \
20972134
loads; run socket-patch apply to rebuild it"
@@ -2315,6 +2352,7 @@ mod tests {
23152352
"vendor_artifact_missing",
23162353
"vendor_manifest_unverifiable",
23172354
"stale_install",
2355+
STALE_INSTALL_UNVERIFIED,
23182356
"mirror_overrides_source",
23192357
DEPENDENCY_NOT_INSTALLED,
23202358
DEPENDENCY_MANIFEST_UNVERIFIABLE,
@@ -2576,12 +2614,13 @@ mod npm_prior_tests {
25762614
compact: false,
25772615
assume_applied: Vec::new(),
25782616
known_stale: Vec::new(),
2617+
unjudged: Vec::new(),
25792618
hosted_gem_mirror_refused: false,
25802619
hosted_gem_mirror_check: false,
25812620
dry_run: false,
25822621
product_flag: "--vex-product",
25832622
npm_prior: prior,
2584-
hosted_records: Default::default(),
2623+
hosted_uuids: Vec::new(),
25852624
api_client: RunApiClient::new(),
25862625
};
25872626
let manifest_path = common.resolved_manifest_path();
@@ -2655,6 +2694,7 @@ mod npm_prior_tests {
26552694
mod mirror_refusal_tests {
26562695
use super::*;
26572696
use crate::commands::vex_sources::HostedWiring;
2697+
use socket_patch_core::manifest::schema::PatchRecord;
26582698
use std::collections::{BTreeMap, HashSet};
26592699

26602700
/// The current scan can refuse routing while VEX rediscovers an older pin
@@ -2855,3 +2895,96 @@ mod mirror_refusal_tests {
28552895
}
28562896
}
28572897
}
2898+
2899+
#[cfg(test)]
2900+
mod unjudged_tests {
2901+
use super::*;
2902+
use crate::commands::vex_sources::HostedWiring;
2903+
use socket_patch_core::manifest::schema::PatchRecord;
2904+
use std::collections::{BTreeMap, HashSet};
2905+
2906+
/// A copy the in-run hosted probes could not judge is withheld as
2907+
/// `stale_install_unverified`, whatever `assume_applied` or
2908+
/// `--no-verify` say; positive stale evidence keeps `stale_install`.
2909+
#[tokio::test]
2910+
async fn unjudged_installs_are_withheld_even_when_assumed_applied() {
2911+
let tmp = tempfile::tempdir().unwrap();
2912+
let common = GlobalArgs {
2913+
cwd: tmp.path().to_path_buf(),
2914+
json: true,
2915+
no_telemetry: true,
2916+
..Default::default()
2917+
};
2918+
let purls = [
2919+
"pkg:pypi/judged@1.0.0",
2920+
"pkg:pypi/unjudged@1.0.0",
2921+
"pkg:gem/both@1.0.0",
2922+
];
2923+
for no_verify in [false, true] {
2924+
let mut view = PatchManifest::new();
2925+
for (index, purl) in purls.iter().enumerate() {
2926+
let record: PatchRecord = serde_json::from_value(serde_json::json!({
2927+
"uuid": format!("00000000-0000-4000-8000-00000000000{index}"),
2928+
"exportedAt": "2026-01-01T00:00:00Z",
2929+
"files": {"m.py": {"beforeHash": "a".repeat(64), "afterHash": "b".repeat(64)}},
2930+
"vulnerabilities": {format!("GHSA-unjudged-000{index}"): {"cves": [format!("CVE-2026-200{index}")], "summary": "test", "severity": "high", "description": "test"}},
2931+
"description": "test", "license": "MIT", "tier": "free"
2932+
}))
2933+
.unwrap();
2934+
view.patches.insert(purl.to_string(), record);
2935+
}
2936+
let hosted = BTreeMap::from(purls.map(|purl| {
2937+
(
2938+
purl.to_string(),
2939+
HostedWiring {
2940+
uuid: view.patches[purl].uuid.clone(),
2941+
refs: Vec::new(),
2942+
},
2943+
)
2944+
}));
2945+
let plan = Plan {
2946+
view,
2947+
vendor_entries: HashMap::new(),
2948+
redirected: purls.map(str::to_string).to_vec(),
2949+
lockfile_basis: purls
2950+
.map(str::to_string)
2951+
.into_iter()
2952+
.collect::<HashSet<_>>(),
2953+
hosted,
2954+
gated: Vec::new(),
2955+
notes: Vec::new(),
2956+
};
2957+
let mut params = VexEmbedArgs {
2958+
vex: Some(tmp.path().join(format!("{no_verify}.json"))),
2959+
vex_product: Some("pkg:generic/test@1.0.0".into()),
2960+
vex_no_verify: no_verify,
2961+
..Default::default()
2962+
}
2963+
.to_build_params(None);
2964+
params.assume_applied = purls.map(str::to_string).to_vec();
2965+
params.known_stale = vec![purls[2].to_string()];
2966+
params.unjudged = vec![format!("{}?artifact_id=x", purls[1]), purls[2].to_string()];
2967+
let summary = generate_vex(&common, &params, plan, &mut Vec::new())
2968+
.await
2969+
.unwrap_or_else(|err| panic!("{}: {}", err.code, err.message));
2970+
assert_eq!(summary.statements, 1, "no_verify={no_verify}");
2971+
let doc = serde_json::to_string(&summary.doc).unwrap();
2972+
assert!(doc.contains(purls[0]), "{doc}");
2973+
assert!(!doc.contains(purls[1]) && !doc.contains(purls[2]), "{doc}");
2974+
let mut failed: Vec<(&str, &str)> = summary
2975+
.failed
2976+
.iter()
2977+
.map(|f| (f.purl.as_str(), f.reason.as_str()))
2978+
.collect();
2979+
failed.sort();
2980+
assert_eq!(
2981+
failed,
2982+
[
2983+
(purls[2], "stale_install"),
2984+
(purls[1], STALE_INSTALL_UNVERIFIED),
2985+
],
2986+
"no_verify={no_verify}"
2987+
);
2988+
}
2989+
}
2990+
}

0 commit comments

Comments
 (0)