Skip to content

Commit 57923a3

Browse files
Keep vendored artifacts a lock references through escaped paths (#2022)
* Keep vendored artifacts a lock references through escaped paths The vendored revert's keep-check (`any_live_file_references`) did a plain text match of `.socket/vendor/<eco>/<uuid>` against the live wiring files. An older composer.lock writes `.socket\/vendor\/composer\/...`, a Windows checkout may write single or doubled backslashes, and either spelling missed the match: the check read "unreferenced" and the revert deleted an artifact the lock still installs from. Discovery's identity sweep already folds every one of those spellings (`\uXXXX` / XML-entity decode, then `\/`, `\\` and `\` to `/`). Move that fold into a pure util, `utils::path_spelling` (`fold_path_spellings`, plus `mentions_path` for "does this text name this path"), have the sweep call it unchanged, and route every vendored deletion gate that matched a vendor path as raw text through it: - `vendor::common::any_live_file_references` (composer / Maven / NuGet drift-keep in `revert::finish`) - `vendor::npm_flavor::lock_text_mentions_uuid` (npm unwired and post-restore guards, yarn classic and berry post-restore guards) - `vendor::pypi::pypi_reference_clause` (pypi unwired / post-restore keep) - `vendor::nuget_feed::config_references` (keeps the lock pin while a config still routes at the feed) - the CLI's `scan_vendor_references`, which the orphan sweep (`vendor --revert`, `scan --prune`, `rollback`, `remove`) uses to decide which un-ledgered dirs are still wired. It folded `\` to `/` only, so `\/` became `//` and the reference was lost. `mentions_path` also matches ASCII case-insensitively. Every change only widens what counts as a reference, so the failure mode is a kept artifact, never a deleted one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fold every spelling of vendor paths in all keep and refusal checks Review fixes for #2022: - One matcher, `utils::path_spelling::PathNeedle`: escapes decoded, every run of `/` and `\` collapsed to one `/` (so `\\\\` and `\\\/` match), an exact `contains` fast path, then a single KMP pass over the folded byte stream with the needle folded once. No copy of the scanned file. - Keep-vs-delete checks fold ASCII case (`PathNeedle::keep`). Checks that REFUSE fold case only where the project's file system does (`vendor::common::refusal_needle`, probed via `utils::fs::folds_case`), so a differently-cased path on a case-sensitive disk no longer blocks `vendor --revert`. - pypi: `unwire_superseded`'s residual check and `superseded_entry` read through the refusal needle (a `\/` or backslashed leftover was missed and the old uuid dir then swept as stale); `still_references_artifact` takes the caller's needle (broad for the pypi-lock drift keep, refusal for Hatch); `pypi_reference_clause` is broad for the post-restore keep and refusal-mode for the unwired guard and `vendor --check`; the requirements residual warning and the uv converged/drift checks use the broad needle. - npm: `lock_text_mentions_uuid` uses the refusal needle; its "textual and EXACT" doc is corrected. - CLI scan: `canonical_path_text` collapses separator runs; the orphan sweep reads `scan_vendor_references_any_case` (ASCII case-insensitive, like the revert keep-checks) while the refusing callers fold case only on a case-insensitive disk. - Test doc placement fixed; a different-uuid assertion added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fold case in every check whose miss deletes an artifact Second review round. Round 1 made the "refusal" checks fold ASCII case only where the project's disk does. Several of them are also deletion gates: a miss lets the revert delete the artifact. A committed lock or config is installed on case-folding disks (macOS, Windows) too, where a re-cased `.Socket/Vendor/...` path does resolve to the artifact, so a case-sensitive CI run deleted (and committed the deletion of) an artifact teammates still install from. These now use `PathNeedle::keep` (every spelling, any case): - the npm / yarn classic / yarn berry post-restore check (`lock_text_mentions_uuid`); - the unwired pypi revert guard; - the Hatch revert's leftover-reference check (now the same as the pylock revert's); - the superseded pypi entry's residual check before the old uuid dir is swept. `refusal_needle` remains only where a miss is safe (lifting another refusal; the `vendor --check` report). The module docs and the discovery identity-sweep comment say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 616e259 commit 57923a3

17 files changed

Lines changed: 924 additions & 147 deletions

File tree

‎crates/socket-patch-cli/src/commands/vendor_orphans.rs‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,9 @@ pub(crate) async fn sweep_orphan_vendor_dirs_where(
6868
if candidates.is_empty() {
6969
return out;
7070
}
71-
let references = crate::commands::vendored_backend::repair::scan_vendor_references(cwd).await;
71+
// Keep-vs-delete: any case counts, as in the revert keep-checks.
72+
let references =
73+
crate::commands::vendored_backend::repair::scan_vendor_references_any_case(cwd).await;
7274
let wired: HashSet<(&str, &str)> = references
7375
.iter()
7476
.map(|(eco, uuid, _path)| (eco.as_str(), uuid.as_str()))

‎crates/socket-patch-cli/src/commands/vendored_backend/repair.rs‎

Lines changed: 146 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,78 @@ struct Candidate {
3737
/// path parser. Used by repair (references the ledger does not cover), by
3838
/// the orphan sweeps (`vendor --revert`, `scan --prune`: a dir a lockfile
3939
/// still points at is never deleted) and by rollback.
40+
///
41+
/// Every escape and separator spelling of a reference counts (an older
42+
/// composer.lock's `\/`, any run of Windows backslashes, `\u` /
43+
/// XML-entity escapes). This form REFUSES on what it finds (`vendor`'s
44+
/// ledger-missing gate, rollback's ledger-less gate, `vendor --check`), so
45+
/// it folds ASCII case only where the project's file system does: on a
46+
/// case-sensitive disk another case names another directory. The orphan
47+
/// sweep's keep-vs-delete reads [`scan_vendor_references_any_case`].
4048
pub(crate) async fn scan_vendor_references(project_root: &Path) -> Vec<(String, String, String)> {
49+
let fold_case = socket_patch_core::utils::fs::folds_case(
50+
&project_root.join(socket_patch_core::constants::SOCKET_DIR),
51+
)
52+
.await;
53+
scan_vendor_references_folding(project_root, fold_case).await
54+
}
55+
56+
/// [`scan_vendor_references`] matching the `.socket/vendor/<eco>/<uuid>`
57+
/// anchor in any ASCII case, as the revert keep-checks
58+
/// ([`socket_patch_core::utils::path_spelling::PathNeedle::keep`]) do: the
59+
/// orphan sweep only ever KEEPS a dir on what it finds, so erring toward
60+
/// "referenced" is the safe direction and the two deletion paths agree.
61+
pub(crate) async fn scan_vendor_references_any_case(
62+
project_root: &Path,
63+
) -> Vec<(String, String, String)> {
64+
scan_vendor_references_folding(project_root, true).await
65+
}
66+
67+
/// Recover a vendored reference from a folded `candidate`, ASCII
68+
/// case-insensitively when `fold_case` (eco and uuid lower-cased, the leaf
69+
/// and the relpath kept as spelled).
70+
fn parse_reference_folding(candidate: &str, fold_case: bool) -> Option<(String, String, String)> {
71+
if let Some(parts) = parse_vendor_reference(candidate) {
72+
let rel = candidate
73+
.trim_start_matches("./")
74+
.trim_end_matches('/')
75+
.to_string();
76+
return Some((parts.eco, parts.uuid, rel));
77+
}
78+
if !fold_case {
79+
return None;
80+
}
81+
let parts = parse_vendor_reference(&candidate.to_ascii_lowercase())?;
82+
// ASCII lower-casing keeps every byte offset, and the leaf is the
83+
// candidate's own tail.
84+
let trimmed = candidate.trim_end_matches('/');
85+
let leaf = &trimmed[trimmed.len() - parts.leaf.len()..];
86+
let mut rel = format!(".socket/vendor/{}/{}", parts.eco, parts.uuid);
87+
if !leaf.is_empty() {
88+
rel.push('/');
89+
rel.push_str(leaf);
90+
}
91+
Some((parts.eco, parts.uuid, rel))
92+
}
93+
94+
/// The byte offset of the first `.socket` in `text`, ASCII
95+
/// case-insensitively when `fold_case`.
96+
fn find_socket_anchor(text: &str, fold_case: bool) -> Option<usize> {
97+
const ANCHOR: &str = ".socket";
98+
if !fold_case {
99+
return text.find(ANCHOR);
100+
}
101+
text.match_indices('.').map(|(i, _)| i).find(|&i| {
102+
text.as_bytes()
103+
.get(i..i + ANCHOR.len())
104+
.is_some_and(|s| s.eq_ignore_ascii_case(ANCHOR.as_bytes()))
105+
})
106+
}
107+
108+
async fn scan_vendor_references_folding(
109+
project_root: &Path,
110+
fold_case: bool,
111+
) -> Vec<(String, String, String)> {
41112
let mut seen: HashSet<(String, String)> = HashSet::new();
42113
let mut out = Vec::new();
43114
// The binary lock is read unless a readable text lock shadows it: a
@@ -72,8 +143,13 @@ pub(crate) async fn scan_vendor_references(project_root: &Path) -> Vec<(String,
72143
let Ok(text) = read_regular_to_string(&project_root.join(file)).await else {
73144
continue;
74145
};
75-
let mut rest = text.as_str();
76-
while let Some(idx) = rest.find(".socket") {
146+
// Every spelling folded first: escapes decoded and every run of `/`
147+
// and `\` collapsed to one `/` (an older composer.lock's `\/`, any
148+
// number of Windows backslashes). Missing one here lets the orphan
149+
// sweep delete a dir a lock still installs from.
150+
let text = socket_patch_core::utils::path_spelling::canonical_path_text(&text);
151+
let mut rest: &str = &text;
152+
while let Some(idx) = find_socket_anchor(rest, fold_case) {
77153
let slice = &rest[idx..];
78154
// `:` ends a reference too: pnpm snapshot keys are
79155
// `name@file:<path>:` and yaml mappings suffix the path with a
@@ -84,18 +160,11 @@ pub(crate) async fn scan_vendor_references(project_root: &Path) -> Vec<(String,
84160
'"', '\'', '`', ' ', '\t', '\n', '\r', ',', ')', ']', '}', ';', ':', '<',
85161
])
86162
.unwrap_or(slice.len());
87-
let candidate = slice[..end].replace('\\', "/");
163+
let candidate = &slice[..end];
88164
// NuGet's feed and Maven's repository name the uuid dir itself.
89-
if let Some(parts) = parse_vendor_reference(&candidate) {
90-
if seen.insert((parts.eco.to_string(), parts.uuid.clone())) {
91-
out.push((
92-
parts.eco.to_string(),
93-
parts.uuid.clone(),
94-
candidate
95-
.trim_start_matches("./")
96-
.trim_end_matches('/')
97-
.to_string(),
98-
));
165+
if let Some((eco, uuid, rel)) = parse_reference_folding(candidate, fold_case) {
166+
if seen.insert((eco.clone(), uuid.clone())) {
167+
out.push((eco, uuid, rel));
99168
}
100169
}
101170
rest = &rest[idx + ".socket".len()..];
@@ -1275,6 +1344,70 @@ mod tests {
12751344
);
12761345
}
12771346

1347+
/// An older composer.lock spells slashes `\/` and a Windows checkout may
1348+
/// double (or quadruple) its backslashes: the scan folds every run to the
1349+
/// canonical path, so the orphan sweep keeps a dir such a lock still
1350+
/// installs from. A different uuid's dir is never this one's reference.
1351+
#[tokio::test]
1352+
async fn scan_recovers_escaped_and_doubled_backslash_references() {
1353+
let uuid = "33333333-3333-4333-8333-333333333333";
1354+
let other = "44444444-3333-4333-8333-333333333333";
1355+
let path = format!(".socket/vendor/composer/{uuid}/psr/log@3.0.2");
1356+
for lock in [
1357+
format!("{{\"url\": \"{}\"}}", path.replace('/', "\\/")),
1358+
format!("{{\"url\": \"{}\"}}", path.replace('/', "\\\\")),
1359+
format!("{{\"url\": \"{}\"}}", path.replace('/', "\\\\\\\\")),
1360+
format!("{{\"url\": \"{}\"}}", path.replace('/', "\\\\\\/")),
1361+
format!("{{\"url\": \"./{}\"}}", path.replace('/', "\\/")),
1362+
] {
1363+
let tmp = tempfile::tempdir().unwrap();
1364+
tokio::fs::write(tmp.path().join("composer.lock"), &lock)
1365+
.await
1366+
.unwrap();
1367+
let refs = scan_vendor_references(tmp.path()).await;
1368+
assert_eq!(
1369+
refs,
1370+
vec![("composer".to_string(), uuid.to_string(), path.clone())],
1371+
"{lock}"
1372+
);
1373+
assert!(
1374+
refs.iter().all(|(_, u, _)| u != other),
1375+
"a different uuid's dir is never this one's reference"
1376+
);
1377+
assert_eq!(scan_vendor_references_any_case(tmp.path()).await, refs);
1378+
}
1379+
}
1380+
1381+
/// The orphan sweep's scan matches the anchor in any case, as the revert
1382+
/// keep-checks do, so a re-cased reference keeps the dir (eco and uuid
1383+
/// reported canonical, the leaf as spelled). The refusal-side scan folds
1384+
/// case only where the disk does.
1385+
#[tokio::test]
1386+
async fn any_case_scan_recovers_a_recased_reference() {
1387+
let uuid = "55555555-3333-4333-8333-333333333333";
1388+
let tmp = tempfile::tempdir().unwrap();
1389+
let lock = format!(
1390+
"{{\"url\": \"{}\"}}",
1391+
format!(
1392+
".Socket\\Vendor\\Composer\\{}\\Psr\\Log@3.0.2",
1393+
uuid.to_ascii_uppercase()
1394+
)
1395+
);
1396+
tokio::fs::write(tmp.path().join("composer.lock"), &lock)
1397+
.await
1398+
.unwrap();
1399+
assert_eq!(
1400+
scan_vendor_references_any_case(tmp.path()).await,
1401+
vec![(
1402+
"composer".to_string(),
1403+
uuid.to_string(),
1404+
format!(".socket/vendor/composer/{uuid}/Psr/Log@3.0.2")
1405+
)]
1406+
);
1407+
// No `.socket/` to probe: the refusal-side scan keeps case.
1408+
assert!(scan_vendor_references(tmp.path()).await.is_empty());
1409+
}
1410+
12781411
/// The scanner's false-positive guard: a `.socket` mention that is NOT
12791412
/// a parseable vendored-artifact path (the committed manifest, a
12801413
/// non-uuid path segment) must never be reported as a vendor reference

‎crates/socket-patch-core/src/utils/fs.rs‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,48 @@ pub(crate) async fn is_dir(path: &Path) -> bool {
8888
.unwrap_or(false)
8989
}
9090

91+
/// Whether the file system holding `path` resolves names ASCII
92+
/// case-insensitively, probed by looking `path` up with the case of its
93+
/// last component flipped: the same file answers yes. `false` when that
94+
/// cannot be shown (`path` missing, no ASCII letter in its name, a distinct
95+
/// file under the flipped name).
96+
pub async fn folds_case(path: &Path) -> bool {
97+
let Some(name) = path.file_name().and_then(|n| n.to_str()) else {
98+
return false;
99+
};
100+
let flipped: String = name
101+
.chars()
102+
.map(|c| {
103+
if c.is_ascii_lowercase() {
104+
c.to_ascii_uppercase()
105+
} else {
106+
c.to_ascii_lowercase()
107+
}
108+
})
109+
.collect();
110+
if flipped == name {
111+
return false;
112+
}
113+
let (Ok(a), Ok(b)) = (
114+
tokio::fs::symlink_metadata(path).await,
115+
tokio::fs::symlink_metadata(path.with_file_name(&flipped)).await,
116+
) else {
117+
return false;
118+
};
119+
#[cfg(unix)]
120+
{
121+
use std::os::unix::fs::MetadataExt;
122+
a.dev() == b.dev() && a.ino() == b.ino()
123+
}
124+
#[cfg(not(unix))]
125+
{
126+
// No stable file id on stable Rust here; Windows file systems fold
127+
// case, and a second, distinct entry under the flipped name is
128+
// practically never there.
129+
a.file_type() == b.file_type()
130+
}
131+
}
132+
91133
/// Blocking twin of [`list_dir_entries`] for walkers that run whole on the
92134
/// blocking pool (one hop per crawl instead of one per filesystem call).
93135
///

‎crates/socket-patch-core/src/utils/mod.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ pub mod lock_fragments;
2020
pub mod notice;
2121
pub(crate) mod npm_range;
2222
pub(crate) mod package_manager;
23+
pub mod path_spelling;
2324
pub mod pdm_lock;
2425
pub(crate) mod pep440;
2526
pub mod pipenv;

0 commit comments

Comments
 (0)